On iOS, when the “Versions” text shows up on an album, one can click that text, then click the three dots next to a version, then click “Delete Album.”
If an album doesn’t have multiple versions, no “Versions” text shows up, and thus there is no way to delete the album.