Simple one but a simple A-Z (and maybe 0-9 too?!) like apple use for iTunes / contacts etc… when quickly hunting for an artist.
Anytime you are on an artist or album overview screens - just start typing the first letters of your search and they will appear dynamically.
There has already been discussion of the return of the A-Z bar along the bottom of the browsers, so I think this is coming.
The A-Z is on the list… but as @fusionfive said, just start typing.