In Artist and Album views, you’ll find a blue ABC toggle bottom left of the screen that does exactly what you want.
On systems with a hardware keyboard, you can also just press the letter keys to filter/sort based on your input.
As for list views – since Roon favors the opposite paradigm, I doubt you’ll see them anytime soon.