I’m using the same approach. On the one hand it’s convenient to not have to fiddle with the groups every time you want to change where you’re listening. On the other hand, my reasoning is also that when playing to multiple zones it usually means I’m moving around the house doing something or other, in which case I’m not actively listening and the reduced quality of Airplay is less of a concern.
There is a long standing feature suggestion to address your question: