Tick the adaptive output rate box. It is currently grayed in your screenshot but try to tick it.
But you may be correct that the DAC can’t support DSD256x48k rates.
I have the same Mac Mini doing DSD256-5EC with gauss-long. Check out my recent post here in the Desktop thread:
HQPlayer Desktop thread - #10 by dabassgoesboomboom
How is your fan noise with gauss-xla after some 30 minutes ? 
gauss-long is a good compromise for me until I see what the next M1x / M2 Mac can do.