Thanks @john , good idea. It double checked that DSP was disabled (convolution too for good measure). When I set “Send stereo/mono content as 7.1” to on, I receive the same error message as above, unable to play anything.
Also just for more colour, when playing without convolution and “Send stereo/mono content as 7.1” disabled - channels 5, 6, 7, 8 are blasted full volume with some kind of noise, ignoring the DSP volume control - hopefully I did not damage my speakers :((( See image below (the RME has a nice level display for all the channels).
Given that it all works fine plugged into Macbook, I’m starting to wonder if this is a ROCK/linux USB audio kernel driver problem?