You have two types of ultrasonic content appearing in DAC outputs. Modulator noise which appears on SDM converter outputs and it is static (should be) regardless of your signal. Similar output appears at much much higher level on output of any class-D amplifier. This is uncorrelated noise, sounding like a tape hiss. So it is not related to the music content. It is there same way even if you listen to a track containing silence. On a generally good DSD DAC, the level is not higher than inherent audio-band noise floor of RedBook (44.1/16) PCM. On better ones it is much lower (beyond what you can find as inherent audio band noise floor on best hires recordings). Whether these exist on audio output of your system depends on what you use for listening. Generally none of the headphones or speakers can reproduce frequencies beyond 100 kHz at sufficient levels.
Then you have correlated ultrasonic images, that are result of PCM sampling, appearing around multiples of the source sampling rate. Level of these depend on oversampling ratio and attenuation of the oversampling filters. These sound more like harmonic and intermodulation distortion and are fully correlated with the source signal. If you listen to silence, they don’t exist, but immediately when you have some source signal, these appear. Choice of filter affect these.
Intermodulation products of uncorrelated noise sounds like a tape hiss, or radio background noise. While intermodulation products of correlated images/errors sound like distortion.
In addition, choice of filter affects things like transient speed, ringing around the source Nyquist frequency, etc. And choice of modulator affects things like transient speed, reconstruction quality, etc.
Plus, then in the source content you can have errors that can certainly cause listening fatigue. Some of these can be corrected with apodizing filters. HQPlayer as indicator for these kind of errors. The “Apod” counter that increments during playback whenever such are detected. Most of these errors live in 16 - 22.05/24 kHz region. This is provided to as guidance for selecting suitable oversampling filter.
In addition, ESS chips don’t have DSD Direct path (to bypass on-chip DSP), so any input, PCM or DSD, will always go through the ESS’ on-chip DSP. Just different path depending on PCM or DSD input.
Main difference I notice in your settings is minimum-phase vs linear-phase filter choice. CLANS modulator creates some amount of correlated ultrasonic “whisper”, but that shouldn’t be the issue here. So I would suggest first going to check poly-sinc-short-mp-2s for comparison. And for more similar “softer” Roon like character, you can try ASDM7 or ASDM5.
Those apply to PCM inputs. For DSD inputs, it has choice of (gentle) 50/60/70 kHz filter.