HQPlayer is one of those things worth trying. I’d download desktop onto your machine there and get started. Play with the modulator and filter combinations. You may find one that answers all your questions. Note, DSD512 is fairly CPU intensive. It’s OK to run DSD128 or DSD256 to give yourself a solid taste for what it can do.
Others may have specific suggestions based on your DAC which I’m not familiar with.
Since it is based on ES9039, if you have enough processing power, I would recommend to try running it at DSD512, with the default filters in HQPlayer for starters at least, and you could try ASDM5EC-ul modulator. If this works, you could also try -light and -super (and x512 fs) versions of those modulators. This takes quite a bit of processing power, so depends on how powerful your PC is, and whether it has an Nvidia GPU for offloading some of the processing. If DSD512 doesn’t work, I would try with DSD256.
Once this works, you can also switch to ASDM7EC version of the same modulator (like ASDM7EC-ul) for example and compare. Once you know which modulator you prefer, you can look into the oversampling filters. In the PDF manual, there’s a filter table and some rough suggestions (by no means definitive) which ones may be good fit for your source material.
Please also remember not to turn HQPlayer volume higher than -3 dB, this leaves (usually) enough headroom for inter-sample overs that commonly happen with modern source material. Otherwise there is risk of triggering the limiter, which is best avoided. If limiter is triggered, Limited-counter increments and the volume knob turns red.
Yes, those default filters are the best general purpose filters I could come up with, thus the defaults. Those work very well in a balanced way for wide variety of source content.
