DSD 512 upsampling is a bit of the chicken and the egg. It depends on which filters you want to engage. Since you mention CUDA, I am assuming you are going to be using HQPlayer. Are you going to be using Roon and HQPlayer on the same machine? If so, then you need to plan for the resources to run both at once.
Also, CUDA is not a program that is run on any graphics card. CUDA is a processing language/module built into NVidia graphics cards. I do not believe that there has been an iMac with an Nvidia card made since 2013? Either way, any internal graphics card is never going to be as good as a real discrete NVidia card for compute processing.
Other points, that memory is more than enough, I would not use a Fusion Drive, a real SSD is the way to go. The quad core CPU depends upon again which filters you want to engage. And finally, Mac OS and DSD512 are not friends. If you use a MAC then you will have to use a different endpoint to feed the DAC and the Linux endpoints work on an individual DAC basis.
IMHO, high end DSD upsampling is squarely in the realm of a DIY Windows computer. The highest processing non - 2s filters need a spectacular amount of processing power: See this post from the author of HQPlayer (and I’d suggest you read the whole thread actually):