First reason would be conversion from 48/96/192k rate family to 44.1k-base family DSD output. This is not really heavier from calculations point of view, but it more demanding on the memory bandwidth. So the difference depends mostly on memory access speed.
With two stage filters for example 176.4k and 192k are also heavier than 44.1k or 48k because the first (heavier) stage converts by factor of 8 or 16. So for example 48k becomes 768k, while 192k becomes 3072k. So the first stage is four times heavier when the source rate is four times higher. Only then the second stage filter is applied to convert it to the final output rate, which in this case is 5.6 or 6.1 MHz.