So you’re asking for trouble. Not enough memory leads to the system swapping out memory to HD/SSD and things start to become really slow.
So your music files are not on a local file system. Your rescan setting might just get ignored. See also:
Especially:
Because we need to detect whether the storage device is still on and available, we do check on a regular basis that the NAS is still on and available