There might just not be enough RAM available for Roon and what it tries to do.
If you need help, expect Roon Labs support to have a look at your log files or in general expect more than other users blindly guessing, then please create a thread in the Support category. Make sure to fill out the template and provide as much additional information (logs from the NAS, memory usage graphs, etc.) as possible. See also: