Done via PM to Vova.
Could someone give me an update please?
It’s been 2 days now and I’m not aware of any real progress.
To reiterate - my setup is not unusual, I followed upgrade instructions to the letter, and I’ve provided support with all they’ve asked for.
I’m afraid my patience is wearing thin.
I just wanted to give you an update on this. We’ve narrowed the problem down to what appears to be an infinite loop in the code that generates the index for searches in 1.3. This was added to make searching with large libraries not painfully fast, but it’s a bunch of new C code and has received relatively less testing due to it’s newness.
I’m currently working on attempting to reproduce the loop here from your logs and database dump, and should be able to solve the problem quite quickly once that’s done. In case I can’t manage that, we’re also planning on adding a new command line flag to generate a file that will let us ‘replay’ the inserts and removals from the index, which should make reproducing this sort of problem much easier in the future.
Thanks for your patience, and I’m sorry my code is causing you so much trouble.
Okay. Let me know if you’d like me to run or test anything. I’m very Mac/Unix savvy (I build and run web servers).
I’m currently running a new install on an old Mac Mini - just so I can listen to my music - but I can switch to the Ubuntu/Brix install whenever.
Are you getting different results with the same library on the Mac Mini? That would be an interesting result to me, although I guess I’m not sure what I’d do with it if so.
I tried to import my old (1.2) database but it hung too initially - so I started from scratch as I just wanted to play some music.
Build 200 of Roon added a command line flag to record everything added or removed to the search index. Can you:
1: Update your Ubuntu RoonServer to build 200. This might require re-downloading from https://roonlabs.com/downloads.html, depending on exactly where it’s hanging
2: Run RoonServer with the -searchindexreplay flag and wait until it gets stuck
3: This will generate a file called “searchindexreplay.txt” in your logs directory. Can you get me that file?
We’re also working through the code, but because we can reproduce the problem we can’t tell whether we’ve fixed it or not.
How do I run RoonServer with a specific flag? It’s not in my path, so I can’t run “roonserver - searchindexreplay”
Should I run the executable with the flag – i.e. “/opt/RoonServer/Mono/bin/RoonServer -searchindexreplay”. ??
Just making sure I get what you want …
There should be a “start.sh” script in “/opt/RoonServer”, so something like “/opt/RoonServer/start.sh -searchindexreplay”
You’ll also need to kill any existing RoonServer/start.sh instances first, “pkill start.sh” or whatever.
Thanks - give me 5 minutes.
Hmm … if I run it that way, from the command line, it asks me to sign into Roon and then asks where I keep my music – as though it were a fresh install.
Should I proceed with this pseudo-fresh install?
No, I made a mistake. One of the funny side effects of always running a dev version of Roon that I’ve just compiled.
New and improved way to run on Linux with command line arguments: Add them to the end of the last line in the script at “/opt/RoonServer/Appliance/RoonAppliance”, then kill start.sh and wait for the restart.
You probably also want to remove them after collecting the new log file.
Search index log file:
And another - after restarting RoonServer a second time. Smaller this time - perhaps it gets stuck indexing in more than one place?
These are great, thanks. I’m still not able to get this to get stuck on my machine, but I did find a bug in our replay code, so there’s some progress.
Build 202 has some re-writes that might fix the your problem here. Could you try it on your Linux system and let us know if it helps?
I upgraded half an hour ago and it seems to have got rid of the spinning logo issue. Also CPU usage is now back to normal (one CPU was constantly at 100% previously).
I’ve tried restarting RoonServer a few times and still no sticking. I’ll monitor it - but that does seem to have done the trick.
What was the issue?
I’m glad it worked, we weren’t quite sure it would
As for what the issue was, we still don’t know exactly, but:
One of the things we did for 1.3 was try to improve performance generally. In particular for this problem, we replaced our very simple search algorithm (just look at each album in order) with a more complicated and much faster one with an index. I wrote the index code in C to hide it from the C# garbage collector (and allow really fine control over memory usage generally). The problem you were experiencing seems to have been that that C code had an infinite loop in the code to add new search terms to the index, which caused Roon to hang with one core spinning 100%. Last week Danny and I spent some time re-writing and cleaning up the index code in an attempt to smoke out the bug, and found a couple places were the search term null-termination was handled inconsistently. One of those seems to have fixed the problem.
I hope that explanation made sense and hit the correct level of detail
It does. Thanks Ben.