Background Audio Analysis Seems Stuck (Linux 1.2 build 128, largeish library)[Solved]

Let’s drop a note for @mike and @vova to take a look, I’m fresh out of ideas.

1 Like

My last report above is of observations made after restarting, waiting for initial watched-directory scans to finish, and waiting for the usual re-adding of a subset of the unchanged library to complete. It just didn’t seem necessary to recount that part again.

Hi Jeffrey,
I just pulled down that .mp3 file from your link and tried it on my Roon system … seems ok.
Though I am running a slighter later Roon build that you.
This is what the waveform looked like.

I have had on occasion some issues with certain FLAC files, a quick recode with dBpoweramp has typically fixed them.

Once you removed the cycosmos.mp3 file did the Audio analysis complete?

It’s encouraging that your Roon build was able to make it through profiling this file! Are you running the Linux flavor of RoonServer?

Well, no, it moved on to a different file to try to scan next… and then hung there.

My case may be pretty extreme… this is a big, messy library of music collected over years by a number of people in order to make those tracks available in-house for radio DJs to use on-air. The quality and provenance of the tracks is extremely variable. So… manually quarantining files correlating with misbehavior, one by one after observing a scanning halt, seems a hopeless task. I’m hoping for more robustness in the face of troubles.

One datapoint: after filtering out duplicates across restarts, I found 20,794 tracks (of the nearly 250,000 total) which Roon flagged “Failed to extract audio format […] CorruptFile” in logs. But I’m not finding the files at which scanning is stopping in that CorruptFile list (so presumably Roon does flag those CorruptFile files as to be ignored later in that uptime instance).

Of course I don’t know that the reason scanning is stopping is necessarily because of poison files. I don’t really know what’s going on, I’m just guessing wildly and hoping someone with keys to the clue closet will get an idea.

1 Like

So, I had another thought. This Linux install of RoonServer listed as a prerequisite either ffmpeg or avconv. Neither was already installed, so I installed the first one I identified a straightforward-to-install package for - which happened to be avconv, from the libav-tools package.

Is avconv being used in this analysis phase, and is there any chance avconv is more fragile than ffmpeg?

Because I’d be glad to install ffmpeg - either poke around for a PPA, or probably just build the latest version from source. Might that be a worthwhile exercise to see if it’ll help keep Roon from seizing up trying to analyze tracks?

And - an additional question for the Roon technical folk (maybe @brian or someone he designates could comment): are there parts of the analysis chain (the decode of MP3s to PCM, say) which are based on external-to-Roon software libraries which I could perhaps exercise using command-line tools which call those libraries? Because I’d be happy to script blasting through large swaths of the tracks I’m trying to get Roon to ingest, to see if I can catch the available external commands pooing the bed when they encounter possible “poison” tracks. I’d feel more in control than I do now trying to guess what’s going on inside the opaque Roon box from a few messages in a log, and maybe I could provide more useful feedback to the Roon dev team.

Quick answers:

  • Yes, they are used during the analysis phase
  • I don’t think avconv is more fragile than ffmpeg. Most of the code is the same between the two–avconv is a fork of ffmpeg.
  • Running them externally isn’t an apples-to-apples test because Roon still handles all of the demuxing internally–so a lot of file-handling details still happen on our end.
  • There is a known bug where “long” (meaning 10-20minute+) MP3/AAC files cause the decoder to hang during analysis, playback, etc. Only on Linux. This will be fixed in our next release (soon).
  • If when you get stuck, you have an avconv process hanging around forever using 0% of the CPU, that’s probably the bug.
  • If you submit logs to us, in a situation like this, we generally look towards the end for trace that says stuff like “analyzing (file)” or “extracting (file)” to see if there’s a pattern. If we see a pattern, we grab the file and debug here.

Thanks for the really quick and informative reply!

Yeah, avconv is a fork of ffmpeg - but it’s an acrimonious fork, and last I heard the two dev teams didn’t communicate very well; so different bugs might be getting patched on different sides of that divide.

Understood about not really being able to emulate what’s going on during analysis with command-line tools. Just spitballing.

Oh, look what I just found, apparently hanging around since Friday:
root 3157 0.0 0.0 186352 15700 pts/2 S May06 0:00 /usr/bin/avconv -f mp3 -i - -f s16le -acodec pcm_s16le - root 28202 0.0 0.0 186352 16016 pts/2 S May06 0:00 /usr/bin/avconv -f mp3 -i - -f s16le -acodec pcm_s16le -

…and after shutting down RoonServer, removing libav-tools, building the latest ffmpeg, restarting RoonServer, waiting for the initial rescans to die down, then turning background analysis back on, this is now hanging around:
root 32149 0.2 0.0 73076 11952 pts/2 Sl 21:02 0:00 /usr/local/bin/ffmpeg -f mp3 -i - -f s16le -acodec pcm_s16le -

apparently corresponding to this log line:
05/10 21:02:03 Trace: [analysis] analyzing trackid=2944050 url=/mnt/depot-library-readonly/Sounds_and_Effects/Reign.mp3

That sure sounds like what you’re describing. Except that the file is short - under 6 minutes. Here’s a copy, in case it’s useful.

Do you have any use for logs or anything else from me, or is it pretty likely what I’m describing is something you’ve already figured out?

Thanks again…

Woohoo! Build 142 just installed, and the server is starting to find all sorts of tracks it must have gagged on earlier!

Build 142 just installed, and the server is starting to find all sorts of tracks it must have gagged on earlier!

Same here :slight_smile:

All good here! :slight_smile:

And here - well done guys :grinning:

Guys, this is still broken. it doesn’t seem to matter what I have the “Speed” set to, every time I add new music in a watched folder, it’s stuck at Analyzing 0/NM until I change the speed to something else, at which time it proceeds to analyze the newly added songs.

Can you describe your setup @Krutsch? Thanks!

Sure. I have RoonServer running on a Core i5 Mac Mini with 8 GB of RAM. Audio files are on a locally attached F/W 800 drive. The audio files are an iTunes library, but without iTunes synchronization (i.e. I watch the Music folder).

Clients are various: a MacBook Retina w/ 16 GB of RAM running HQP and a Cubox-iPro4 with HQP’s NAA. Everything works fine, other than adding new music.

This has been a problem from the beginning and I’ve learned to go into settings and switch from ‘Fast’ to ‘Normal’ and back to ‘Fast’ to un-stick the process.

I still get the same occasionally. I find restarting the core usually gets the show on the road again when all else fails.

HI,

Analysis is very very slow here. Doesn’t halt, but very slow. Been busy for 4 days now, and it has analyzed some 8k tracks…
I suspect the hdd with MP3’s attached to my MacbookAir (which runs the Roon server) to be analyzed quickly, and the files on the NAS folders (flac, 16 bit to 24 bit hires) taking longer.
But i can’t seem to find a definitive clue about that.

Is there a tool in Roon to tell what it is analyzing at the moment?
If not, That would be a feature request :wink:

Cheers,
Marius

Just wondering, if you have set the audio analysis speed to fast in Roon’s settings. If not that will help.

yes, i have, changed it there and back again to kick it back alive once or twice, but now it is steadily counting. Albeit very slowly.

Cheers,
Marius

A post was split to a new topic: Audio analysis doesn’t appear to be correct