1.3: Linux RoonServer doesn't seem to be seeing music files on a local disk [solved]


(Olivier Verberckmoes) #8

I am running Roon as root.


(Danny Dulai) #9

@Olivier_Verberckmoes – we need more logs… that stacktrace you pasted is actually not a crash or error, it’s just an info trace telling us what code is kicking off the track tags … it looks like an error, but it isnt.


(Danny Dulai) #10

@Olivier_Verberckmoes, @Jeffrey_Moore, @mitch

I’m going to talk tech since clearly you guys have a clue of what you are doing…

in 1.3, we bind storage not to paths, but to filesystem UUID, because it helps prevent issues with moving drives around

to get the filesystem information, including the UUID, we use udevadm or blkid (which requires root).

the commands we execute are:

udevadm info --query all SPEC

where SPEC is the /dev/xxxx from /proc/mounts

and if that fails, we try:

blkid SPEC and parse the output looking for LABEL= and UUID= and load up /sys/block/DEVICE/device/model and /sys/block/DEVICE/device/vendor to get the model/vendor info

all that blkid and sys/block stuff requires root. udevadm does not.

can you possibly run those commands and tell us which work and which dont?

my guess is that because both @mitch and @Olivier_Verberckmoes are using containers + filesystem pools, this is not going to work unless… i forget how the pools work, but is it like a union fs? where its just a mount into another dir in another mountpoint?


(Jeffrey Moore) #11

Setup details:

Server:

OS Release: Ubuntu 12.04.5 LTS

Kernel: Linux bloo 3.13.0-86-generic #130~precise1-Ubuntu SMP Tue Apr 19 19:21:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

CPU:
Intel® Xeon® CPU E3-1270 v3 @ 3.50GHz
stepping : 3
microcode : 0x7

Filesystems (all local):

EXT4 root and SSD for Roon database/logs

/dev/md0            206G  6.0G  189G   4% /
/dev/sdi1           413G  3.4G  389G   1% /roonssd0

ZFS for music files:

ted                 5.5T  256K  5.5T   1% /ted
ted/home            5.5T   44G  5.5T   1% /z/home
ted/musicjbm        6.0T  507G  5.5T   9% /z/musicjbm
ted/musicjbm-local  5.5T  896K  5.5T   1% /z/musicjbm-local
ted/nonitunes       8.9T  3.5T  5.5T  39% /z/nonitunes

jbm@bloo:~$ ls -l /Volumes
total 0
lrwxrwxrwx 1 root root 11 May  8  2016 musicjbm -> /z/musicjbm
lrwxrwxrwx 1 root root 12 May  8  2016 nonitunes -> /z/nonitunes

Seeing log lines in RoonServer_log.txt of the form

02/02 00:54:32 Debug: [broker/filebrowser/volumeattached] trying to load from mount, but failed to get PartitionInfo: /z/nonitunes

for every filesystem… but I don’t know if that’s normal. Doesn’t sound it.

My whole log directory is uploaded as:

jeffrey_moore-var_roon_RoonServer_Logs-20170202.tar.gz


(Danny Dulai) #12

so for @Jeffrey_Moore – i need you to tell me how to get that filesystem’s uuid – udevadm or blkid is clearly not cutting it…

btw, if none of this works by filesystem uuid, then you might want to just use the drive mounted on / and pick your storage by path

make sure you edit the existing storage locations, or else your music will reimport

we noticed we are failing on / too for you – why is that? better than df would be cat /proc/mounts


(Eric Stewart) split this topic #13

A post was merged into an existing topic: SonicTransporter Storage Issues (Fix Coming)


(mitch) #14

@danny I guess my container doesn’t see any block devices. blkid with no args is blank and the man page says it should probe all devices.

These commands are all inside the container.

root@roon:~# blkid
root@roon:~#

udevadm is also an issue. My mount isn’t a /dev mount. Looks like this:

root@roon:~# cat /proc/mounts
tank/subvol-138-disk-2 / zfs rw,noatime,xattr,posixacl 0 0
storage/incoming /storage/incoming zfs rw,noatime,xattr,noacl 0 0
storage /storage/media zfs rw,noatime,xattr,noacl 0 0
storage/media/music /storage/media/music zfs rw,noatime,xattr,noacl 0 0
none /dev tmpfs rw,relatime,size=492k,mode=755 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
proc /proc/sys/net proc rw,nosuid,nodev,noexec,relatime 0 0
proc /proc/sys proc ro,nosuid,nodev,noexec,relatime 0 0
proc /proc/sysrq-trigger proc ro,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs ro,nosuid,nodev,noexec,relatime 0 0
sysfs /sys/devices/virtual/net sysfs rw,relatime 0 0
sysfs /sys/devices/virtual/net sysfs rw,nosuid,nodev,noexec,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
lxcfs /proc/cpuinfo fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
lxcfs /proc/diskstats fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
lxcfs /proc/meminfo fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
lxcfs /proc/stat fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
lxcfs /proc/swaps fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
lxcfs /proc/uptime fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
devpts /dev/lxc/console devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=666 0 0
devpts /dev/lxc/tty1 devpts rw,relatime,gid=5,mode=620,ptmxmode=666 0 0
devpts /dev/lxc/tty2 devpts rw,relatime,gid=5,mode=620,ptmxmode=666 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run tmpfs rw,nosuid,nodev,mode=755 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset,clone_children 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=6593976k,mode=700,uid=1000,gid=1000 0 0
root@roon:~#

Actually, lsblk sees some stuff, but I’m not seeing any labels or uuids.

> root@roon:~# lsblk -fs
> lsblk: dm-0: failed to get device path
> lsblk: dm-1: failed to get device path
> lsblk: dm-2: failed to get device path
> lsblk: dm-3: failed to get device path
> lsblk: dm-4: failed to get device path
> lsblk: dm-5: failed to get device path
> lsblk: dm-6: failed to get device path
> lsblk: dm-7: failed to get device path
> lsblk: dm-8: failed to get device path
> lsblk: dm-9: failed to get device path
> lsblk: dm-10: failed to get device path
> lsblk: dm-11: failed to get device path
> lsblk: dm-12: failed to get device path
> NAME    FSTYPE LABEL UUID MOUNTPOINT
> md0
> |-sdh2
> | `-sdh
> `-sdi2
>   `-sdi
> sdg1
> `-sdg
> sdh1
> `-sdh
> sdi1
> `-sdi
> zd0p1
> `-zd0
> zd16p1
> `-zd16
> zd16p5
> `-zd16
> zd32p1
> `-zd32
> zd48p1
> `-zd48
> zd48p2
> `-zd48
> zd80p1
> `-zd80
> zd80p2
> `-zd80
> zd80p3
> `-zd80
> zd96p1
> `-zd96
> zd96p2
> `-zd96
> zd96p3
> `-zd96
> zd112p1
> `-zd112
> zd112p2
> `-zd112
> zd128
> zd144p1
> `-zd144
> zd160p1
> `-zd160
> zd192p1
> `-zd192
> zd192p2
> `-zd192
> zd192p5
> `-zd192
> zd208p1
> `-zd208

(Olivier Verberckmoes) #15

Similar behaviour here.
In my log, I see also this:
02/02 17:17:16 Debug: [broker/filebrowser/volumeattached] trying to load from mount, but failed to get PartitionInfo: /mnt/vault

/mnt/vault is the mount-point in the host.

I am running Roon in Containerstation from QNAP – which was running fine with release 1.2.


(Patrick Lang) #16

Same here, although I have a SMB/CIFS share mounted to a local path. Here’s my setup:

Ubuntu 16.04

  • persistent mount configured to a fileserver at /mnt/musicarchive on the host
    Roon 1.3 running in a container with /mnt/musicarchive bind mounted to /mount in the container
  • docker run --name RoonServer --net=host --restart=always -d -v roondata:/var/roon -v /mnt/musicarchive:/music roonserver

Here’s what the mount looks like in the container:

//press.coffee.local/MusicArchive on /music type cifs (rw,relatime,vers=1.0,sec=ntlmssp,cache=strict,username=roon,domain=coffee,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.1.2,file_mode=0755,dir_mode=0755,nounix,mapposix,rsize=61440,wsize=16580,actimeo=1)

I see there’s multiple scenarios where people are using remote file-based storage for their collection. SMB, NFS, ZFS, … none of these have a UUID.

Is there any way to go back to the old mapping? I really want my local collection back. Right now all it sees are my tracks on Tidal.

I don’t want to add the SMB path, username, and password to Roon’s database. In that case I would have to fall back to a less secure authentication method and manually roll the passwords.


(Danny Dulai) #17

i think we are thinking we should always expose / on linux, and if we cant find your other stuff, you can always do it by path from /


(Patrick Lang) #18

Can you clarify a bit on what you mean by expose / ? I don’t want all folders scanned, just the ones I specify.

If you mean use the UUID from /, then scan a specified relative path that might work but would mean my database is no longer portable. Today I keep my /var/roon on separate storage so that I can back up data separate from the whole container/VM or run it on another machine. For example, right now I’m running it on an older 3ghz Athlon x3 440, but I may want to move the container (along with the database) to my new Intel core i5-4670S so that I can use the DSP features.

Additionally, containers don’t provide a UUID. blkid returns nothing


(Jeffrey Moore) #19

This is interesting. First off, /proc/mounts isn’t consistently showing me devices:

jbm@bloo:~$ cat /proc/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=16430896k,nr_inodes=4107724,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=3288316k,mode=755 0 0
/dev/disk/by-uuid/79d79fd6-f9e1-4b04-9ae7-7a7f02c88c00 / ext4 rw,noatime,errors=remount-ro,data=ordered 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sdi1 /roonssd0 ext4 rw,noatime,data=ordered 0 0
ted /ted zfs rw,relatime,xattr,noacl 0 0
ted/backups /z/backups zfs rw,relatime,xattr,noacl 0 0
ted/home /z/home zfs rw,relatime,xattr,noacl 0 0
ted/musicjbm /z/musicjbm zfs rw,relatime,xattr,noacl 0 0
ted/musicjbm-local /z/musicjbm-local zfs rw,relatime,xattr,noacl 0 0
ted/nonitunes /z/nonitunes zfs rw,relatime,xattr,noacl 0 0
ted/varlog /z/varlog zfs rw,relatime,xattr,noacl 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0

…the plain old mount command does:

jbm@bloo:~$ mount
/dev/md0 on / type ext4 (rw,noatime,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sdi1 on /roonssd0 type ext4 (rw,noatime)
ted on /ted type zfs (rw)
ted/backups on /z/backups type zfs (rw)
ted/home on /z/home type zfs (rw)
ted/musicjbm on /z/musicjbm type zfs (rw)
ted/musicjbm-local on /z/musicjbm-local type zfs (rw)
ted/nonitunes on /z/nonitunes type zfs (rw)
ted/varlog on /z/varlog type zfs (rw)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)

…although you’ll note it doesn’t show devices for anything from the ZFS pool.

Also interestingly, the version of udevadm I have seems to want different arguments from what you’re expecting:

jbm@bloo:~$ udevadm info --query all /dev/md0
query needs a valid device specified by --path= or --name=
jbm@bloo:~$ udevadm info --query all --path /dev/md0
device path not found
jbm@bloo:~$ udevadm info --query all --name md0
P: /devices/virtual/block/md0
N: md0
L: 100
S: disk/by-id/md-name-bloo:0
S: disk/by-id/md-uuid-0ae75e34:992dd807:7f579021:a559ea85
S: disk/by-label/Root
S: disk/by-uuid/79d79fd6-f9e1-4b04-9ae7-7a7f02c88c00
S: md/0
E: DEVLINKS=/dev/disk/by-id/md-name-bloo:0 /dev/disk/by-id/md-uuid-0ae75e34:992dd807:7f579021:a559ea85 /dev/disk/by-label/Root /dev/disk/by-uuid/79d79fd6-f9e1-4b04-9ae7-7a7f02c88c00 /dev/md/0
E: DEVNAME=/dev/md0
E: DEVPATH=/devices/virtual/block/md0
E: DEVTYPE=disk
E: ID_FS_LABEL=Root
E: ID_FS_LABEL_ENC=Root
E: ID_FS_TYPE=ext4
E: ID_FS_USAGE=filesystem
E: ID_FS_UUID=79d79fd6-f9e1-4b04-9ae7-7a7f02c88c00
E: ID_FS_UUID_ENC=79d79fd6-f9e1-4b04-9ae7-7a7f02c88c00
E: ID_FS_VERSION=1.0
E: MAJOR=9
E: MD_DEVICES=2
E: MD_DEVNAME=0
E: MD_LEVEL=raid1
E: MD_METADATA=1.2
E: MD_NAME=bloo:0
E: MD_UUID=0ae75e34:992dd807:7f579021:a559ea85
E: MINOR=0
E: SUBSYSTEM=block
E: UDEV_LOG=3
E: USEC_INITIALIZED=13049478

lsblk is no good, it only seems to see native filesystem types like EXT*.

I note that blkid with no arguments yields what looks like a useful list:

jbm@bloo:~$ sudo blkid
/dev/sda1: LABEL=“ted” UUID=“3054010665425163387” UUID_SUB=“10771728253838099820” TYPE=“zfs_member”
/dev/sdb1: LABEL=“ted” UUID=“3054010665425163387” UUID_SUB=“1023546073426357262” TYPE=“zfs_member”
/dev/sdc1: UUID=“869f6b6d-e34c-43fd-b8b4-90f80593e368” TYPE=“swap”
/dev/sdc2: UUID=“0ae75e34-992d-d807-7f57-9021a559ea85” UUID_SUB=“bc18e6eb-33b7-6d17-3678-9fa92310932d” LABEL=“bloo:0” TYPE=“linux_raid_member”
/dev/sdd1: LABEL=“ted” UUID=“3054010665425163387” UUID_SUB=“8895064174128488013” TYPE=“zfs_member”
/dev/sde1: LABEL=“ted” UUID=“3054010665425163387” UUID_SUB=“17056913316539801988” TYPE=“zfs_member”
/dev/md0: LABEL=“Root” UUID=“79d79fd6-f9e1-4b04-9ae7-7a7f02c88c00” TYPE=“ext4”
/dev/sdg1: LABEL=“ted” UUID=“3054010665425163387” UUID_SUB=“7184473094157558192” TYPE=“zfs_member”
/dev/sdh1: LABEL=“ted” UUID=“3054010665425163387” UUID_SUB=“12440179508878276264” TYPE=“zfs_member”
/dev/sdf1: UUID=“be8086b1-50cc-443a-9018-1700142f9ac1” TYPE=“swap”
/dev/sdf2: UUID=“0ae75e34-992d-d807-7f57-9021a559ea85” UUID_SUB=“7067d8a1-36a5-50ec-3c3f-d5086b5f57f1” LABEL=“bloo:0” TYPE=“linux_raid_member”
/dev/sdi1: LABEL=“roonssd0” UUID=“1704f492-fd26-4934-86fd-d014debbe689” TYPE=“ext4”

…although it isn’t showing mount points.


(Jeffrey Moore) #20

Yeah, if I could just navigate everywhere starting from from /, that’d make finding all the music possible.

I know you want to get away from paths, but paths work.


(Danny Dulai) #21

no, we like paths too… i just want usb disks to be able to be in different paths and it work properly :slight_smile:

ok, ive made a quick change that would make your /dev/md0 work – build later today.


(Jeffrey Moore) #22

Oh, BTW - if I were to add a chunk of disk which had previously been known as

/Volumes/nonitunes

(redirecting through a symlink for compatibility with a former life when Roon was installed on a Mac) as, instead, the current native path:

/z/nonitunes

…with everything below that point being the same, would Roon recognize the music as the same, such that it would keep its metadata edits and Roon tags and playlist memberships and such, or is a path component part of the personal identity of tracks such that I shouldn’t risk it?


(Danny Dulai) #23
  1. backup your roon db before trying :slight_smile:
  2. if you edit the storage location, it should work – if you add another storage location, it will lose your import dates and much more

(Jeffrey Moore) #24

Cool. Of course, we want software RAID to work in general; in my case, I happen to have no music in a filesystem on /dev/md0, but since the root partition is on that mirror, perhaps Roon needs to be able to navigate through / to get to where the ZFS partitions are mounted.


(Danny Dulai) #25

check your PM – i need more zfs lore… also, @mitch has / on zfs :frowning:


(Jeffrey Moore) #26

Oh, yeah. I have a 1.2 backup from last week.

Is this something the interface will allow once Roon sees the filesystem, or a global search-and-replace in some database files?


(Danny Dulai) #27

the db holds a “storage location id” and a path relative to that root – editing the storage location’s path will keep the id the same, but re-root it. in settings -> storage, do you not see your 1.2 storage locations? thats what i mean for you to “edit”