Roon Extension Manager v1.0 beta3

In another thread I announced my plans for the Extension Manager:


This thread is all about the beta version that is now available. The purpose is to get user feedback and allow extension developers to convert their extensions to Docker images and get them included again.

Prerequisites

Version 1.0 requires Docker for Linux. You can run it on a Linux PC, on a Raspberry Pi running e.g. DietPi, in a Linux VM or on a NAS that supports Docker. In general, if you already run version 0.x with the Docker based extensions available you should be good to go.

Installation

There are two main methods available for setting up version 1.0, via the setup script or by manually creating a Docker container.

Note: Version 0.x and 1.x cannot run side by side on the same system. If you are running v0.x via the Linux service then it will be disabled by the v1.x setup script. It is possible to switch between the two versions by enabling/disabling services.

Running the setup script

The easiest installation method is by using the setup script.

  • Download the setup script
    wget https://github.com/TheAppgineer/roon-extension-manager/raw/v1.x/rem-setup.sh

  • Set the execute permission for the setup script
    chmod +x rem-setup.sh

  • Run the setup script
    sudo ./rem-setup.sh



Creating a Docker container

Manually creating the Docker container can be useful if you are already running other containers or if you want to use the interface provided by e.g. a NAS.

In such a setup also the updates of the Extension Manager have to be done manually as the standalone image doesn’t support the update action for the manager itself.

Using docker run

The docker run command can be used to create a container from the image, the full command to be use is:

docker run -d --network host --restart unless-stopped --name roon-extension-manager --group-add <docker-gid> -v rem_data:/home/node/.rem/ -v /var/run/docker.sock:/var/run/docker.sock -e "TZ=<timezone>" --log-driver journald theappgineer/roon-extension-manager:v1.x-standalone

Replace <docker-gid> with the output of this command:

stat -c '%g' /var/run/docker.sock

And <timezone> with the local timezone, according to the TZ column in this list.

Using docker-compose

It is also possible to use the Docker compose file.

Change log

As version 1.0 only has to deal with Docker based extensions it focusses on improving the user experience during extension installation and use. The changes made compared to the current 0.11.9 version are:

  • Extension specific installation settings
    • Installation settings are better integrated in install flow
    • Settings of pending installations can be changed
    • Environment variables are taken over from Extension Manager if existing
  • Extension installation
    • Status information about downloading and extracting layers gets reported
  • Extension updates
    • Already up-to-date extensions get detected and reported during update
    • Status information about downloading and extracting layers gets reported
  • Repository
    • Extensions that are not available for the host architecture are hidden
  • Logging
    • Extension Manager link always provides latest logs of installed extensions
    • Extension logs include timestamp for each line
  • Setup script
    • System gets prepared for auto-rip feature of CD Ripper extension
    • Timezone of host system gets detected and taken over by Extension Manager

Extension Snapshots

Extension Snapshots bundles those extensions from the 0.x Extension Repository that are not yet available as a Docker image. It is meant to smoothen the transition to 1.0 and to provide a means to run extensions that are no longer maintained. It is installed from the System category in the Extension Manager. You can start the individual extensions via the Extension Snapshots Settings.



The Roon Extension Generator

To make the transition quick and easy for extension developers a tool has been created that can adapt the extension code, create and publish the Docker image on Docker Hub and integrate the image in the Extension Manager for testing purposes. The tool is called the Roon Extension Generator and its use is described in the README file.

2 Likes

If you still see this, Jan, I made it almost through Extension 1.0 setup when it hung, “waiting for volume creation”.

You’re dealing with a NOOB. Any help appreciated

update: earlier msg said cannot create ROOT dir; denied.

update2: a ROOT directory is already created; your script tries to make a new ROOT directory but it can’t; script stalls (I think)

Hi John,

Can you provide the full output of the setup script? It is unclear to me at what point in the setup the error occurs.

Are you running the script as root, or as a normal user with sudo?

I will see if I can find the error text. I used your command text which I believe was sudo.

Back soonest. Here it is: I started with your three command sets, the last one being sudo…

Roon Extension Manager setup script - version 1.0.0-beta1

Checking for root privileges… [ OK ]
Checking for Docker socket… [ OK ]
mkdir: cannot create directory ‘/root’: Permission denied
Downloading shell script… [ OK ]
Starting service… [ OK ]
Waiting for volume creation… [stalls out here]

As it looks like you are logged in as root, is there any difference if you run the last command without sudo? ./rem-setup.sh

BTW on what device and OS are you trying this?

rpi 3b with rasbian

result w/o sudo: zsh: no such file or directory: ./rem-setup.sh

I haven’t tried Raspbian yet, can you provide a link to the image that you use?

Ok @John_V, I should have a fix for you.

I installed Raspberry Pi OS and could reproduce the problem. The default username is pi so sudo is indeed required.

Problem was in detecting the home directory of the user when executing with sudo, the setup script is updated to fix his. The steps to take to recover:

rm rem-setup.sh
wget https://github.com/TheAppgineer/roon-extension-manager/raw/v1.x/rem-setup.sh
chmod +x rem-setup.sh
sudo ./rem-setup.sh --uninstall
sudo ./rem-setup.sh

The extra uninstall step is to cleanup some files created during the failed install.

Thanks much. Will report back.

Update 1: Install changes worked fine. Through Roon, installed Deep Harmony and enabled, but so far it hasn’t detected my hub. Will it work now, or is there further development work?

Everything went well.

REM-Beta

1 Like

That is strange. Although the Extension Manager changed, the Deep Harmony image that gets installed is still the same. Have you updated Deep Harmony via its own update mechanism to the latest version?

All is well now. Restarting solves about 90% of problems.

1 Like

I updated the first post with a description of Extension Snapshots. It allows you to move over to the 1.0 version of the Extension Manager while still depending on extensions that have not yet moved over to Docker.

With this addition I could move over to 1.0 on my NUC that runs my Roon Core and my “production” extensions. In the screenshot below you see the Extension Manager that manages the Alarm Clock, Entrypoints Gateway and the Extension Snapshots, but it is Extension Snapshots that manages the Web Controller.

1 Like

@Jan_Koudijs
I have already installed Extension Snapshots yesterday and also Web Controller. Everything is running fine. Thanks for your great work.

I run extension manager on my nas via a docker. Anway to do the same for the 1.0 beta @Jan_Koudijs

Hi @CrystalGipsy,

I updated the first post with a description on how to manually create the Docker container. There is now a dedicated image available for this use-case, tagged v1.x-standalone.

1 Like

build fails with the following output (RPi, DietPi):

(6/6) Installing git (2.24.4-r0)
Executing busybox-1.31.1-r9.trigger
Executing ca-certificates-20191127-r2.trigger
OK: 16 MiB in 22 packages
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead

node-hid@0.7.9 install /home/node/node_modules/node-hid
prebuild-install || node-gyp rebuild

prebuild-install WARN install No prebuilt binaries found (target=12.16.3 runtime=node arch=arm libc=musl platform=linux)
gyp ERR! find Python
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON
gyp ERR! find Python checking if “python” can be used
gyp ERR! find Python - “python” is not in PATH or produced an error
gyp ERR! find Python checking if “python2” can be used
gyp ERR! find Python - “python2” is not in PATH or produced an error
gyp ERR! find Python checking if “python3” can be used
gyp ERR! find Python - “python3” is not in PATH or produced an error
gyp ERR! find Python
gyp ERR! find Python **********************************************************
gyp ERR! find Python You need to install the latest version of Python.
gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
gyp ERR! find Python you can try one of the following options:
gyp ERR! find Python - Use the switch --python=“/path/to/pythonexecutable”
gyp ERR! find Python (accepted by both node-gyp and npm)
gyp ERR! find Python - Set the environment variable PYTHON
gyp ERR! find Python - Set the npm configuration variable python:
gyp ERR! find Python npm config set python “/path/to/pythonexecutable”
gyp ERR! find Python For more information consult the documentation at:
gyp ERR! find Python GitHub - nodejs/node-gyp: Node.js native addon build tool
gyp ERR! find Python **********************************************************
gyp ERR! find Python
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Python installation to use
gyp ERR! stack at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
gyp ERR! stack at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
gyp ERR! stack at PythonFinder. (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
gyp ERR! stack at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
gyp ERR! stack at exithandler (child_process.js:310:5)
gyp ERR! stack at ChildProcess.errorhandler (child_process.js:322:5)
gyp ERR! stack at ChildProcess.emit (events.js:310:20)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:273:12)
gyp ERR! stack at onErrorNT (internal/child_process.js:469:16)
gyp ERR! stack at processTicksAndRejections (internal/process/task_queues.js:84:21)
gyp ERR! System Linux 5.10.17-v7l+
gyp ERR! command “/usr/local/bin/node” “/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js” “rebuild”
gyp ERR! cwd /home/node/node_modules/node-hid
gyp ERR! node -v v12.16.3
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok
npm WARN roon-spacenav@1.0.0 No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! node-hid@0.7.9 install: prebuild-install || node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the node-hid@0.7.9 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-03-29T11_20_50_457Z-debug.log
The command ‘/bin/sh -c apk add --no-cache git && npm install && apk del git’ returned a non-zero code: 1

I assume that you try to build a Docker image for your extension from the files generated by the Extension Generator. As your extension uses USB it requires some extra packages that have to be added manually to the Dockerfile (be aware that the generated Dockerfile contains the bare minimum and has to be adjusted by the developer).

For the Powermate extension, that also uses USB, I used the following RUN command:

RUN apk add --no-cache g++ git libusb-dev linux-headers eudev-dev make python && \
    npm install && \
    apk del g++ git make python

You also have to make an addition to the .reg/etc/repository.json file that contains your Extension Repository entry. This device option allows the user to specify the USB device during installation:

            ...
            ],
            "options": {
                "devices": [
                    ":USB Device Path"
                ]
            }

You insert this in the file after the binds array.

If you run into other issues, just let me know.

Thank you, Jan! Yes, i build an extension very similar as roon-powermate (for 3Dconnexion SpaceNavigator) with HID USB communication. It builds fine using “npm install” and runs perfectly. But when i use the build script i get those weird errors. So i don’t think any packages are missing for building the docker image. Thanks for the tip with the RUN scrupt and additional entry, but before i can take care of those things, i first need to find out why the build script fails with those Python errors.