Let's develop a platform for extension distribution [Repository open]

(Hays Clark) #19

You can access a single file in GitHub via a single HTTPS curl call, thus one could easily CURL the url and then parse the file.

You want to make a registry for all of the Roon Extensions?

(Hays Clark) #20

Out of curiosity, how familiar are you with NodeJS and it’s ecosystem of tools as well as authoring Web applications? Others have mentioned being new to NodeJS, so I am trying to ask in earnest. (I’m not trying to be a jerk.)

Have you looked at NPM’s website and look how other companies just use naming conventions? https://www.npmjs.com/

The other technology which has become critical to distributing modern server environments is Docker. Are you familiar with Docker and/or DockerCompose? https://www.docker.com/

In general, I wouldn’t expect the Roon Community to tackle a problem on the scale of distributing a NodeJS/Javascript application. :slight_smile: Docker is the only solution to date which enables the safe installation of Databases and other low-level utilities without the huge resource hit of VM’s.

It sounds like Roon is hoping to store all of the data for a given extensions in its own API supplied data store. So, NPM should be powerful enough to do all the heavy lifting to install the needed programs for a given Roon Extension. NPM can install other needed like Bower, etc. If NPM is the end goal for RoonLabs, then NPM itself would be the correct distribution system and repository for Roon itself. All Roon would need to specify is that all extension use the same ‘nvm’ command if Roon were to install and them and launch them. Of course, this is easier said than done… (see nombom).

RoonLabs might not have published their Extensions officially yet on NPM because you can no longer delete Project once published. However, they still might assume that NPM will be the official repository for Roon Extensions. But that is just a guess on my part.


(Jan Koudijs) #21

‘All’ is still countable on two hands.

Have you looked at the Roon API and Extensions code?
I always try to adapt my coding style to the “surrounding” code.

A bit about my goal:
I’m not trying to create something that scales to infinity. My goal is to make extensions available to a broader audience, just to fill the gap till Roon comes up with their solution. Maybe it is obsolete soon, maybe it turns into something bigger. Lets say, I just like to take action :slight_smile:

(Hays Clark) #22

Yes, I am very familiar with the current RoonLabs Github pages. Are you referring to the fact they are pointing their package.json files at Git repos?

NPM is just where most NodeJS project modules are hosted. Projects like Yeoman for examples allows for community created generators. Thus, users of Yeoman just search for Yeoman Generators and voila. https://www.npmjs.com/search?q=yeoman+generator

(Jan Koudijs) #23

Things indeed become more straight forward when making use of some npm functionality.

By setting NPM_CONFIG_PREFIX to <location>, extensions can be installed globally to the specified location. The full path of an installed extension is:

<location>/lib/node_modules/<extension name>

Extension installation is carried out by:

npm install -g <git url>

Updates are done by:

npm update -g <extension name>

In the mean time I split my code in the three defined layers and created a 'child node' implementation for the lowest layer, mainly for testing purposes.

Next I want to replace the git / npm commands in my install and update functions with the above alternatives.

When all this is working pretty well I want to put it on GitHub so it can be used by others for integration.

(Jan Koudijs) #24

I have put the first implementation of the layering on GitHub, the current status is the following:

This is the front-end towards the lower layers. The functionality behind the auto-update time hasn’t been implemented yet, I’m working on it right now. There is no state information of the managed extensions in this layer.

This layer wraps npm commands in a Node API. I could not find an existing API for this that handles packages outside the npm repository, Roon extensions currently have to be accessed via a git url. Installed packages and available updates are all queried via standard npm commands.

This is a child node implementation for starting and stopping extensions. It is rather limited in its functionality, it e.g. doesn’t store the state of extensions, preventing an auto restart of running extensions after a reboot.

@spockfish, It would be great if you can workout your idea to implement a ‘runner’ layer based on systemd, I can then also test this on my NUC that is running Arch Linux. I hope you can find time for this, you are very busy with the development of your image and are making great progress!

As a next step maybe we can combine the different pieces of the puzzle and improve the user experience by including the Roon Web Controller in the repository of the Extension Manager and then include the manager in the RoPieee image. That would allow a user to choose between the different user interface extensions and also turn all this into a real community effort!

@Mike_Plugge, @spockfish what do you think about this? Does it sound like a plan?

(Mike Plugge) #25

Just sent you a pull request.

I think this is adding up to be a nice system!

(Jan Koudijs) #26

Thanks @Mike_Plugge, Your Web Controller is in the repository now.

How to update the Extension Manager

A repository update still has to be done manually (will change in next release):

npm update -g roon-extension-repository

After that the Extension Manager has to be restarted for the new extension to show up.

More extensions are in the pipeline…

(Jan Koudijs) #27

Two new extensions are added to the repository:

Logitech Harmony Source Control
Denon/Marantz AV receiver volume control

Thanks @Boris_Pruessmann for committing your extensions!

(Jan Koudijs) #28

I updated the title and the first post of this topic to let it reflect the current status.

(Harry ten Berge) #29

Hi Jan,

Looks great!

I’m not sure when I get the time to play around a bit with my idea on a systemd based runner, but I think it’s still an interesting idea as you can leave out a lot of process management complexity. If we wrap this in an ArchLinux package then bootstrapping it is just a matter of installing it with pacman. How cool would that be :wink:

Ok. Back to searching for time then :wink:

Regards Harry

(Jan Koudijs) #30

Any holidays planned? :wink:

(Jan Koudijs) #31

Hi @spockfish, This is what we need for the runner as well.

Do you know if it catches the case of a core update? After a core update I always have the issue that the node is still running (the process is there) but the pinging has stopped, causing the extension to no longer appear in the extension list.

@danny, is there a possibility to set a callback on the pinging mechanism, or should an extension survive a core update?

(Harry ten Berge) #32

Hi @Jan_Koudijs,

I’m not sure if this catches the core update situation. What it at least does it make sure that if the extension craps out for some reason (mine did in a few occasions :wink: ) it’s being properly restarted.

I’ve discovered another issue yesterday with the Roon API in general. I was rebooting my router and in that case my extension did not came back. It was running, but it became invisible inside Roon. Looking into this I’ve found that it got a ‘core unpaired’ event, but it did not recover from that.

I’ve ‘fixed’ that with an ugly hack: in that case I’m deliberately exiting the extension which is then being restarted by the watchdog construction and thus re-establishing connection with Roon.

(Jan Koudijs) #33

This sounds similar to the core update case.

An alternative would be to call stopWatchdogMode() from the core_unpaired() callback.

(Harry ten Berge) #34

yeah. but that’s just a ‘nicer way’ then the exit. I’m still not too happy that the extension needs to be restarted in those scenario’s. Can’t help to think ‘there’s a better way’.

(Jan Koudijs) #35

I agree. Let’s wait and see if Danny can show us the better way.

(Mr Fix It ) #36

@Jan_Koudijs @spockfish Harry did you ever integrate Jan’s alarm into the ropieee build?

I’m asking as I am considering killing my win10 core and just running ROCK and of course rock has no means to run node and I don’t want to have to have a pc/Mac running 24/7 just for api running.

ROCK is picking up the ropieee extension just fine by the way, but not the alarm extension that is running on windows core pc

(Jan Koudijs) #37

Can it be that the Windows Firewall is blocking Node from accessing the core that is now running on ROCK?

(Mr Fix It ) #38

All firewalls are off…and the OS X core doesn’t see it either.