Every piece of software has bugs, I donât mind as long as they fix it fast. This is a long running issue at Roon. They push fixes really late. I hate it how they prioritize things over there. It should be functionality first, features second.
Not sure if it was explicitly- I certainly noticed some pretty funky stuff happening within this general orbit:
Iâve done that for Linux projects in the past, but not software I pay for. I expect a functional delivered product. I do take into account bugs that can surface in uncommon use cases, as everything cannot be foreseen, but this is not one of those cases. I expect Roon Labs to test the basics before release themselves and not rely on customers for that.
In the same way I donât beta test my grocery app.
I see. Maybe it got buried by the too-many âcanât find the featureâ posts ;). And then Mike wrote that he thinks the reported issues were fixed and to open new threads if not - I donât know if anyone did.
And thatâs fine, but then I donât think you complained that people who do donate their own time for EA didnât find it.
I did complain that the bug was not found before release, the testing is for RL to manage, I didnât specify EA specifically.
Is there a bug reporting tool or template for EA or is it just postings within that sub category as described here?
I just expect better from commercial software, Roon is not cheap.
Thereâs a thread for new releases/features in EA and anyone can open a new one if they want
Seems EA is a waste of time if bugs are found but no action is taken on the development side. Screaming in the desertâŠ
I think we can all agree that these kinds of bugs really ought to be detected during testing. But lighten up people, this is new functionality that we have all happily done without up until a few weeks ago. Roon did not break down. Can we please all just patiently wait for the fix and get on with our lives? Getting worked up out of principle when the stakes are so low is bad for your health.
Everything isnât black and white. Many bugs are found and fixed, and you guys would all be worse off without it. Sometimes a bug slips through, maybe too often, but this still doesnât mean itâs a waste of time.
Normally I agree with you @Suedkiez But not this time. If you have a group of beta testers you should pay attention to their bug reporting and not push versions into the market that are flawed. In this case, this specific bug, was detected in EA testing. Itâs not a big thing for me it doesnât work. But it should be a big thing for a developer if feedback is ignored.
Thanks and thatâs OK. However, I still think that âthe whole thing is uselessâ does not follow from âmistakes were madeâ. Itâs just a fact that more bugs would have made it into production without EA.
This is, of course, true. It also happened a while ago with the broken play queue where I posted warnings for days during EA and it was still pushed, which I found quite annoying, too.
This is not very clear to me. The report that was mentioned was a bit unspecific, then Roon said that they believe the mentioned issues were fixed and people should please open a new thread if anything remains, and I donât think this happened.
The original thread was closed because, unfortunately, it was overrun by people who canât read and half the posts were âI donât see the featureâ by people who donât use EA. Arguably, the EA section can be a bit chaotic due to this and generally because of its nature. I and others suggested in Roon Community Site some improvements to wording in the EA section but who knows if it would help.
Sure. But thereâs a difference between actively being ignored and not seeing everything or judging the significance wrong. Mistakes happen and there is no software company in the world that has never pushed an update with a known flaw in it that was commented on by beta testers.
Anyway, in the current case I find it more questionable that this was not found in formal internal testing, if itâs as easy to reproduce as people say. (I donât know if the issue happens every time for everyone and canât test as I am still without my Roon server due to moving apartment). Betas are never more than a safety net to try and find issues that didnât happen in formal testing procedures as you can never expect unpaid volunteers to exhaustively test everything.
They donât need to release the update though, they can fix obvious bugs before doing so, release outside of their normal release cycle.
If there was a more robust bug reporting process, that are graded in severity, this shouldnât happen and reports shouldnât get missed.
Feedback should be given to the submitter of the report.
Right now it seems they just monitor a sub category in the forum with no transparency beyond that.
With the huge fanfare rollout LL received, Iâd expect it to work.
I do think this is the main issue here - bug reports are mixed into feedback threads. This means thereâs both a risk of them getting lost and no audit trail on any actions relating to the bugs. I would favour a specific EA bug reporting mechanism (also one which doesnât require us to document our systems more than once!).
Sure, but they thought (wrongly) that everything important was fixed
Agreed, the process could be more reliable, but on the other hand one canât expect that volunteers routinely follow protocol and provide exact reproductions. Itâs also a big effort and most consumer software companies I am aware of donât have anything like that, either. At Microsoft, you have to pay 250 euros per developer hour to be able to report a bug with tracking, and they refuse this for Beta completely. At Apple thereâs essentially a black hole to report bugs to.
That said, I agree that someone should be monitoring EA (and other forum sections for that matter) and formalize the reports for internal tracking
Having recently been in contact with one of Roonâs Devs and knowing a few others whoâve also recently discussed Roon with the Devs, I can tell you this;
The Roon Dev team is small, but looking to expand.
Also, they test Roon and also Arc on a small selection of devices due to resources.
Having discussed things Iâm now more sympathetic towards Roonâs choice to ask users to assist in EA builds for bug finding.
I guess the way forward is for those who want rock solid stable is to remain on their current build until the new features have been released for a while, then update when the noise dies down. Advice Iâll follow.
My experience of frustration has been with Arc. My discussion with Roon has been mainly surrounding that. Their resource pool for testing all Android devices is small. So they test what appears to be the main devices. Google Pixels, Samsung Galaxyâs (guessing the S range) and iPhones.
Android phone manufacturers can put their own flare on top of the baseline Android OS. My phone was not one Roon tested but it looks like my phones manufacturer tweaked the battery optimisation and Arc wasnât liking it. But on my new iPhone 16 Pro Max, Arc works very well.
Roon on a general purpose computer or NUC or Nucleus, I feel the experience of cross hardware issues is less. I installed Roon on my main computer on Dietpi. It was constantly using a single CPU core at a 100%. For about an hour I was getting rather fed up. I uninstalled Roon. Rebooted, installed Roon and and all was fine.
So, I wonât be complaining about bugs in new features moving forward as Iâll hold off on updates.
I may participate in the EA cycles, time permitting and would aim to fire back detailed reports and send logs accordingly. If a reported bug crept through, sh1t happens and thereâs bigger things in the world to worry about.
Apple and MS rinse their customers IMO, theyâd charge you for whatever they can get away with, again IMO.
I still get occasional automated email status updates on a Google bug I submitted many years ago with one of itâs products. The opensource community, projects like Debian, have robust bug reporting processes too.
Harman and certainly Samsung must have robust internal systems like Jira.
I get that users might not be prepared to submit detailed bug reports, maybe regularly for EA, but that is part of beta testing. Testing and reporting, good and not so good.
Thatâs why I wonât beta test a product I pay for.
Maybe a report tool in the apps, with the ability to upload logs? Like the Kef Connect app.
This is all doable for Roon Labs now.
I donât see this with Apple, TBF. At work we would be willing to pay for it, but they donât have any visible tracking at all, paid or not.
True, but none of this means that they necessarily get fixed. You wouldnât still be getting Google updates for something you reported years ago if they had fixed it.
And 99% of Roon users would be completely overwhelmed by Debianâs Bugzilla or Ubuntuâs Launchpad and would either not be able or willing to use them or flood them with inactionable stuff. Thereâs also literally tens of thousands of unfixed bugs there.
Itâs everyoneâs choice
And yet for years the same KEF wifi bugs are being discussed here on the forum. Good reporting is important but donât replace the real problem of developer & QA resources.
Of course, you need the resources to fix the issues, thatâs a given.
And Kef are also guilty of releasing the worst kind of listening sabotaging updates, Iâll never buy their equipment again.
Itâs the method of reporting bugs I am referring to, the process does not have to be difficult.
My point is that itâs not as easy as some in this thread claim. âJust do Xâ is not a solution.
Either the process is simple, then it needs resources to sift through and reproduce informal reports, make them actionable, and track them properly. And yes, I think there can be some amount of expectation that this happens if there is an EA at all, but mistakes will make it through.
Or the process is more formal (like in Debian etc.), then it excludes and alienates the vast majority of Roon users (and still doesnât guarantee fixes).