You are here

SOS Product Reviews

For feedback and suggestions about the SOS magazine, app, web site or forums.

Re: SOS Product Reviews

Postby Watchmaker » Fri Sep 18, 2020 2:00 am

I like the use of stars from people who have bought and used the gear. Theoretically, in aggregate, well made, highly functional gear will skew towards more stars, IF the user base is large enough AND the advertising cretins (sales) don't game the system with bots and other lying, cheating, low-down, no-good, snake-eyed deeds of ne'er do wellery designed to goose their ratings.
User avatar
Watchmaker
Frequent Poster
Posts: 867
Joined: Wed Apr 13, 2016 12:00 am
Location: Upstate NY, USA
Take my advice, I'm not using it.

Re: SOS Product Reviews

Postby Folderol » Fri Sep 18, 2020 7:45 am

That still fails. What if the buyer actually bought the wrong type of kit for what they wanted?
They'd give it a 1 star rating quite undeservedly.
User avatar
Folderol
Jedi Poster
Posts: 11674
Joined: Sat Nov 15, 2008 1:00 am
Location: The Mudway Towns, UK
Yes. I am that Linux nut.
Onwards and... err... sideways!

Re: SOS Product Reviews

Postby Mike Senior » Fri Sep 18, 2020 7:51 am

I'm a bit late to this conversation, but I hear this accusation all the time, and (as in a lot of cases) the best rebuttal in my opinion is just some examples of reviews that categorically bite the hand that feeds. I see Gordon's Kyra review and the Slate VMS review have both been mentioned, but there are plenty more. For instance, here's one I wrote that brought the ire of various distributors, but which SOS printed without any 'editorial softening' at all and stood behind staunchly following publication:
https://www.soundonsound.com/reviews/au ... sound-cube
"I don’t reckon people with more expensive main monitors will be particularly well served by the new Auratone as a supplementary mid-range balancing tool — at least not by comparison with its most obvious competitor."

Plus, there have been numerous other reviews where I've given aspects of a product a bit of a kicking. A small selection:

https://www.soundonsound.com/reviews/izotope-neutron
"...as for Track Assistant: at best, I found it scarcely more useful than static presets; at worst, it felt like a time-wasting red herring."

https://www.soundonsound.com/reviews/m- ... iom-pro-49
"It's probably not a tremendously good sign that, after some days spent gamely trying to control Cubase as much as possible exclusively via Hyper Control for the purposes of this review, I heaved a sigh of relief when I finally abandoned it entirely."

https://www.soundonsound.com/reviews/sa ... lv-a6-120a
"Lack of definition in both frequency and time domains makes balance and EQ judgements too unpredictable for serious mixing work."

And that's just stuff I've written! Ah, how fondly I recall subbing Hugh's reviews of the Roland modular V-Mixing system or M-Audio's SP5 back in the day... ("Summary: A two-way active loudspeaker which sounds rather small and boxy.")

Now, irrespective of whether you agree with any of the sentiments in those reviews, what they clearly demonstrate is that SOS has consistently been willing to publish and stand behind critical comments in the mag, even when those critical comments have seriously damaged product sales for the magazine's biggest commercial sponsors.

And that's something that's tough to say about any of SOS's mainstream competitors.
User avatar
Mike Senior
Frequent Poster
Posts: 1071
Joined: Fri Aug 08, 2003 12:00 am
Location: Cambridge, UK
Mixing Secrets for the Small Studio | Recording Secrets for the Small Studio

Re: SOS Product Reviews

Postby ef37a » Fri Sep 18, 2020 8:36 am

Hugh, yes, I know the bandwidth of a pre amp is not related to its RFi rejection (although why anyone thinks they need 'flat to 200kHz' I am at a loss to understand!)

My point was really "different Strokes" In MY case, if the rest of that pre amp's properties fitted my purpose, the half a dB loss at 20kHz is something 'I' could live with but yes, I agree, THAT name at THAT price it should be cutting edge in all departments.

A "star" system? FFS no! We have a world gagging for competitive 'ratings'. Look at the bloody farce that has ensued with schools?

IMHO Sound on Sound reviews* should be seen as 'shop windows' for products. The vast majority of readers will probably buy less than 1% of one year's review output. I and I am sure most people, read about gear because we LIKE to do so. Never going to own most of it, even those that can afford it. So...next best thing...read an in depth report on how it works, how well and get some idea, once in a very rare time, if a purchase might happen.

For sure, not all reviews are to MY liking. Things not covered, as I have said before I want tekky stuff others don't but in general I am happy.

*One of the few times I did buy a fairly expensive product (expensive for me, then) was many years ago when I got a Behringer BCA2000 on the strength of an SOS report.
Disaster! Took 3 iterations to get an even passing worker. I will say ONCE working the device was very good, in fact the concept was brilliant, the electronics very quiet and there has been nothing since that comes close to its facilities at any price AFAICT.

But, reliability was ***t and the drivers worse. Once out of warranty I had to replace one of the mic pre chips then a month later the other channel went down! In the loft now.

Do I blame SOS for their review? No, it did not say the unit was "incredibly brilliant" just very useful. They were not to know the horrors awaiting me!

Can I suggest, 'extended field tests'? These used to be common in Studio Sound where a person would take a product related to their job and use it in the field for several months then report back. (the SD Mix Pres come to mind?)
I know a few such tests have been published and I am sure I am going to be told they are "tricky to fit in"...Just a thought?

Dave.
ef37a
Jedi Poster
Posts: 12276
Joined: Mon May 29, 2006 12:00 am
Location: northampton uk

Re: SOS Product Reviews

Postby CS70 » Fri Sep 18, 2020 9:03 am

Watchmaker wrote:Theoretically, in aggregate, well made, highly functional gear will skew towards more stars

Er.. have you visited the internet lately? 90% of it goes like this

- Person has a nebulous idea of what he wants to do and none at all about how it is done
- Person buys gear/software
- Person lacks the competence to put a plug in the socket and complains publicly gear/software is hard to use and it should be more intuitive
- Person searches for videos explaining how to put a plug in a socket
- Person spends three hours watching cat videos
- Person loudly complains it takes too much time to find information
- Person finds video on plug and sockets
- Video is about how you absolutely need to have a cable made of gold to make the plug/socket work well
- Person tries another video
- Video is made by a guy who couldn't put a plug in a socket and complains that the gear/software is crap
- Person spends another three hours watching cat videos
- Person tries another video on plugs and sockets
- Video made by a guy that has put one plug in one socket and has decided to write a book about it
- Person manages to put plug in socket (alternatively: he drops out and decides the gear is crap)
- Gear requires minimal configuration
- Person has no idea that gear requires minimal configuration
- Gear does not work
(cycle)
- Person complains that the gear/software is crap on Facebook/GS/Reddit/other forum.
- If there are stars to assign, person takes pride in giving a score of 1 out of 5
- Other people read Facebook/GS/Reddit/other forum and the star scoring.
- Other people, without having ever been in the vicinity of gear/software, agree gear/software is crap and complain on Facebook/GS/Reddit/other forum.
- (optional discussion on how everything is crappy nowadays ensue)
- Person makes a video about gear/software
(repeat)

:mrgreen:
User avatar
CS70
Jedi Poster
Posts: 6492
Joined: Mon Nov 26, 2012 1:00 am
Location: Oslo, Norway
Silver Spoon - Check out our latest video and the FB page

Re: SOS Product Reviews

Postby shufflebeat » Fri Sep 18, 2020 9:18 am

Alvin58 wrote:1) If something as complex as a car or even Universities can be numerically ranked by many media outlets, so could much of the hardware/software reviewed in SOS.

I've had to skim some of the posts due to the brevity of the human lifespan so apologies if this point has been made and I've missed it.

Anyone with recent experience of schools and universities will be very aware that the ranking of a complex system with such a low resolution scale ends up with the system either simplifying greatly to become the thing that will gain ranking points and thereby chopping off any characteristic that doesn't get reflected in the rankings (dumbing down) or developing a persona that chases the ranking but doesn't represent the reality of the system itself (public-facing information officials [professional sales dept.]) which ends up sucking all the resources away from the task at hand.

Both of these are dangerous evolutionary limbs to go very far along.
shufflebeat
Jedi Poster
Posts: 5387
Joined: Sun Dec 09, 2007 1:00 am
Location: Manchester, UK
"Dance, dance. wherever you may be, for I am the Lord of the damp settee..."

Do yourself a favour, wear earplugs at gigs.

Re: SOS Product Reviews

Postby Hugh Robjohns » Fri Sep 18, 2020 10:14 am

We're distracting from the theme of the thread here, but briefly...

ef37a wrote:In MY case, if the rest of that pre amp's properties fitted my purpose, the half a dB loss at 20kHz is something 'I' could live with...

I did point out that the HF loss was all but inaudible, and of course there are revered 'classic' preamps that roll off much more heavily...

...but yes, I agree, THAT name at THAT price it should be cutting edge in all departments

And that's exactly the point. It wasn't what they said it was. If any company makes a big fuss about their product being 'state of the art' and 'designed for maximum transparency and sonic purity' I'm going to make sure it is... and the Neumann V402 most definitely falls down on both of those counts and I've said so in the review. And then there's all the other aspects of poor design... but, sadly, I know people will buy it just because of the badge on the front.

Back on topic...

Can I suggest, 'extended field tests'?

I'm fully in favour if this idea where relevant, but it is quite a challenge to organise as it essentially requires the manufacturer to donate the equipment, which is much less possible now than in years gone by. I've thought about writing follow-up reviews for the gear I've ended up buying over the years, but they would almost all just say, "still working like it did when I bought it!" :lol:
User avatar
Hugh Robjohns
Moderator
Posts: 29048
Joined: Fri Jul 25, 2003 12:00 am
Location: Worcestershire, UK
Technical Editor, Sound On Sound

Re: SOS Product Reviews

Postby blinddrew » Fri Sep 18, 2020 11:27 am

Hugh Robjohns wrote:I'm fully in favour if this idea where relevant, but it is quite a challenge to organise as it essentially requires the manufacturer to donate the equipment, which is much less possible now than in years gone by. I've thought about writing follow-up reviews for the gear I've ended up buying over the years, but they would almost all just say, "still working like it did when I bought it!" :lol:
That would add value though, even if there was just a single column with a 'long term reviewers' update that just said "Yep, still good." or "Nope, turns out to be a turkey."
User avatar
blinddrew
Jedi Poster
Posts: 12064
Joined: Sun Jul 05, 2015 12:00 am
Location: York
Ignore the post count, I have no idea what I'm doing...

Re: SOS Product Reviews

Postby shufflebeat » Fri Sep 18, 2020 1:08 pm

blinddrew wrote:That would add value though, even if there was just a single column with a 'long term reviewers' update that just said "Yep, still good." or "Nope, turns out to be a turkey."

Especially to those of us lurking in the second hand market. Suggestions like, "if the doors have survived the engine's probably good" are very useful in the right circumstances.
shufflebeat
Jedi Poster
Posts: 5387
Joined: Sun Dec 09, 2007 1:00 am
Location: Manchester, UK
"Dance, dance. wherever you may be, for I am the Lord of the damp settee..."

Do yourself a favour, wear earplugs at gigs.

Re: SOS Product Reviews

Postby ef37a » Fri Sep 18, 2020 1:14 pm

Well, I was thinking more Hugh that you 'tend' to test the more 'sedentary' gear? (but I can understand you pulling rank for the Mix Pres! )

I meant more of the gigging gear like PA and pedals and that pedal power block in the current issue?
Such stuff could be traipsed around for six months and a report given on ease of use, annoyances and if bits fell off.

But, yes, I understand that you will tell me you all have enough to do as it is!

Dave.
ef37a
Jedi Poster
Posts: 12276
Joined: Mon May 29, 2006 12:00 am
Location: northampton uk

Re: SOS Product Reviews

Postby desmond » Fri Sep 18, 2020 1:26 pm

ef37a wrote:Such stuff could be traipsed around for six months and a report given on ease of use, annoyances and if bits fell off.

I'm not sure that's exactly practical with review gear - it's not usually *given* to the publication/reviewer, it's usually lent for a defined period of time before it's then returned, or passed on to the next reviewer etc...
User avatar
desmond
Jedi Poster
Posts: 10632
Joined: Tue Jan 10, 2006 1:00 am
mu:zines | music magazine archive | difficultAudio

Re: SOS Product Reviews

Postby ManFromGlass » Fri Sep 18, 2020 1:27 pm

There are 2 places where I find a star system somewhat useful -
Amazon. Mostly because the ratings are from buyers who are using the products in a real world situation. But jeziz, there are some real dim bulbs writing up their experiences so a serious grain of salt is required with all assessments. These reviews are useful because a lot of people write about if the product was as described and If it was a poorly made piece of shit.
One of the music stores here allows star ratings from purchasers. There is also a question In the user review section - do you own this product. These reviews restore my faith in musicians and intelligence. Almost all reviews I’ve read have been articulate and well written almost like mini SOS reviews. I mostly expected “Ya it’s cool, dude” And there are a few of those but a lot of musicians take the time for a good analysis. Much better than so many of the music mag reviews. Perhaps the articulate reviews are a fluke as the public education system here is average.
User avatar
ManFromGlass
Frequent Poster (Level2)
Posts: 3619
Joined: Sun Jul 24, 2011 12:00 am
Location: O Canada

Re: SOS Product Reviews

Postby ef37a » Fri Sep 18, 2020 2:20 pm

desmond wrote:
ef37a wrote:Such stuff could be traipsed around for six months and a report given on ease of use, annoyances and if bits fell off.

I'm not sure that's exactly practical with review gear - it's not usually *given* to the publication/reviewer, it's usually lent for a defined period of time before it's then returned, or passed on to the next reviewer etc...

Practical or nay Des' it was done a lot back in the day.

Dave.
ef37a
Jedi Poster
Posts: 12276
Joined: Mon May 29, 2006 12:00 am
Location: northampton uk

Re: SOS Product Reviews

Postby desmond » Fri Sep 18, 2020 2:29 pm

ef37a wrote:Practical or nay Des' it was done a lot back in the day.

I'm well aware of magazine reviews from the 70s and 80s, various bench tests and road tests, and how common it was back then in the heyday of electronics enthusiasts to whip the lids off and poke around inside and report of components used, construction and so on (because I'm still engaged in re-publishing them)... ;)

40 or 50 years on though, the general environment is rather different...
User avatar
desmond
Jedi Poster
Posts: 10632
Joined: Tue Jan 10, 2006 1:00 am
mu:zines | music magazine archive | difficultAudio

Re: SOS Product Reviews

Postby ef37a » Fri Sep 18, 2020 2:37 pm

desmond wrote:
ef37a wrote:Practical or nay Des' it was done a lot back in the day.

I'm well aware of magazine reviews from the 70s and 80s, various bench tests and road tests, and how common it was back then in the heyday of electronics enthusiasts to whip the lids off and poke around inside and report of components used, construction and so on (because I'm still engaged in re-publishing them)... ;)

40 or 50 years on though, the general environment is rather different...

Ok, well tell us who won't let their miserable little £99.99 mixer/amp/pedal be tested for six months on the road and we will be sure not to buy them!

Dave.
ef37a
Jedi Poster
Posts: 12276
Joined: Mon May 29, 2006 12:00 am
Location: northampton uk

PreviousNext