Alvin58 wrote:But let me see if I can be more specific -- the issue becomes this:
- every audio interface is fantastic
- every orchestral software library sounds great
- every reverb module is amazing
Given that, which one should I buy? That's where I would look to SOS for guidance, and that's what I'm not receiving.
I wonder if part of the problem is the way you digest and interpret
our reviews. Your earlier quoting of the pros/cons summary and mention of a star rating system both suggest you want a very direct, simple guide without needing to think about or interpret a nuanced review. Unfortunately, the world isn't that simple.
Most interfaces are
fantastic, and most reverb modules are
amazing -- at any given price point. (I don't use or review sample libraries, so I won't comment on them! ;-) ) And even the most basic budget equipment available today is capable of making release-quality recordings if used skillfully.
There really is very little truly crap gear around these days, and what little there is doesn't often make it into the pages of SOS anyway. After all, when there are a limited number of pages, why waste the space on junk when there's already far too much good stuff to talk about?
Of course, there are small and often quite subtle differences between similar products, different features, different optimisations or compromises each of which will appeal to some users more than others -- but mostly they are all perfectly legitimate differences rather than outright good or bad.
So the crux of the matter is that different people have very different requirements and expectations of their equipment. An absolutely essential feature to one person is often completely irrelevant to another -- and those kinds of differences can't be captured in a simple pro/con summary -- and even less so in a star rating or ranking system. What one person would rate reasonably as 3* might equally justifiably get a 5* from another user with different requirements.
Added to which, those kinds of simplistic ratings detract from the actual review which is where all the real information resides. SOS is famous for printing reviews that are routinely two, three or four times longer than any
of our competitors -- and they're longer because they contain more detailed information and more useful opinion -- something which we all think matters a lot.
Moreover, all of our reviewers and editors are recording musicians, engineers, or producers, all writing about the gear they are interested in and use every day. That matters too.
I'm guessing you're American, and having talked to a lot of Americans over the years I know some struggle with our 'quaint' British ways, and particularly because our criticisms always seem polite and restrained. Some reading between the lines
is usually required to perceive the true strength of opinion... But it is always all in there for those that take the time and trouble to really read and understand what is printed!
And as for 'not biting the hand'... Firstly, while roughly half of the SOS income does indeed come from advertising, the other half comes from newsagent sales and subscriptions from the readers -- and if we didn't have something worthwhile for those readers to enjoy there'd be nowhere for the manufacturers to advertise. So it's actually more important to us to keep the readers happy than the manufacturers. The policy here is that if a review upsets a manufacturer, that's just the way it is: our duty is primarily to the readers.
Obviously we are careful to make sure our facts are correct, and our opinions are valid, and when we have a review which is significantly critical of a product we generally discuss it with the manufacturer before press so they can respond if they want to. Naturally, we try and maintain good relationships with the manufacturers and distributors, but that doesn't mean we don't fall out now and again.
It's also worth noting that we often get to see prototypes or pre-production models specifically so that we can provide constructive feedback -- and often those comments are incorporated into firmware or hardware improvements before the production version is released and subsequently reviewed. I've long since lost count of the number of products I've reviewed or examined that have been improved specifically because of my feedback to the manufacturers. Sometimes that was before I completed a review, and sometimes it was after publishing a critical review! And I've had several review products which have been withdrawn from market after I've found issues and written about them in a critical review!
...what would be even more useful, when they rated a new XYZ, they ended up putting it in a ranking against the previous XYZ's they have reviewed.
I think we generally do that already -- relevant previous-generation products or obvious competitors are usually mentioned and compared as part of the review process. But our view is that a simplistic ranking systems wouldn't be fair to the products or the manufacturers, and wouldn't actually be very helpful to the readers, either.
You can't, for example, say a Neumann KM184 ranks 23 and a Rode TF-5 ranks 35... or 12... it's all utterly meaningless because the 'value' of the mic depends on what you need it to do in each situation... Mics -- like all products -- have a certain unique set of qualities that suite some things better than others. A star rating or ranking list inherently ignores all that. A pros/cons list can highlight the major strengths and weaknesses of a product, but only a full, detailed review can discuss all aspects of a product and place things in contexts that relate to different ways of using it.