You are here

Forum Topic Pruning: SOS needs your help

Arrangement, instrumentation, lyric writing, music theory, inspiration… it’s all here.

Re: Forum Topic Pruning: SOS needs your help

Postby John Willett » Wed May 06, 2020 11:31 am

Hugh Robjohns wrote:Think of it as an incentive, John, to post helpful things... :D

My declared post count is currently over 26.6k... :shocked:

7,000 now - back to half of what I originally had. :cry:

But I post helpful things all the time - in areas that I have the expertise. :thumbup:
User avatar
John Willett
Jedi Poster
Posts: 7233
Joined: Tue Mar 07, 2000 12:00 am
Location: Oxfordshire UK
John
Sound-Link ProAudio
Circle Sound Services
Sound-Link are UK Distributors for: Microtech Gefell, ME-Geithain, AETA, HUM, Håkan, Meyer Turtle

Re: Forum Topic Pruning: SOS needs your help

Postby BJG145 » Wed May 06, 2020 11:50 am

Forum Admin wrote:The SOS Forum currently has 616,472 Posts and 57,317 Topics. Regrettably, we need to reduce its Post Count by approximately one-third, circa 200,000...We need to action this initial pruning as soon as possible and propose to cull the oldest Topics/Posts on one or two forums at a time to reduce the massive server overhead

Couldn't you just archive the Word Association thread instead?

:beamup:
User avatar
BJG145
Jedi Poster
Posts: 4597
Joined: Fri Aug 05, 2005 11:00 pm

Re: Forum Topic Pruning: SOS needs your help

Postby fatbenelton » Wed May 06, 2020 12:32 pm

It's a great shame - though I understand the very real reasons. There is a ton of info from the past that may be incredibly useful. Generally, it's probably really niche specialist stuff from the past that may be useful. Eg 99.9% aren;t interested in Atari Notator, Opcode Vision etc etc anymore, but the groovy hipster setting up a retro set up may need info that will be lost forever.

I'm sure you've explored this but was apaid archive considered? A bit like specialist archives in libraries, we could perhaps have 2/3 years on the 'public' forum with a 'Pro SoundonSound Archive' that is the Alexandria LIbrary of Music Tech wisdom. Accessible for day visits for those doing some arcane research (£2/day?) or annual subs - £10/year? Perhaps it can be included with digital subscriptions etc.
User avatar
fatbenelton
Frequent Poster
Posts: 621
Joined: Wed Aug 21, 2002 11:00 pm
Location: Liverpool, UK

Jonny


Re: Forum Topic Pruning: SOS needs your help

Postby CS70 » Wed May 06, 2020 1:53 pm

Without any intention of be a know it all (because of course I am not privy with the details): from a purely technical standpoint, have you considered checking the technical details with someone?

Because 616.472 posts and 57.317 topics is not that much amount of data at all. Even assuming a hefty 100K for each post (which I seriously doubt), the grand total would be.. 61 Gb. The forum does not host memory consuming data such as images or video, so that would hundred kilobytes of basically only text - an enormous amount of writing.. Even considering UTF-16, so maybe 2 bytes per character, 50K of text is perhaps the equivalent of one SOS issue's worth of text - for every single post . So the actual db is likely much smaller. Or did u mean 616 million posts?

It's actually (with no disrespect) laughably small in data bases terms.

Even ten times that amount shouldn't impact performance in absolutely anything, so long things are designed in a minimally ok way.

The main bottleneck to data processing performance nowadays is generally network bandwidth between collaborating machines... but then we are talking petabytes of data - hundreds of thousands of millions etc of SOS forums - with concurrent access from hundreds of thousands of users.

The cost of a small review (perhaps paid only if the review yield actionable results) could well be much lower than pruning the forum and creating and maintaining a proper auto-prune mechanism.

As a forum user, anything that works for you guys works for me - SOS forum is one of the best free resources of the internet so you're always right. So let's just say I have put on here my old hat of consultant - for once, at no cost ;-)
User avatar
CS70
Jedi Poster
Posts: 6243
Joined: Mon Nov 26, 2012 12:00 am
Location: Oslo, Norway
Silver Spoon - Check out our latest video and the FB page

Re: Forum Topic Pruning: SOS needs your help

Postby ConcertinaChap » Wed May 06, 2020 10:05 pm

Of course there's the Internet Wayback Machine as a possible way to retrieve a page if you know what you're looking for.

BJG145 wrote:Couldn't you just archive the Word Association thread instead?

Fair point.

CC
User avatar
ConcertinaChap
Jedi Poster
Posts: 9127
Joined: Tue Jul 19, 2005 11:00 pm
Location: Bradford on Avon
Making music: Eagle Alley, recording music: Mr Punch's Studio
A full wine rack is a happy wine rack.
 

Re: Forum Topic Pruning: SOS needs your help

Postby Martin Walker » Thu May 07, 2020 12:35 pm

ConcertinaChap wrote:
BJG145 wrote:Couldn't you just archive the Word Association thread instead?

Fair point.

I've already had a request from a regular forum user not to prune that one :headbang:


Martin
User avatar
Martin Walker
Moderator
Posts: 16418
Joined: Wed Jan 13, 2010 8:44 am
Location: Cornwall, UK

Re: Forum Topic Pruning: SOS needs your help

Postby Forum Admin » Thu May 07, 2020 1:02 pm

ConcertinaChap wrote:Of course there's the Internet Wayback Machine as a possible way to retrieve a page if you know what you're looking for.

Just checked and indeed the superb Wayback Machine does appear to have archived the old website's Forum posts, as this example demonstrates:

https://web.archive.org/web/20150403153000/http://www.soundonsound.com/forum/showflat.php?Cat=&Number=516806&page=0&view=collapsed&sb=5&o=&fpart=1#516806

I've had a rummage to see if it keeps an archive of the current SOS website's Forum -- and indeed, it does!

https://web.archive.org/web/20200502123830/https://www.soundonsound.com/forum

Wayback obviously crawls our site frequently and grabs "captures" of it. Here's the Wayback link to the existing Forum, so one must assume the Forum posts it has archived will never go away and are frozen in the sands of time?

If so, everyone has access to them as long as Wayback Archive stays around.
User avatar
Forum Admin
Moderator
Posts: 3406
Joined: Mon Jul 21, 2003 10:00 pm
Location: A studio deep in the fenlands of Cambridgeshire, UK
SEARCH 11,561 SOS Reviews/Techniques articles: www.soundonsound.com/search

Re: Forum Topic Pruning: SOS needs your help

Postby Forum Admin » Thu May 07, 2020 1:10 pm

BJG145 wrote:Couldn't you just archive the Word Association thread instead?

I have made the WA topic a 'sticky' so it will not get pruned. However, at some point I will have to go back in and split it up into a few locked read-only topics to reduce the burden on the server of its current 1,168 pages. But that can wait a while yet...
User avatar
Forum Admin
Moderator
Posts: 3406
Joined: Mon Jul 21, 2003 10:00 pm
Location: A studio deep in the fenlands of Cambridgeshire, UK
SEARCH 11,561 SOS Reviews/Techniques articles: www.soundonsound.com/search

Re: Forum Topic Pruning: SOS needs your help

Postby John Willett » Thu May 07, 2020 6:36 pm

Martin Walker wrote:
ConcertinaChap wrote:
BJG145 wrote:Couldn't you just archive the Word Association thread instead?

Fair point.

I've already had a request from a regular forum user not to prune that one :headbang:


Martin

Maybe we could just delete the existing thread and start a new Word Association Thread with the last post of the old thread as the first of the new one.

How many people read old posts in that thread? It is just making it go forward that we want. :mrgreen:
User avatar
John Willett
Jedi Poster
Posts: 7233
Joined: Tue Mar 07, 2000 12:00 am
Location: Oxfordshire UK
John
Sound-Link ProAudio
Circle Sound Services
Sound-Link are UK Distributors for: Microtech Gefell, ME-Geithain, AETA, HUM, Håkan, Meyer Turtle

Re: Forum Topic Pruning: SOS needs your help

Postby Sam Spoons » Thu May 07, 2020 8:35 pm

I'd agree with that.
User avatar
Sam Spoons
Jedi Poster
Posts: 13615
Joined: Thu Jan 23, 2003 12:00 am
Location: Manchester UK
Finally taking this recording lark seriously (and recording my Gypsy Jazz CD)........

Re: Forum Topic Pruning: SOS needs your help

Postby BJG145 » Thu May 07, 2020 8:53 pm

Forum Admin wrote:I've had a rummage to see if it keeps an archive of the current SOS website's Forum -- and indeed, it does!

https://web.archive.org/web/20200502123830/https://www.soundonsound.com/forum

Maybe...although if you start clicking through to topics, pages and threads, the links seem to break.
User avatar
BJG145
Jedi Poster
Posts: 4597
Joined: Fri Aug 05, 2005 11:00 pm

Re: Forum Topic Pruning: SOS needs your help

Postby jimjazzdad » Sun May 10, 2020 2:15 am

Probably not helpful in the current context, but how the heck does Gearslutz - whoops! I mean the site that must not be named - manage it? According to their site, they have 1,050,284 threads with 12,579,207 posts. And I have never noticed any 'pruning'.
User avatar
jimjazzdad
Regular
Posts: 99
Joined: Sun Dec 15, 2013 12:00 am
Halifax, NS, CANADA

Re: Forum Topic Pruning: SOS needs your help

Postby Forum Admin » Sun May 10, 2020 5:20 am

jimjazzdad wrote:Probably not helpful in the current context, but how the heck does Gearslutz - whoops! I mean the site that must not be named - manage it? According to their site, they have 1,050,284 threads with 12,579,207 posts. And I have never noticed any 'pruning'.

Different beasts -- GS is purely a forum. SOS is an entire, very large site with many strings to its bow.
User avatar
Forum Admin
Moderator
Posts: 3406
Joined: Mon Jul 21, 2003 10:00 pm
Location: A studio deep in the fenlands of Cambridgeshire, UK
SEARCH 11,561 SOS Reviews/Techniques articles: www.soundonsound.com/search

Re: Forum Topic Pruning: SOS needs your help

Postby John Egan » Sun May 10, 2020 10:26 am

You guys know what SOS can afford to store, but it seems to me that the topics below are the "SOS Core" topics :

Recording: Gear & Techniques
Music Theory, Songwriting & Composition
Mixing, Mastering & Post Production
Live Sound & Performance
SOS Support Forum

so my suggestion would be to prune these as lightly as possible, commensurate with the need to reduce the storage burden. Obviously other topics would then be subject to greater pruning.

Regards, John
User avatar
John Egan
Regular
Posts: 422
Joined: Wed Nov 27, 2002 12:00 am
Location: Staffordshire, England

Re: Forum Topic Pruning: SOS needs your help

Postby ManFromGlass » Sun May 10, 2020 11:42 am

OK I take back my post, Martin. Prune away at WA. I’ll count on the SOS hive mind to continue with witty and some downright funny associations!
Am I asking too much??
:headbang:
User avatar
ManFromGlass
Frequent Poster (Level2)
Posts: 3451
Joined: Sat Jul 23, 2011 11:00 pm
Location: In Canada, eh?

PreviousNext