Signal-To-Noise Ratios Online

Having recently learned that Karl Rove is coming to Miami to give a talk, I decided to attend and, if possible, ask him a question during the Q&A session. However, I confess I do not know a lot about the man, other than that he’s a behind-the-scenes GOP strategist who was involved in a few scandals and subsequently resigned last fall. So I submitted requests at two popular social media websites for help in determining a few possible questions to ask Mr. Rove: Reddit and Ask MetaFilter (”AskMeFi” for short).

The Request

The wording of the two requests was nearly identical. On Reddit though, you can only submit links and their anchor text. With no place for a broader description of my request, I followed the common Reddit convention of posting a comment on the link as soon as I submitted it, so users who saw the story could also see and read my comment. On AskMeFi, however, they provide a longer description section, so I posted almost the exact same text there:

First off, I’m not looking to be a complete d*ck and waste my potential question scolding him for his disgusting politics. I want to ask a legitimate, coherent question, because whether I like it or not he is an important political figure. Ideally, I’m looking for a simple query that will simultaneously be good enough that the Republicans in the crowd don’t immediately demand that I be tased, but also pointed enough to reveal his underhanded tactics, numerous scandals, etc.

Once the comments and suggestions started to roll in, it quickly became apparent which site was giving me a higher signal-to-noise ratio. (In online discussion forums, off-topic posts and spam are regarded as “noise” that interferes with the “signal” of appropriate discussion.)

The Results

On Reddit, I received several helpful suggestions, but a large majority of the comments (including the one with the most upvotes) were pathetically irrelevant. Out of the 93 comments my submission has now, the final score comes to 58 unhelpful comments and 14 helpful ones, with the remainder being essentially neutral. Of course, what constitutes a helpful comment and an unhelpful comment in this context is entirely subjective, but I think I’ve provided a reasonably accurate count.

Over on AskMeFi, however, I got exactly one unhelpful comment, with the remaining 24 comments being pertinent to the discussion in some way. In fact, I actually received quite a few suggestions that were very well thought-out and constructive.

The Analysis

The question now becomes: why did I get awesome results on one site, and pathetic results on the other? I believe the answer is that MetaFilter charges a one-time fee of $5 to actively participate on it’s website. Now, this idea isn’t exactly original, but I believe in can simplify Mr. Kelly’s theory a little.

He states that the main reason MetaFilter is devoid of spammers is that most such people are younger, and without access to PayPal and debit/credit cards. While this may be true, I think the real reason is much simpler: nobody wants to bother with a registration process that takes more than a minute or so.

On Reddit, anybody can create an account in less than 10 seconds. All you need to do is choose a name, a password, solve a CAPTCHA, and you’re done. You don’t even need an email address. And once you’re registered, you can submit links and comments immediately. Thus, there are effectively no real barriers keeping spammers and immature trolls out of the community.

MetaFilter, however, has a lengthier sign-up process. Indeed, you have to wait some time before you’re able to post to the front page or ask a question on AskMeFi. Of course, the nominal membership fee also helps weed out the unwanted trolls and idiots.

In sum, while sites like Digg and Reddit are probably more popular, they simply do not match MetaFilter in the amount of quality content and discussion produced per day. If you’re at all interested in joining a first-class Internet community, I would consider signing up for a MeFi account. It’s completely worth it.