There is a perceived phenomenon among online social news communities (e.g. Digg, reddit) that as the popularity of the community increases, the average member becomes baser and the overall quality of discourse decreases. This process of deterioration goes by many names (e.g. Jumping the Shark, Eternal September, Evaporative Cooling), but for the purposes of this post, we’ll drop the metaphors and call it simply Social News Deterioration (SND).
HackerNews has been attempting to combat this phenomenon for the last two months, with the site’s owner, Paul Graham, leading the charge. However, most of the reasons given are anecdotal evidence that the quality of HackerNews comments are degrading. This smacks of the “good ‘ol days” mentality, where people remember the past as more enjoyable than they felt it was at the time.1
It would be helpful to have some objective insight into the evolution of the site. To that end, I have examined ~1.8M comments posted on HackerNews from February 2007 through April 20112 in an effort to answer the question:
Is Social News Deterioration real or is it just the “good ‘ol days” mentality?
In this post, the HN community as a whole is compared against comments posted by Paul Graham (PG).
Hypothesis
If a community were suffering from SND, there are a few symptoms we might observe:
- Anxiety, hostility, and depression would rise.
- Confidence, compassion, and happiness would fall.
- The ratios of anxiety/hostility, hostility/compassion, depression/happiness would rise.
Since each of the negative emotions has an opposite emotion, the last point enables us to measure the general negative/positive trend in each of the three main categories.
Analysis: HackerNews vs. Paul Graham
The graphs below show the change in the emotional impact of the comments belonging to the HackerNews community as a whole (blue) and those only belonging to Paul Graham (red) over the 51 months of data.
Interestingly, the community comments show the opposite trends from those of Paul Graham: the community has become slightly more negative in impact while PG has become more positive.
It’s worth noting that there is a strong correlation between emotional impact and writer sentiment. Thus, if we look at the PG analysis as a measure of his mental state, we can speculate this is due to the prosperity he’s experienced in these last few years. No doubt feeling happier and more confident causes one to make others feel similarly.3
Conclusion
People are very stubborn when it comes to being told they’ve changed. We’re more likely to believe the rest of the world is different. Thus, it’s possible that the shrinking impact gap between the HackerNews community and PG is causing the deterioration– though real– to seem more severe to Paul Graham than it actually is.
So is Social News Deterioration a real phenomenon on HackerNews or is it just an illusion? I think the answer is both.
Thanks
Big thanks to Ronnie Roller for providing me with the database of comments.
Footnotes
- Daniel Gilbert gives several great examples of this in Stumbling on Happiness
. ↩
- Note that HackerNews launched in October 2006, but some of the 2006 data was missing or junked. February 2007 is the earliest date that has meaningful data. ↩
- This is supported by David’s upcoming blog post analyzing several Amazon reviews. ↩
Interesting reading. I’m a new Hacker news reader and find it a better news source than my previous favorite reddit. However, I still visit reddit not so much for the content these days but because my local subreddit hosts several events a week and it can be a great way to meet new people in your area. So the way I’ve used the site has changed, it is inferior to what I was using it for previously but now offers something else.
Hacker News has gone exactly the way slashdot did. It started out as a location where engineers hung out, and was overrun with pseudo-geeks.
The real problem with Hacker News is that Paul Graham is does not tolerate viewpoints that disagree with his own, no matter how carefully argued or documented.
Accounts that disagree with the “Group think” (generally centered around the cult of personality PG has created, and thus centered around his personal opinions) find their accounts unceremoniously terminated on HN, without warning, and without cause.
Thus, over time, the community becomes more homogenous and the nature of disagreements become more shrill. The effect is like the joke about the bitterness of university politics– the disputes are so dramatic because the stakes are so low.
Since hacker news will not tolerate anyone who actually writes code (rather than just “Designs” stuff or just pontificates) or anyone who actually is working on a startup (rather than the palo-alto centered YC echo chamber) it is impossible for there to be any real community.
Before thinking this is an external phenomena, you should look at the censorship. A similar thing happened at slashdot, after they made all the Richard Stallman clones be moderators, anyone who was not a strict adherent to that particular ideology was muzzled.
At least in that case, the community was doing the silencing, even though certain members were given extra power.
At HN, PG (or whomever it is) outright censors people for having mildly controversial, or original opinions of things. For instance, a quick way to get the boot is to argue too persistently against a sacred cow like global warming or TDD (which is hilarious as TDD is sort of a cargo cult phenomena and global warming is easy to disprove– at least to anyone who knows the science. But that’s my point- scientific proof is not the standard, conformity to ideological positions is the standard.)
The real hacker community– that is, the set of people who are hackers– come in quite a variety of types, personalities and perspectives. Get rid of that and naturally the quality will nosedive.