Hacker News metrics (first rough approach)

[mjg59]

I'm not a huge fan of Hacker News[1]. My impression continues to be that it ends up promoting stories that align with the Silicon Valley narrative of meritocracy, technology will fix everything, regulation is the cancer killing agile startups, and discouraging stories that suggest that the world of technology is, broadly speaking, awful and we should all be ashamed of ourselves.

But as a good data-driven person[2], wouldn't it be nice to have numbers rather than just handwaving? In the absence of a good public dataset, I scraped Hacker Slide to get just over two months of data in the form of hourly snapshots of stories, their age, their score and their position. I then applied a trivial test:

1. If the story is younger than any other story
2. and the story has a higher score than that other story
3. and the story has a worse ranking than that other story
4. and at least one of these two stories is on the front page

then the story is considered to have been penalised.

(note: "penalised" can have several meanings. It may be due to explicit flagging, or it may be due to an automated system deciding that the story is controversial or appears to be supported by a voting ring. There may be other reasons. I haven't attempted to separate them, because for my purposes it doesn't matter. The algorithm is discussed here.)

Now, ideally I'd classify my dataset based on manual analysis and classification of stories, but I'm lazy (see [2]) and so just tried some keyword analysis:

Keyword	Penalised	Unpenalised
Women	13	4
Harass	2	0
Female	5	1
Intel	2	3
x86	3	4
ARM	3	4
Airplane	1	2
Startup	46	26

A few things to note:

1. Lots of stories are penalised. Of the front page stories in my dataset, I count 3240 stories that have some kind of penalty applied, against 2848 that don't. The default seems to be that some kind of detection will kick in.
2. Stories containing keywords that suggest they refer to issues around social justice appear more likely to be penalised than stories that refer to technical matters
3. There are other topics that are also disproportionately likely to be penalised. That's interesting, but not really relevant - I'm not necessarily arguing that social issues are penalised out of an active desire to make them go away, merely that the existing ranking system tends to result in it happening anyway.

This clearly isn't an especially rigorous analysis, and in future I hope to do a better job. But for now the evidence appears consistent with my innate prejudice - the Hacker News ranking algorithm tends to penalise stories that address social issues. An interesting next step would be to attempt to infer whether the reasons for the penalties are similar between different categories of penalised stories[3], but I'm not sure how practical that is with the publicly available data.

(Raw data is here, penalised stories are here, unpenalised stories are here)


[1] Moving to San Francisco has resulted in it making more sense, but really that just makes me even more depressed.
[2] Ha ha like fuck my PhD's in biology
[3] Perhaps stories about startups tend to get penalised because of voter ring detection from people trying to promote their startup, while stories about social issues tend to get penalised because of controversy detection?

Comments

Meritocracy

[(Anonymous)]

What's wrong with meritocracy?
It seems grounded in logic to me. The people who are objectively best at their job are the most qualified to make executive decisions pertaining to it. Therefore the best concrete results will be obtained by promoting those people. It seems very practical and inclusive to me, since everybody gets a fair chance at demonstrating their best work.

Re: Meritocracy

[(Anonymous)]

Leaving aside whether the statistics posted in this article are an issue of meritocracy, the problems with meritocracy itself are quite well documented. Do you want to select for "best" or "would turn out best under optimal conditions"? Meritocracy typically selects for the former, not the latter; it highlights those who shine, and ignores those who have not had an opportunity to shine.

Also see https://news.ycombinator.com/item?id=8534078 for another issue with focusing only on the highlights of an entire community.

Re: Meritocracy

[(Anonymous)]

Well, how could you select the "best under optimal conditions" if they not had "an opportunity to shine" and not be wrong? Since everybody you not selected could turn out later to be the better pick, no? Also who says what that optimal conditions are where a certain person will shine more then others if they are granted there best optimal conditions?

Spining future: if we assume we all have other optimal conditions who would you pick and change conditions for to let the person shine knowing that you may change conditions for otherd so they not can shine either anymore or not more like before?

Serious, the only answer is trying to improve conditions for anybody and look who shines most what givesnyou option 1, Meritocracy.

Re: Meritocracy

[nadyne]

Meritocracy assumes that there is such a thing as "objectively best at their job". "Best at their job" is subjective, and there are many different metrics that could be a part of that. You also make the assertion that "everybody gets a fair chance at demonstrating their best work", which is an interesting one. If we assume that your assertion is true (which I'm frankly not convinced by, and I hope you can provide some citations for this assertion), the next step is critical: how do we know that everyone who has demonstrated their best work is getting evaluated fairly?

Those who have been identified to be best at their job are not necessarily the most qualified to make executive decisions pertaining to that. Bias, conscious or unconscious, is a trap that even the best of us can fall into.

If you want to get into some of the theory around meritocracy, Nature had an article earlier this year that attempts to model a true meritocracy. The online version is here.

It's also worth noting that the people who are most likely to believe in meritocracy are young, upper-class, white men. The people who are least likely to believe in meritocracy are older, lower-class, minorities. I couldn't find the full article online; it's abstract is here.

Re: Meritocracy

[(Anonymous)]

I think nobody assumes meritocracy to be perfect. Its run and executed by humans so for sure its biased, not absolute objective, has flaws, etc. etc. Like with democracy, it may not optimal but other options are neither.

What do you suggest as better alternate?