Hacker News Ranking Algorithm(vigneshwarar.substack.com)
vigneshwarar.substack.com
Hacker News Ranking Algorithm
https://vigneshwarar.substack.com/p/hackernews-ranking-algorithm-how
81 comments
What would happen is that the old accounts would accumulate "CommentRank" and the new users would always be sorted to the bottom. This is good for search, since we are looking for "the" answer, the site, a limited number of best pages for a topic etc., but it's quite bad for discussion; a bad take from Animats would forever be above a good comment from a greenhorn.
I thought about it while writing this post. It would be useful to have a decay factor for PageRank. In simple words, if someone wants to maintain their PR score, they need to actively engage in the HN community.
> if someone wants to maintain their PR score, they need to actively engage
Wouldn't that encourage people to "actively engage" even when they have nothing to say just to keep their score?
Goodhart's law comes into mind here. People who want to stay relevant on this site will take your measure as their target. There's no system that can't be gamed if you know the rules. But the more "rock solid" you try to make those rules, the more you embed intrinsic discrimination between participants. You put "Proof of Work" in the rules and you alienate newcomers while all the rest do "busywork".
Your own comments and submissions would sink like a stone because you didn't engage enough, regardless of how interesting your points are.
Wouldn't that encourage people to "actively engage" even when they have nothing to say just to keep their score?
Goodhart's law comes into mind here. People who want to stay relevant on this site will take your measure as their target. There's no system that can't be gamed if you know the rules. But the more "rock solid" you try to make those rules, the more you embed intrinsic discrimination between participants. You put "Proof of Work" in the rules and you alienate newcomers while all the rest do "busywork".
Your own comments and submissions would sink like a stone because you didn't engage enough, regardless of how interesting your points are.
I think this part of the post answers your question.
> Will I ever publicly write about how HN ranks posts if I am Dang (HN moderator)? No, because Pagerank can be manipulated by people despite its reputation. In fact, Pagerank is being exploited for years. Moreover, there are financial incentives for companies to get on the first page in HN.
> Will I ever publicly write about how HN ranks posts if I am Dang (HN moderator)? No, because Pagerank can be manipulated by people despite its reputation. In fact, Pagerank is being exploited for years. Moreover, there are financial incentives for companies to get on the first page in HN.
> Will I ever publicly write about how HN ranks posts if I am Dang (HN moderator)? No
Lack of transparency is not a solution, and certainly not one that will sell well in a place where "security by obscurity" is disdained and open source cherished. It's particularly bad when you're dealing with a high profile site filled with intelligent people who would most likely make short work of reverse engineering just enough of the algorithm to understand how it works.
Lack of transparency is not a solution, and certainly not one that will sell well in a place where "security by obscurity" is disdained and open source cherished. It's particularly bad when you're dealing with a high profile site filled with intelligent people who would most likely make short work of reverse engineering just enough of the algorithm to understand how it works.
[deleted]
With 100% certainty dang ("Dang") read this comment.
I would be really happy to know his thoughts.
This: https://www.newyorker.com/news/letter-from-silicon-valley/th... https://archive.ph/V9z3G is as close as you're likely to get.
You can write [email protected] . I’ve chatted with dang on that email address several times.
You can put epicycle around epicycle, but there's a lot of computational efficiency in a score which can be calculated from an integer and a timestamp.
"…would provide weird incentives like frequent comments, or higher volume, could rank a person higher than if they had occasional comments further down the thread."
I know I'll be out of step with most who comment on HN but I question the necessity of having a ranking/voting system at all. All it seems to do is to act as an ego booter for commentators and it messes up orderly discussion when people insist on posting replies to the first commentator.
As a news aggregator, no doubt HN needs some method of ranking stories to set their listing lifetimes and their importance (page ranking) but that could be accomplished by the number of clicks a storey receives and or how long a visitor/commentator stays on a story (or the time between link clicks).
I also question aspects of the voting system as currently implemented. It's not unusual for me to receive down-votes on comments to controversial stores but not receive a comment from the down-voter. Why bother with this system if the voter hasn't the gumption to say why he/she disagrees with one's views? (One could discuss the psychology and social philosophy behind this but here is not the place).
Another annoyingly aspect of the voting system is the lack of stats. The time I take from posting to checking to see if someone has replied varies wildly—from minutes to days to never. This wide variation is significant when I post to controversial stores. Days later I can check back to see if there's been any votes or replies only to find there has been none.
At other times, I've monitored the incoming votes almost in realtime and watched them oscillate around 0 or 1, that is I've received many up and down votes but the long-term average is zero. Come back and check days later and one hasn't a clue from the stats if anyone at all has actually read one's comment let alone the fact that it was equally controversial to both sides of a polarized audience. HN should allow users access to these long-term stats.
I know I'll be out of step with most who comment on HN but I question the necessity of having a ranking/voting system at all. All it seems to do is to act as an ego booter for commentators and it messes up orderly discussion when people insist on posting replies to the first commentator.
As a news aggregator, no doubt HN needs some method of ranking stories to set their listing lifetimes and their importance (page ranking) but that could be accomplished by the number of clicks a storey receives and or how long a visitor/commentator stays on a story (or the time between link clicks).
I also question aspects of the voting system as currently implemented. It's not unusual for me to receive down-votes on comments to controversial stores but not receive a comment from the down-voter. Why bother with this system if the voter hasn't the gumption to say why he/she disagrees with one's views? (One could discuss the psychology and social philosophy behind this but here is not the place).
Another annoyingly aspect of the voting system is the lack of stats. The time I take from posting to checking to see if someone has replied varies wildly—from minutes to days to never. This wide variation is significant when I post to controversial stores. Days later I can check back to see if there's been any votes or replies only to find there has been none.
At other times, I've monitored the incoming votes almost in realtime and watched them oscillate around 0 or 1, that is I've received many up and down votes but the long-term average is zero. Come back and check days later and one hasn't a clue from the stats if anyone at all has actually read one's comment let alone the fact that it was equally controversial to both sides of a polarized audience. HN should allow users access to these long-term stats.
Not having an explicit ranking system means that things are ranked by submission time. This too incentivizes certain behavior (frequent posting). Maybe one can get around that with certain guardrails like rate limiting submissions/comments but that's yet another can of worms to open.
I agree with you mostly (and will give you an upvote). I think we've seen enough to understand that upvotes and downvotes are dolled out based on how well a comment agrees or disagrees with the voters preconceived notion of whatever the topic is.
I also question the concept of 'hiding' comments. Are my eyes so delicate that I can't see certain comments by default? It seems like just a feature that exists to bias subsequent viewers against comments.
It'd be really interesting to see a system where voters have both one upvote/downvote for the submission and the ability to upvote one comment and downvote one comment.
I also question the concept of 'hiding' comments. Are my eyes so delicate that I can't see certain comments by default? It seems like just a feature that exists to bias subsequent viewers against comments.
It'd be really interesting to see a system where voters have both one upvote/downvote for the submission and the ability to upvote one comment and downvote one comment.
I believe hidden comments can be seen if you enable showdead, and if you do so you'll learn why they are hidden (they add virtually nothing to discussions).
Author here. Yes, reading time will be a helpful indicator. In fact, I already mentioned this in the post.
> Reading time for the article.
> Reading time for the article.
I see where you included metrics as a starting point like how long an article takes to read. Maybe like a word count approach as a starting point.
Thinking about understanding the content of an article and seeing who's most likely to be referencing that content in their post is more of what I had in mind.
Thinking about understanding the content of an article and seeing who's most likely to be referencing that content in their post is more of what I had in mind.
I am often faced with a dilemma when trying to decide whether to vote on a particular post or comment. There are times when it is very informative, well written, and you can tell a lot of thought went into it; but I happen to disagree with the author's point of view. The same occurs when I agree with the author's sentiment, but the comment itself is poorly written and not well argued.
There is only a single up/down vote. You can't upvote a thoughtful comment for its quality, while also downvoting its conclusion. Sometimes I have strong opinions about something; but I also try to educate myself with the other side of the issue by reading thoughtful articles and/or comments from people with different points of view.
It would be nice if you could choose between two different 'modes' in the interface. One where the highest quality comments are near the top. The other where the most popular viewpoints are prioritized.
There is only a single up/down vote. You can't upvote a thoughtful comment for its quality, while also downvoting its conclusion. Sometimes I have strong opinions about something; but I also try to educate myself with the other side of the issue by reading thoughtful articles and/or comments from people with different points of view.
It would be nice if you could choose between two different 'modes' in the interface. One where the highest quality comments are near the top. The other where the most popular viewpoints are prioritized.
I think it's probably better to just upvote well-written or well-argued comments whether you agree with the author's point of view or not. Makes for a more interesting place where people occasionally change their minds.
I was going to say that the HN guidelines advise doing this, but I cannot actually find anything about what an upvote 'means' on HN. Plenty about what makes a good submission, or a good comment, but nothing on voting. Maybe they just gave up on trying to do that!
The above notwithstanding, I would like to see something like your proposal implemented, just to see how it works out empirically.
I was going to say that the HN guidelines advise doing this, but I cannot actually find anything about what an upvote 'means' on HN. Plenty about what makes a good submission, or a good comment, but nothing on voting. Maybe they just gave up on trying to do that!
The above notwithstanding, I would like to see something like your proposal implemented, just to see how it works out empirically.
I upvoted your comment on both criteria. It would be very useful to have 2 dimensions. It's amazing nobody does afaik!
@dang, as a Arc expert, let me assure it's only 4 lines of code, 1 field on db, 52 bytes of js, css, and html, and it won't cause performance, security, accessibility, privacy, or usability issues /s
@dang, as a Arc expert, let me assure it's only 4 lines of code, 1 field on db, 52 bytes of js, css, and html, and it won't cause performance, security, accessibility, privacy, or usability issues /s
The LessWrong forum has a split between "upvotes/downvotes" and "agree/disagree votes" to address this.
In practice, it doesn't really seem to affect outcomes that much, but it's a cute feature.
In practice, it doesn't really seem to affect outcomes that much, but it's a cute feature.
The dilemma you face stems from the ambiguous nature of the upvote button:
1. As an "agree" button - you upvote to get the perspective you share to be displayed more prominently to other people. It is a way for you to influence other readers.
2. As "it was useful to me and a good use of my time" button. You use it because you think other users will find it informative as well. In this case you are doing purely for the benefit of others.
3. As "it made me laugh or evoked some other emotions" button.
All three could be viewed as different dimensions of content and one item/comment can have different combinations along these dimensions.
It is complicated by the fact that content creators are actively optimizing their content along these dimensions to gain more visibility.
When I come to HN, as a reader, I want to see mostly informative content - which is only dimension 2. I don't want to be influenced or persuaded. And while I enjoy a good laugh - that's not my goal coming to HN. This makes the other two content dimensions contribute to noise.
Your suggestion to split out the upvote into separate labels like "agree", "informative" and "funny" [1] and to let users sort by the dimension of their choice may not solve the problem of separating "agree" from "informative" because there is no incentive for the voters to vote truthfully. People who upvote content they agree with want other people to see that content. If most users are sorting by informative then those voters will upvote content they agree with using the "informative" button.
I think you need to change the system of incentives in order for voters to upvote only informative content and not 1 and 3.
One way to do this is to make what you upvote have more consequences for the type of content you see in the future and less directly affect the ranking of that content for all users [2]. With a feedback loop like this the upvote button is no longer "what content do I want other people to see", but "what type of content do I want my future self to see". It makes you think before voting - "Was it worth my time?", "Is this kind of content aligned with my long term goals?"
[1] - Slashdot has something like this: "funny", "interesting", "insigthful" https://en.wikipedia.org/wiki/Slashdot#Peer_moderation but I don't think you can sort by it. Also, only a subset of users are given these moderator points.
[2] - I'm building https://linklonk.com that works this way.
1. As an "agree" button - you upvote to get the perspective you share to be displayed more prominently to other people. It is a way for you to influence other readers.
2. As "it was useful to me and a good use of my time" button. You use it because you think other users will find it informative as well. In this case you are doing purely for the benefit of others.
3. As "it made me laugh or evoked some other emotions" button.
All three could be viewed as different dimensions of content and one item/comment can have different combinations along these dimensions.
It is complicated by the fact that content creators are actively optimizing their content along these dimensions to gain more visibility.
When I come to HN, as a reader, I want to see mostly informative content - which is only dimension 2. I don't want to be influenced or persuaded. And while I enjoy a good laugh - that's not my goal coming to HN. This makes the other two content dimensions contribute to noise.
Your suggestion to split out the upvote into separate labels like "agree", "informative" and "funny" [1] and to let users sort by the dimension of their choice may not solve the problem of separating "agree" from "informative" because there is no incentive for the voters to vote truthfully. People who upvote content they agree with want other people to see that content. If most users are sorting by informative then those voters will upvote content they agree with using the "informative" button.
I think you need to change the system of incentives in order for voters to upvote only informative content and not 1 and 3.
One way to do this is to make what you upvote have more consequences for the type of content you see in the future and less directly affect the ranking of that content for all users [2]. With a feedback loop like this the upvote button is no longer "what content do I want other people to see", but "what type of content do I want my future self to see". It makes you think before voting - "Was it worth my time?", "Is this kind of content aligned with my long term goals?"
[1] - Slashdot has something like this: "funny", "interesting", "insigthful" https://en.wikipedia.org/wiki/Slashdot#Peer_moderation but I don't think you can sort by it. Also, only a subset of users are given these moderator points.
[2] - I'm building https://linklonk.com that works this way.
I think the article ranking is probably good enough.
Where I have more trouble with is with comment-ranking within an article. The interface makes it hard to understand conversational flow, and of course that flow changes over time according to whatever the algorithm does. This makes it very hard to catch up later on an article’s comments and get a view of what has changed since my last view. Maybe there is a way to do this via the interface, but if so it’s not obvious.
Where I have more trouble with is with comment-ranking within an article. The interface makes it hard to understand conversational flow, and of course that flow changes over time according to whatever the algorithm does. This makes it very hard to catch up later on an article’s comments and get a view of what has changed since my last view. Maybe there is a way to do this via the interface, but if so it’s not obvious.
> This makes it very hard to catch up later on an article’s comments
My biggest complaint related to this is that all threads are dead within 24 hours, which kills a lot of interesting discussion. And commenters have no incentive to reply to dead threads.
This discourages in depth back and forth conversation which I usually find is the most interesting.
My biggest complaint related to this is that all threads are dead within 24 hours, which kills a lot of interesting discussion. And commenters have no incentive to reply to dead threads.
This discourages in depth back and forth conversation which I usually find is the most interesting.
Admittedly, it varies, but I tend to look back for responses to older discussions if it was an interesting engagement to me. I agree with you that long form discussion suffers from current format, but I think it is part of the deal. It is hacker news and not hacker discussions.
TBH, I am not sure if there is a good non-invite-only forum that facilitates that.
TBH, I am not sure if there is a good non-invite-only forum that facilitates that.
And even if someone replies to you specifically, the only way you would know is noticing your karma increased on the top right of the page.
Someone replying to your comment doesn’t change your karma points. Someone upvoting your comment may probably increase your karma points (the may probably is because of the undisclosed algorithms that attempt to compensate for gaming the system).
In other words, looking at karma won’t tell you whether someone replied to your comments or not. There is nothing on HN to indicate replies unless you dig into your comments history. There nay be other sites or browser extensions that could help on getting notified.
In other words, looking at karma won’t tell you whether someone replied to your comments or not. There is nothing on HN to indicate replies unless you dig into your comments history. There nay be other sites or browser extensions that could help on getting notified.
At one point you could email [email protected] and ask to participate in the new beta, which among other things highlights new comments since you’ve last visited the page.
I don’t know the status of it, but it seemed like an interesting idea.
I don’t know the status of it, but it seemed like an interesting idea.
That should be pretty easy to do in a browser extension. Keep track of the last time the comment section was opened, highlight comments newer than that.
Comments I haven't seen are a different color. But I read HN from a "custom" UI.
> why don't we apply PageRank for every user based on the upvotes they receive for the comments they leave on any post
Absolutely not, think about the incentives this creates! Imagine yourself to be an adversary, how would you game this system? Easy: spam low-effort highly agreeable comments across the site. The more the better, they're basically guaranteed to get some upvotes and boost your karma. Now, with your newfound power, you can shape the HN front page to your will.
Instead of the front page being a signal of quality, it would be correlated to the posters' karma. And because the front page is a limited resource, you basically get a crappier version of Twitter where relatively low-karma accounts have practically no chance to get on the front page even with high-quality posts, and high-karma accounts will be able to roll their front-page posts into even more karma. This is pretty much one of the biggest flaws of major social media websites today: they concentrate power in a few users with massive influence.
Absolutely not, think about the incentives this creates! Imagine yourself to be an adversary, how would you game this system? Easy: spam low-effort highly agreeable comments across the site. The more the better, they're basically guaranteed to get some upvotes and boost your karma. Now, with your newfound power, you can shape the HN front page to your will.
Instead of the front page being a signal of quality, it would be correlated to the posters' karma. And because the front page is a limited resource, you basically get a crappier version of Twitter where relatively low-karma accounts have practically no chance to get on the front page even with high-quality posts, and high-karma accounts will be able to roll their front-page posts into even more karma. This is pretty much one of the biggest flaws of major social media websites today: they concentrate power in a few users with massive influence.
> I can't find the official page on why Paul Graham created HN, but to me, HN is the place to find the most interesting links and have a healthy discussion about various topics. In fact, discussions are my favorite part, except for the useless comments.
Is there an official reason? I’ve wondered this as well and sort of assumed it was because hackers like having some community aspect and are novelty-seeking so YC created this due to the largesse of PG for the YC group and someone said “why not just make it public so applicants and future YCers can participate.” And job postings were added on to generate some revenue so it wasn’t a pure cost center.
This headcanon reenforces my belief that the best community sites are accessories to something else and that you can’t make money from a community without diminishing its value (since the incentives for monetizing conversation are not the same and I’d guess 90% at odds with good conversation- mainly because bad things draw attention).
Is there an official reason? I’ve wondered this as well and sort of assumed it was because hackers like having some community aspect and are novelty-seeking so YC created this due to the largesse of PG for the YC group and someone said “why not just make it public so applicants and future YCers can participate.” And job postings were added on to generate some revenue so it wasn’t a pure cost center.
This headcanon reenforces my belief that the best community sites are accessories to something else and that you can’t make money from a community without diminishing its value (since the incentives for monetizing conversation are not the same and I’d guess 90% at odds with good conversation- mainly because bad things draw attention).
Remember that there’s often the story and then the ret-conned “story.” From Wikipedia:
> Initially called Startup News or occasionally News.YC.
HN exists to serve the interests of YC. It’s a marketing/advertising tool. It’s great to believe that it’s a community, a place where interesting conversations happen, etc. (and those things may also be true) but remember what it is at its core.
> Initially called Startup News or occasionally News.YC.
HN exists to serve the interests of YC. It’s a marketing/advertising tool. It’s great to believe that it’s a community, a place where interesting conversations happen, etc. (and those things may also be true) but remember what it is at its core.
AFAIK the "official" reason for creating HN was as a proof of concept for a viable web forum written in Arc, the lisp dialect PG was writing at the time. And bear in mind it was called Startup News before it became Hacker News.
HN is not the place for new ranking algorithms since it is more than good enough. However, the other social networks could benefit from better sorting. Would it be possible to create a Youtube client that only shows the interesting comments?
Would it be possible to create a Youtube client that only shows the interesting comments?
But what is interesting? To you and I, perhaps, it is an intellectually well thought out comment.
To YouTube, perhaps, it is what makes people engage emotion.
But what is interesting? To you and I, perhaps, it is an intellectually well thought out comment.
To YouTube, perhaps, it is what makes people engage emotion.
We may be getting to the point where we can directly predict the quality, interestingness, and originality of a comment before it has a chance to receive viewer feedback. This would be similar to how we can now rank historical chess players according to how chess bots evaluate their moves. A forum could then start comments off where it approximates their rank will end up, and then potentially let viewer feedback take over from there.
What's more: if you can't stop bots from dominating your signal, then directly predicting things like quality may be one of the only good options left.
What's more: if you can't stop bots from dominating your signal, then directly predicting things like quality may be one of the only good options left.
Two obvious points: why not machine learning, and a more explicit mix of exploration and exploitation
Meaning give more posts a chance, a certain number of views to determine the upvote rate.
Conventional wisdom would apply spam filter, boost new users (e.g. first post), detect blatant manipulation (or more sophisticated manipulation using clustering).
Meaning give more posts a chance, a certain number of views to determine the upvote rate.
Conventional wisdom would apply spam filter, boost new users (e.g. first post), detect blatant manipulation (or more sophisticated manipulation using clustering).
With machine learning you have the same kind of problems with an added factor of unpredictability.
ML to make the recommendations more dynamic per user means people are not seeing the same things and engagement tanks.
ML to make the recommendations more global means a lot of people are going to see stuff they don't care about and they're going to have no idea why. It's usually this family of problems that ML adds an extra layer of opacity to the most.
Then you add extra tabs to give the user more control, and now you've added a fresh complexity trade off.
The sanest aggregators have simplistic recommendation algorithms and a bunch of janitorial work on top in the form of moderation/censorship. There must be something to this formula.
ML to make the recommendations more dynamic per user means people are not seeing the same things and engagement tanks.
ML to make the recommendations more global means a lot of people are going to see stuff they don't care about and they're going to have no idea why. It's usually this family of problems that ML adds an extra layer of opacity to the most.
Then you add extra tabs to give the user more control, and now you've added a fresh complexity trade off.
The sanest aggregators have simplistic recommendation algorithms and a bunch of janitorial work on top in the form of moderation/censorship. There must be something to this formula.
The article mentions a couple of formulas for the ranking algorithm. I'd like to point out my data-driven reverse-engineering of the HN ranking algorithm from 2013. The basic ranking formula uses votes and age with various exponents. But in my analysis, I found that "penalties" had a large effect on the ranking. Some penalties are applied automatically based on title words or the domain. Other penalties would be applied later. Too many comments would trigger the "controversy" penalty which would cause a sudden and drastic ranking drop.
Link: https://www.righto.com/2013/11/how-hacker-news-ranking-reall...
Link: https://www.righto.com/2013/11/how-hacker-news-ranking-reall...
The actual algorithm seems to take into account post rate in some way. I think the assumption is that high early post rate implies controversial topic. If the ratio is too high it seems to get penalized pretty hard. Maybe downvotes and flaggings of posts is a factor too?
Great to see more people thinking about this stuff!
Our group, "Social Protocols" recently published a concrete idea on how to improve the ranking algorithm by introducing a new metric: https://github.com/social-protocols/news/blob/master/README....
The metric in action: https://news.social-protocols.org/
HN Discussion: https://news.ycombinator.com/item?id=35183317
The metric in action: https://news.social-protocols.org/
HN Discussion: https://news.ycombinator.com/item?id=35183317
I recently launched a tool called HN+
https://hn.plus
It's a tool where you can create your own HackerNews clone. Obviously, while working on this, I had to emulate the HN ranking algorithm. But what I found is that the algorithm actually doesn't seem to work well for a "young" forums.
I had to tweak it so that on the frontpage, the posts that are more relatively recent rises to the top over older posts of higher votes. I know these are considered in the original algorithm, but I had to tweak it.
Which made me think that the algorithm should be considering the age of the community and also how active it is.
https://hn.plus
It's a tool where you can create your own HackerNews clone. Obviously, while working on this, I had to emulate the HN ranking algorithm. But what I found is that the algorithm actually doesn't seem to work well for a "young" forums.
I had to tweak it so that on the frontpage, the posts that are more relatively recent rises to the top over older posts of higher votes. I know these are considered in the original algorithm, but I had to tweak it.
Which made me think that the algorithm should be considering the age of the community and also how active it is.
Perhaps I'm in the minority here, but I would like to see the entirety of HN's codebase posted as-is, no redactions, cleanup, etc.
The entire site is officially moderated by ~2 people, and autonomously by the community. Whatever secret sauce they're using, that should be proof positive that it works. Keeping it a secret deprives countless communities of best-in-class tools and knowledge.
Would it be detrimental to HN? In the short term, possibly (gaming the voting ring detector comes to mind), but it's hard to say what kind of impact there would be without knowing what goes on under the surface.
The entire site is officially moderated by ~2 people, and autonomously by the community. Whatever secret sauce they're using, that should be proof positive that it works. Keeping it a secret deprives countless communities of best-in-class tools and knowledge.
Would it be detrimental to HN? In the short term, possibly (gaming the voting ring detector comes to mind), but it's hard to say what kind of impact there would be without knowing what goes on under the surface.
I always read Hacker News through Feedly because the web page has too small letters and it’s hard to follow them, so I hardly had a chance to see the ranking.
I don't like this idea. If hacker news comments were so numerous that the best comments by the "best" commenters always got lost in the maelstrom then there could be some sense to it. However, this has not been my experience of hacker news, and this shift towards a reputation based system seems unwarranted.
Karma is already intended to be a reputation based system - it just doesn't work as intended, since karma doesn't really measure content quality over time.
https://hnrankings.info graphs the history of every HN story, e.g. https://hnrankings.info/35510413/
[deleted]
Downvotes on posts is another way. It can require a minimum karma level and the value of downvotes can reduce with use to avoid abuse by individuals.
last month, "update to kagi search pricing" ranked 275 with metadata 116 points by exist 8 hours ago with 132 comments
its cohorts ranked 270 to 282 (by my screenshoot) are of age 1day+ and with ten or more comments
it's weird that post which is new, popular and engaging then sunk to the bottom 275 very fast within hours
its cohorts ranked 270 to 282 (by my screenshoot) are of age 1day+ and with ten or more comments
it's weird that post which is new, popular and engaging then sunk to the bottom 275 very fast within hours
it seems that someone replied with a link and it's gone
here's the link: https://hnrankings.info/35078392/
it shows "update to kagi search pricing" started and stayed in the front of hn for 1.5 hours
it then left the front page and raced to rank 300 within the next 4 hours
the disturbing thing is that this hn post with id 35078392 generates 116 points with 132 comments
poof, just like that
here's the link: https://hnrankings.info/35078392/
it shows "update to kagi search pricing" started and stayed in the front of hn for 1.5 hours
it then left the front page and raced to rank 300 within the next 4 hours
the disturbing thing is that this hn post with id 35078392 generates 116 points with 132 comments
poof, just like that
[deleted]
Is HN also built with RoR or is it something custom?
HN is built with Arc.
http://arclanguage.org/
http://arclanguage.org/
1. The article suggests running PageRank on a graph of users as nodes and edges as - "has user A upvoted a comment of user B" as an edge:
""" Since it's likely that one user may upvote multiple comments from the same user, we check whether a user has already upvoted a comment from that specific user before considering their upvote. In other words, we treat user profiles as nodes and upvotes for comments as edges. """
This is a very lossy conversion of the actual data:
a. It does not distinguish if user C and D upvoted the same comment of user B or not. Maybe one comment was good and the other was bad. But when you convert it to the above graph you only get that C and D upvoted some comment of user B.
b. It does not account for the number of comments user B left - 1000 or 5? This incentivizes spam because there is no upside to not posting.
c. It ignores users that do not comment but upvote valuable comments themselves. Then the PageRank is used to weigh upvotes of users. This is backwards. The PageRank values should capture the value of each user's past upvotes in order to use it as a prediction of how valuable their future upvotes will be. But the suggested algorithm uses the value of the user's past comments as a weight of their future upvotes.
To fix this I think the graph needs to be changed to a bipartite graph of users and comments as nodes and upvotes and flags as directed edges (when the author posts a comment - this should be represented as an implicit upvote of the comment). Then you can calculate how valuable each user's upvotes (and flags) are.
2. The "gameability" of PageRank stems from the fact that the random walk algorithm treats each users equally as a starting point. It means that you can create a ton of fake users and upvote the comments of a target user you want to artificially boost in upvote-power. Each time the random walk starts at one of those fake users the walk will end up in the target user - increasing their PageRank score.
My proposal to solve the "gameability" problem is to start each walk from you - the user that views HackerNews. It means that your past upvotes become the starting step of the random walk and so the resulting PageRank will be personalized for you. Instead of a single PageRank reputation score (which captures how user A's contributions to HN have been to all users), there is a set of personalized scores that capture how useful other users have been to you.
I'm building https://linklonk.com which uses this kind of algorithm to rank both links and comments. The details of the ranking algorithm are here: https://linklonk.com/item/3292763817660940288
""" Since it's likely that one user may upvote multiple comments from the same user, we check whether a user has already upvoted a comment from that specific user before considering their upvote. In other words, we treat user profiles as nodes and upvotes for comments as edges. """
This is a very lossy conversion of the actual data:
a. It does not distinguish if user C and D upvoted the same comment of user B or not. Maybe one comment was good and the other was bad. But when you convert it to the above graph you only get that C and D upvoted some comment of user B.
b. It does not account for the number of comments user B left - 1000 or 5? This incentivizes spam because there is no upside to not posting.
c. It ignores users that do not comment but upvote valuable comments themselves. Then the PageRank is used to weigh upvotes of users. This is backwards. The PageRank values should capture the value of each user's past upvotes in order to use it as a prediction of how valuable their future upvotes will be. But the suggested algorithm uses the value of the user's past comments as a weight of their future upvotes.
To fix this I think the graph needs to be changed to a bipartite graph of users and comments as nodes and upvotes and flags as directed edges (when the author posts a comment - this should be represented as an implicit upvote of the comment). Then you can calculate how valuable each user's upvotes (and flags) are.
2. The "gameability" of PageRank stems from the fact that the random walk algorithm treats each users equally as a starting point. It means that you can create a ton of fake users and upvote the comments of a target user you want to artificially boost in upvote-power. Each time the random walk starts at one of those fake users the walk will end up in the target user - increasing their PageRank score.
My proposal to solve the "gameability" problem is to start each walk from you - the user that views HackerNews. It means that your past upvotes become the starting step of the random walk and so the resulting PageRank will be personalized for you. Instead of a single PageRank reputation score (which captures how user A's contributions to HN have been to all users), there is a set of personalized scores that capture how useful other users have been to you.
I'm building https://linklonk.com which uses this kind of algorithm to rank both links and comments. The details of the ranking algorithm are here: https://linklonk.com/item/3292763817660940288
I don't understand this blog post. The authors says that HN is great for discussions. Then they go on to suggest a different ranking algorithm? Why? To answer the authors question: I would change absolutely nothing about HN. It's great for discussions as is.
Edit: I suppose there is some insinuation that HN doesn't really use the original algorithm, and this is the author's attempt to reconstruct what's going on. Is that what I was missing?
Edit: I suppose there is some insinuation that HN doesn't really use the original algorithm, and this is the author's attempt to reconstruct what's going on. Is that what I was missing?
Yes, this is just a thought experiment, and the ranking algorithm is based on discussions.
> Given that discussions are my favorite part, why don't we apply PageRank for every user based on the upvotes they receive for the comments they leave on any post, and replace the P (points) value in the current version of the HN algorithm?
> Given that discussions are my favorite part, why don't we apply PageRank for every user based on the upvotes they receive for the comments they leave on any post, and replace the P (points) value in the current version of the HN algorithm?
> why don’t we
Because that would lead to absolutely terrible longterm outcomes due to obvious reasons.
Because that would lead to absolutely terrible longterm outcomes due to obvious reasons.
If it works, don't fix it. You'll most likely break it.
I think product management across the industry needs to take this more seriously.
In my personal experience, YouTube and Windows 11 are some recent examples of usability degradation due to unnecessary changes.
In my personal experience, YouTube and Windows 11 are some recent examples of usability degradation due to unnecessary changes.
Is it working?
Post ranking works pretty well, but as far as discussions go I often see insightful responses sent into low-contrast oblivion whilst a glib comment or quip gets accolades.
Post ranking works pretty well, but as far as discussions go I often see insightful responses sent into low-contrast oblivion whilst a glib comment or quip gets accolades.
Two big issues I've noticed with post ranking are:
1. There seems to be a group of people that abuse the flag button to get stories downvoted. You notice this when highly upvoted stories with a lot of points suddenly drop from the top and fall quickly beyond older stories with many fewer votes. For instance, there seems to be a group that was mass flagging submissions about Rob Lee's recent death until one was made that didn't mention his cause of death and explicitly forbid discussion of it in the comments.
2. Partly because of #1, you can have a topic with a lot of votes at the top of the page suddenly drop to page 5 and disappear within a very short amount of time. This significantly lowers the incentive to spend any time writing a thoughtful response or researching useful information, because even spending 20-30 minutes on a response might mean that no one is going to see what you wrote.
1. There seems to be a group of people that abuse the flag button to get stories downvoted. You notice this when highly upvoted stories with a lot of points suddenly drop from the top and fall quickly beyond older stories with many fewer votes. For instance, there seems to be a group that was mass flagging submissions about Rob Lee's recent death until one was made that didn't mention his cause of death and explicitly forbid discussion of it in the comments.
2. Partly because of #1, you can have a topic with a lot of votes at the top of the page suddenly drop to page 5 and disappear within a very short amount of time. This significantly lowers the incentive to spend any time writing a thoughtful response or researching useful information, because even spending 20-30 minutes on a response might mean that no one is going to see what you wrote.
> There seems to be a group of people that abuse the flag button…until one was made that didn't mention his cause of death
I’m not sure that I’d characterize this as abuse. HN stays interesting by avoiding flame bait topics and part of the realization of the community guidelines happens via this flag feature.
One way to reframe #1 is that the community’s self-moderation features are working as designed, and that any post about Bob Lee’s death that somehow editorializes it or positions it in a way that is likely to lead to political debates is against the site guidelines and thus were removed appropriately.
Regarding #2, this has happened to me more than once, and while frustrating, it made me think more about posts I encounter before commenting. When a post fits the criteria for “potential flame war” or “clearly outside of site guidelines”, I flag it instead of commenting, even if I know that at the core, the topic could be an interesting one. Most things get posted more than once, and usually at least one of the posts is within guidelines.
When I do lose a comment to a flagged post, I’ll reuse it in a more level headed discussion about the topic.
I’m not sure that I’d characterize this as abuse. HN stays interesting by avoiding flame bait topics and part of the realization of the community guidelines happens via this flag feature.
One way to reframe #1 is that the community’s self-moderation features are working as designed, and that any post about Bob Lee’s death that somehow editorializes it or positions it in a way that is likely to lead to political debates is against the site guidelines and thus were removed appropriately.
Regarding #2, this has happened to me more than once, and while frustrating, it made me think more about posts I encounter before commenting. When a post fits the criteria for “potential flame war” or “clearly outside of site guidelines”, I flag it instead of commenting, even if I know that at the core, the topic could be an interesting one. Most things get posted more than once, and usually at least one of the posts is within guidelines.
When I do lose a comment to a flagged post, I’ll reuse it in a more level headed discussion about the topic.
> that any post about Bob Lee’s death that somehow editorializes it or positions it in a way that is likely to lead to political debates is against the site guidelines and thus were removed appropriately.
A lot of discussions on HN lead to discussions of government policies (both U.S. and foreign) when there's a connection to the story, including saying that certain policies lead to the death of individuals. There's only a subset that gets mass flagged. If he was shot to death by police at a protest, killed by a U.S. military strike in another country, died as a result of a mass Covid surge, stories mentioning the reason for his death wouldn't be mass flagged. People wouldn't be forbidden from mentioning the causes of death on the only one that isn't, and it wouldn't be dismissed as "political flamebait."
If Hacker News decides that discussion about government policies is off limits, that's fine. It might even make discussions better. But allowing it in some situations and having it "removed appropriately" in others is only reinforcing bias. I think characterizing that as abuse is appropriate.
Edit: If you want to see an example, look at the Hacker News discussion about the death of Aaron Swartz. The cause of death isn't hidden, and there's a lot of criticism of the government, none of which is getting dismissed as editorializing or political flamebait[1].
[1] https://news.ycombinator.com/item?id=5046845
A lot of discussions on HN lead to discussions of government policies (both U.S. and foreign) when there's a connection to the story, including saying that certain policies lead to the death of individuals. There's only a subset that gets mass flagged. If he was shot to death by police at a protest, killed by a U.S. military strike in another country, died as a result of a mass Covid surge, stories mentioning the reason for his death wouldn't be mass flagged. People wouldn't be forbidden from mentioning the causes of death on the only one that isn't, and it wouldn't be dismissed as "political flamebait."
If Hacker News decides that discussion about government policies is off limits, that's fine. It might even make discussions better. But allowing it in some situations and having it "removed appropriately" in others is only reinforcing bias. I think characterizing that as abuse is appropriate.
Edit: If you want to see an example, look at the Hacker News discussion about the death of Aaron Swartz. The cause of death isn't hidden, and there's a lot of criticism of the government, none of which is getting dismissed as editorializing or political flamebait[1].
[1] https://news.ycombinator.com/item?id=5046845
I think we're discussing different things. I agree that plenty of discussion tends to touch on those subjects, and in the 2020s, it seems increasingly difficult to separate them from many aspects of our lives. But those discussions are in a very literal sense exceptions to the rules based on their relevance to the tech community. And mentioning the cause of death was not forbidden.
The initial post about Bob Lee was split off into two, so one could remain focused on remembering Bob for who he was [0], and the other could explore the surrounding circumstances [1].
With that said, the site guidelines are pretty clear:
> Off-Topic: Most stories about politics, or crime, or sports, or celebrities, unless they're evidence of some interesting new phenomenon ... If they'd cover it on TV news, it's probably off-topic.
> Please don't use Hacker News for political or ideological battle. That tramples curiosity.
So this will often boil down to the framing. If the title is tinged with political implications, that's quite a bit different than mentioning the cause of death.
The circumstances surrounding Bob Lee's death are relevant to quite a few readers here, and so there was a discussion about it. But a default assessment of any given submission if weighed against the guidelines should assume that such content is off-topic, unless there's a reason to believe otherwise (as there was in this and cases like the one you cited that are close to the tech community).
Even with that separation, there was quite a bit of discussion that was clearly far outside of the guidelines in that thread.
- [0] https://news.ycombinator.com/item?id=35457341
- [1] https://news.ycombinator.com/item?id=35448899
The initial post about Bob Lee was split off into two, so one could remain focused on remembering Bob for who he was [0], and the other could explore the surrounding circumstances [1].
With that said, the site guidelines are pretty clear:
> Off-Topic: Most stories about politics, or crime, or sports, or celebrities, unless they're evidence of some interesting new phenomenon ... If they'd cover it on TV news, it's probably off-topic.
> Please don't use Hacker News for political or ideological battle. That tramples curiosity.
So this will often boil down to the framing. If the title is tinged with political implications, that's quite a bit different than mentioning the cause of death.
The circumstances surrounding Bob Lee's death are relevant to quite a few readers here, and so there was a discussion about it. But a default assessment of any given submission if weighed against the guidelines should assume that such content is off-topic, unless there's a reason to believe otherwise (as there was in this and cases like the one you cited that are close to the tech community).
Even with that separation, there was quite a bit of discussion that was clearly far outside of the guidelines in that thread.
- [0] https://news.ycombinator.com/item?id=35457341
- [1] https://news.ycombinator.com/item?id=35448899
> Is it working?
In my experience, HN seems okay for reading and participating in meaningful discussion. Comments that don't contribute or are inappropriate don't stay in sight for long (possibly due to the community upholding the guidelines), whereas useful discussions and references to projects tend to stay towards the top. I don't always agree with what sits at the top, but if nothing else it's civilized and reasoned.
In some respects HN has the typical issue of most attention focusing on the first page posts and most discussion being in one or two top level comments, but that's just how people engage with the content, short of randomization there's no way around that. There will always be a top comment and everything else will receive slightly more lukewarm reception and activity.
In my experience, HN seems okay for reading and participating in meaningful discussion. Comments that don't contribute or are inappropriate don't stay in sight for long (possibly due to the community upholding the guidelines), whereas useful discussions and references to projects tend to stay towards the top. I don't always agree with what sits at the top, but if nothing else it's civilized and reasoned.
In some respects HN has the typical issue of most attention focusing on the first page posts and most discussion being in one or two top level comments, but that's just how people engage with the content, short of randomization there's no way around that. There will always be a top comment and everything else will receive slightly more lukewarm reception and activity.
Others might disagree with your idea of what to upvote. It's very hard to benchmark something like this.
This is Hacker News. Over engineering things is part of our cultural heritage.
<< I don't understand this blog post.
Well, the article is something of an ad. The author discusses current ranking and effectively proves that it could be gamed to an extent ( author links his non-SEO search engine, which obviously you cannot do unless you are deep into SEO stuff, which includes HN ranking algorithm ). The complaint is mildly self-serving, but only mildly so. FWIW, it worked on me.
Still, I will echo what others said. Despite my grievances with HN at times, don't change what ain't broken.
Well, the article is something of an ad. The author discusses current ranking and effectively proves that it could be gamed to an extent ( author links his non-SEO search engine, which obviously you cannot do unless you are deep into SEO stuff, which includes HN ranking algorithm ). The complaint is mildly self-serving, but only mildly so. FWIW, it worked on me.
Still, I will echo what others said. Despite my grievances with HN at times, don't change what ain't broken.
Totally agree. Ain't broke, don't fix it.
[deleted]
If there's more of a ranking algorithm, identifying who is most likely to have actually read the article could be neat.