Show HN: Mental State of HN Users Before/After Lockdown(kaggle.com)
kaggle.com
Show HN: Mental State of HN Users Before/After Lockdown
https://www.kaggle.com/kag888/mental-state-of-hn-users-before-after-lockdown
37 comments
The way this is worded, it seems like the author simply counted the number of posts with negative words in them. And then compared the number to last year.
In that case, the increase could be explained by more posts on HN overall. Without any change in "mental state".
Lets have a look into the calculations ...
Hmm.. as far as I can see, the calculations indeed do not take overall change in posts into account.
In that case, the increase could be explained by more posts on HN overall. Without any change in "mental state".
Lets have a look into the calculations ...
Hmm.. as far as I can see, the calculations indeed do not take overall change in posts into account.
HN didn't have a significant increase in traffic due to covid compared to other sites, hence the more submissions didn't lead to more traffic. OP dropped also submissions that didn't hit the frontpage by removing those with less than 10 comments. So the 19% more submissions couldn't influence the result. And even if, you'd still have 19% vs 40%.
I don't see how other sites play a role in this.
Only counting posts with 10+ posts does not guarantee that the baseline is the same for the two groups. You seem to imply that the number of posts that reach over 10 points means they reach the frontpage and that the number of posts that reach the frontpage is always the same during two time periods of the same length. But I don't see how either of these two assumptions has to be true.
The problem with the "19% vs 40%" comparison is that it does not compare the same thing. According to this article:
19% is the increase in total posts
40% is the increase after posts with less then 10 votes have been removed AND posts without the negative keywords have been removed.
This comparison is not the right one. Either posts with less then 10 votes need to be subtracted from both groups or from none of them.
Only counting posts with 10+ posts does not guarantee that the baseline is the same for the two groups. You seem to imply that the number of posts that reach over 10 points means they reach the frontpage and that the number of posts that reach the frontpage is always the same during two time periods of the same length. But I don't see how either of these two assumptions has to be true.
The problem with the "19% vs 40%" comparison is that it does not compare the same thing. According to this article:
19% is the increase in total posts
40% is the increase after posts with less then 10 votes have been removed AND posts without the negative keywords have been removed.
This comparison is not the right one. Either posts with less then 10 votes need to be subtracted from both groups or from none of them.
That isn't necessarily the mental state of HN. Given the nature of HN, it could just reflect interest in the subject.
Relative to what? How did the dataset have those "in_lockdown" differenciation?
slightly OT and totally non scientific (just my gut feeling), but I have a feeling that HN became more polarized and negative/combative in recent months. I think I notice much more downvotes and flagging taking place than before. I wonder if this is also something that can be analyzed? (not sure HN exposes it, but in a way simpler than sentiment analysis, right?)
I definitely noticed this. In February-March, the whole experience became vastly more hostile and stressful. I stopped getting replies with relevant anecdotes, corrections, or follow-up information, and started getting lots of replies attacking my character and dismissing my experiences.
I changed my HN account's password and logged out for good near the start of April. I still read it and occasionally make a one-off account like this to comment, but I don't see it as a community any more. In my mind, it is the new Reddit front page.
But hey, dang - thanks for all the hard work and good moderation while it lasted.
I changed my HN account's password and logged out for good near the start of April. I still read it and occasionally make a one-off account like this to comment, but I don't see it as a community any more. In my mind, it is the new Reddit front page.
But hey, dang - thanks for all the hard work and good moderation while it lasted.
Genuinely curious, at what point does it become worthwhile to try to create a “new HackerNews”? I myself am still figuring out how to be an adult so I don’t feel that I have much insight to give on this. I’m wondering what any remaining older HN users might think about preserving what’s left (if that makes sense).
Totally non scientific (just my gut feeling), but I have a feeling that this is because the number of non-technical submissions has increased dramatically.
It's really everywhere due to the election. I lurk on a meme video forum that leans conservative and they have started deleting and clamping down on any political content. Some people really enjoy stirring up and having political debate but lots of the regulars can't stand it.
I've been relatively ok (I have an anxiety disorder), but it's been pretty hard on my son who's on the spectrum. He used to love going to parks and playing, but after a few months of lockdown he is afraid of stepping out anywhere. Anxiety is now up through the roof. He taught himself to read (with basic comprehension) during the lockdown (just before his 3rd birthday) - but that's a good thing and a bad thing (hyperlexia). Considering therapy now for him, which we weren't entirely for earlier since many people in the spectrum have highlighted some important issues.
In short, it's been pretty hard on the family. I decided to go back to my anxiety medication which I hadn't needed for quite some time.
In short, it's been pretty hard on the family. I decided to go back to my anxiety medication which I hadn't needed for quite some time.
Thanks! I learned a new word - "hyperlexia" - which I now think applied to me in some form when I was very young. It's been interesting reading about it. I picked up reading when I was very young, much faster and earlier than people expected, and reading things people thought a young child wouldn't be able to.
I don't remember it being a problem, just unusual.
But now I think about it, as an older adult I don't just enjoy reading, I'm addicted to reading, to the extent it stops me getting as much work or other things done as I'd like. It's strange to admit that, but it's true.
It's intriguing to consider that might have roots in a neurological structure associated with what is now a named disorder, rather than just a habit.
Though I wouldn't say I remember lacking comprehension skills, perhaps I did have some aspect of that when I was very young, it's hard to be sure. I completely lacked the ability to write creatively though (stories etc), until about age 13 when a single insight while stuck transformed that from 0% to 100% in moments.
I don't remember it being a problem, just unusual.
But now I think about it, as an older adult I don't just enjoy reading, I'm addicted to reading, to the extent it stops me getting as much work or other things done as I'd like. It's strange to admit that, but it's true.
It's intriguing to consider that might have roots in a neurological structure associated with what is now a named disorder, rather than just a habit.
Though I wouldn't say I remember lacking comprehension skills, perhaps I did have some aspect of that when I was very young, it's hard to be sure. I completely lacked the ability to write creatively though (stories etc), until about age 13 when a single insight while stuck transformed that from 0% to 100% in moments.
What was the insight that boosted your ability to write creatively?
You could probably use NRC Emotion Lexicon[1] to categorize the titles into different emotions and sort it like that, as well and give it some normalized emotional score/category for X. Then use logistic regression for Y being before/after lockdown or linear regression for Y being (current_date - lockdown_date). The linear regression will assume that things get linearly worse over time (it can be smoothed out better probably).
1: https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm#:~....
1: https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm#:~....
> You could probably use NRC Emotion Lexicon
Thanks and good idea! How does this compare to some pre-trained ML model for sentiments analysis?
> Then use logistic regression for Y being before/after lockdown ...
Just googled logistic regression but would love to hear some examples in this context to get a better grasp on this. Thanks!
Thanks and good idea! How does this compare to some pre-trained ML model for sentiments analysis?
> Then use logistic regression for Y being before/after lockdown ...
Just googled logistic regression but would love to hear some examples in this context to get a better grasp on this. Thanks!
I would think you could use the pre-trained ML model and achieve the same result. If you really want to hack away at it, the Machine Learning course at Stanford has really good notes. http://cs229.stanford.edu/syllabus-fall2020.html
I just thought of logistic regression because you are doing binary classification (is the title before or after the outbreak?) and it is the simplest (and most general) way to do it. But I think it would be more interesting to do linear regression (how long has it been since lockdown?) which would analyze how the sentiments change relative to the outbreak. So maybe take the time period between -2 months and +6 months relative to lockdown and then see what role time plays in the sentiment. You could easily extend this to newspapers and what not, I am sure people have tried this before!
You can conceptually change the problem of "how did breakout affect HN titles" to "given some HN title, predict whether it's before or after breakout".
I just thought of logistic regression because you are doing binary classification (is the title before or after the outbreak?) and it is the simplest (and most general) way to do it. But I think it would be more interesting to do linear regression (how long has it been since lockdown?) which would analyze how the sentiments change relative to the outbreak. So maybe take the time period between -2 months and +6 months relative to lockdown and then see what role time plays in the sentiment. You could easily extend this to newspapers and what not, I am sure people have tried this before!
You can conceptually change the problem of "how did breakout affect HN titles" to "given some HN title, predict whether it's before or after breakout".
Anyone wants to offer an alternative hypothesis on why HN traffic didn't get skyrocketed like every other outlet?
Personally for me, before during and after the quarantine my use of HN has pretty much remained the same, I work with computers (like most of us here) and I check HN when I get a little bit of down-time, like while waiting for something to compile or waiting for a file to get transferred or something to finish installing,only thing that changed was during the quarantine I was working from home, the number of hours I worked, the nature of work I do and how I do that work remained pretty much the same.
So, I am guessing this is pretty much what happened to most of us as well? And therefore for a majority of HN users, our HN browsing habits probably stayed the same?
So, I am guessing this is pretty much what happened to most of us as well? And therefore for a majority of HN users, our HN browsing habits probably stayed the same?
in short, HN readers were already living in de facto quarantine pre quarantine
Let me fix that :
In short, HN readers were already living in the future pre quarantine.
Basically applying one of PG maxims in "How to Get Startup Ideas"[0]: "Live in the future, then build what's missing."
http://www.paulgraham.com/startupideas.html
http://www.paulgraham.com/startupideas.html
I am genuinely curious of what triggered the downvote. It makes me feel as if I didn't express myself correctly. If anyone has an idea ?
it was less relevant than my comment, which was already not very relevant.
Thanks for your reply ! Well, I found your comment extremely relevant, and delivered in a very succint (albeit tongue in cheek ) way.
A greater share of the HN readership was already having a quarantine friendly work/lifestyle, than the general population. Due to underlying macro-trends (technology, mobile consumer internet, entrepreneurship...).
And one of these macro-trends, the fragility of our hyper-reactive interconnected world, was illuminated by the covid crisis. I will try to reach you on twitter, to not pollute this thread further.
A greater share of the HN readership was already having a quarantine friendly work/lifestyle, than the general population. Due to underlying macro-trends (technology, mobile consumer internet, entrepreneurship...).
And one of these macro-trends, the fragility of our hyper-reactive interconnected world, was illuminated by the covid crisis. I will try to reach you on twitter, to not pollute this thread further.
i doubt it really matters
for the record i did not downvote you.
for the record i did not downvote you.
It’s possible that the moderators tweaked some knobs to control the rates of submission, the rise to the front page, the time on the front page, etc. Not that they did it individually, but there could be more added mechanisms this year. Maybe they will respond and provide some insights.
Here's a hypothesis. I have no idea if it's true:
HN users tend to be in relatively well protected employment with regard to lockdown, i.e. either already remote, or as software devs / white collar workers who can easily deal with the switch to remote, including the costs.
HN users tend to be in relatively well protected employment with regard to lockdown, i.e. either already remote, or as software devs / white collar workers who can easily deal with the switch to remote, including the costs.
This is a really cool idea. Would be really interesting to run it through a sentiment analysis engine as well.
Wow, very interesting. Thanks for posting this, I really appreciate how clear and simple your notebook is. I had a sense that this sentiment had grown significantly, but I thought that perhaps it might be receding after a temporary spike. Perhaps that was too optimistic.
Was there an uptick in show hn post?
> Edit2: HN blocked me, so I can't reply any more on my own thread: https://news.ycombinator.com/item?id=24738722#
Any idea why sunnysofa has been blocked?
Any idea why sunnysofa has been blocked?
Earlier comments by ‘sunnysofa are marked dead, so their submission and other comments may have been vouched for, or they’re caught in a filter, or something else. Hard to say. For a definitive answer contact the mods using the Contact link in the footer.
@dang why did you block me, so I couldn't reply anymore in my own thread?
Edit: + shadow-banning + super gravity, so my comments land on the bottom
What did I do wrong?
Edit: + shadow-banning + super gravity, so my comments land on the bottom
What did I do wrong?
I'm guessing because your very first comment and submitted title were not the kind of content people would like to see on HN, which could have penalized your account early on.
Anyway, I vouched for this comment to become visible.
Anyway, I vouched for this comment to become visible.
> 40.0% more submissions with depression related keywords in the title after lockdown vs before
18% more comments on posts with depression related keywords in the title after lockdown vs before
9% more upvotes on posts with depression related keywords in the title after lockdown vs before