HackerTrans
TopNewTrendsCommentsPastAskShowJobs

anvaka

no profile record

Submissions

Take this new map of GitHub with 690k repositories

anvaka.github.io
4 points·by anvaka·السنة الماضية·2 comments

1.5B recent comments turned into a map of Reddit (2025 update)

anvaka.github.io
5 points·by anvaka·السنة الماضية·2 comments

Hundreds of millions of stars turned into a map of GitHub projects

anvaka.github.io
471 points·by anvaka·قبل 3 سنوات·85 comments

New map of Reddit built from 334M comments

anvaka.github.io
38 points·by anvaka·قبل 3 سنوات·7 comments

comments

anvaka
·السنة الماضية·discuss
I have updated the map with the latest data. This one is based on 500M stars given to repositories from GitHub's start till end of April, 2025.

Repositories are clustered based on jaccard similarity.

The mao helps you find what other related projects are available. So if you know something good - you can easily find what else is there.
anvaka
·السنة الماضية·discuss
Thank you!
anvaka
·قبل سنتين·discuss
oh wow. Glad you liked it!
anvaka
·قبل سنتين·discuss
appreciate the feedback - I'll take a look
anvaka
·قبل سنتين·discuss
haha thank you!
anvaka
·قبل سنتين·discuss
I couldn't find a universal clustering algorithm yet: Frequently there is more than one way to group data that still makes sense, and as a result whichever final clustering option we choose - it will not be perfect.

Hm... unless maybe we do some sort of quantum clustering, which could be a fun project to explore!

It's a bit hazy now, but I remember trying hdbscan algorithm (hierarchical clustering), and on the graph of the GitHub size - I just couldn't fit it in memory.

I did end up using something similar to hierarchical clustering (mix of louvain/leiden/my own), and that's what we see in the final map.
anvaka
·قبل سنتين·discuss
haha! I love vim.

We shall not quit.
anvaka
·قبل سنتين·discuss
love it =)!
anvaka
·قبل سنتين·discuss
Yes...

Aiming to redo it some time in early 2025!
anvaka
·قبل سنتين·discuss
Jaccard similarity is not particularly good for "celebrity" projects.

They are similar because they are popular, not because there is semantic relationship.

It's the same problem I faced with the map of reddit (https://anvaka.github.io/map-of-reddit/ ) - all popular subreddits are just "similar" to each other.

Stil works great for smaller, non-celebrity projects :D
anvaka
·قبل سنتين·discuss
Thank you for your kind words!

That link you've shared - doesn't open for some reason
anvaka
·قبل سنتين·discuss
Thank you!

I tried quite a few various similarity metrics, and Jaccard was giving me the best results. This is all very subjective, of course.
anvaka
·قبل سنتين·discuss
Thank you =)
anvaka
·قبل سنتين·discuss
This is correct!
anvaka
·قبل 3 سنوات·discuss
I agree. I'm not sure why they are together. Maybe all that tooling stuff that was happening in early days?
anvaka
·قبل 3 سنوات·discuss
You are very kind and your compliment put smile on my face too :)
anvaka
·قبل 3 سنوات·discuss
Thank you so much for your compliment!
anvaka
·قبل 3 سنوات·discuss
thank you!
anvaka
·قبل 3 سنوات·discuss
Thank you so much!

I'm puzzled by some countries in Frontartia's island too. I'm not sure why it was even pulled away, as if there is something I'm missing.
anvaka
·قبل 3 سنوات·discuss
Indeed! A lot of amazing things happens there. A few are getting world wide adoption (echarts are awesome if you are into frontend viz, so is element-plus library for vue components). There are giant communities of ML too