Reddit 1.0 was written in Lisp(github.com)
github.com
Reddit 1.0 was written in Lisp
https://github.com/reddit-archive/reddit1.0
142 comments
Here's an updated version with DB tables (PostGres) that runs: https://github.com/tamurashingo/reddit1.0/
To replace JS a maximum I use HTMX which is language and framework agnostic, life is beautiful.
In the end, you can build a static binary with all the static assets, send it to your server and run it.
- https://github.com/fukamachi/clack
- https://lispcookbook.github.io/cl-cookbook/web.html
- https://github.com/CodyReichert/awesome-cl#web-frameworks
Require:
CommonLisp (tested on SBCL)
PostgreSQL
memcached
smtp server
CL-USER> (reddit:startup-reddit)
#<HUNCHENTOOT:EASY-ACCEPTOR (host *, port 8000)>
To replace JS a maximum I use HTMX which is language and framework agnostic, life is beautiful.
In the end, you can build a static binary with all the static assets, send it to your server and run it.
- https://github.com/fukamachi/clack
- https://lispcookbook.github.io/cl-cookbook/web.html
- https://github.com/CodyReichert/awesome-cl#web-frameworks
HTMX is not anymore framework agnostic than JS. It's a JS library. You can use any backend with either.
He meant that HTMX can work with a backend written in any language/framework. It doesn't constrain you to use any specific server-side tech.
Yes, that is the same with javascript.
So I've read that they wrote it in Lisp at Paul Graham's request or to win favor. Does anyone know if writing your startup code in Lisp still has that benefit?
It never had any particular benefit to write your startup code in Lisp.
That myth came from a few posts by a person who was a huge proponent of Lisp, in an era were web (front and backend) development was much easier, and web-related libraries were lacking in all languages (so it wasn't like Lisp's smaller popularity would hurt you there).
It's an example of one.
If they didn't get lucky and Yahoo! bought some other company in the same domain in their place (they already have a few competitors) the whole "Lisp as a secret weapon" would be moot.
Statistically speaking, 99.9% of succesful startups were written not in any Lisp, and 0.1% of them in Lisp. Not the biggest 0.1% either, more towards the bottom, in the context of web company deals during the ramp up to the dot-com bust. Viaweb was sold for $50 million. For context, Yahoo! bought Broadcast.com a year later for 5 billion and doc.com startups of the era routinely raising and burning $50-$100 million for fun - heck, Razorfish (a design consultancy) built websites for clients like Levis, Sony, Mercedes Benz, etc. for $20-$40 million (and that's late-90s era websites, nothing especially fancy).
Of course all that's irrelevant to whether Viaweb was written in Lisp. Which is my point, exactly. What's more, Yahoo! even scrapped all their Lisp code soon after.
Now, writing your startup code in a productive language, that doesn't slow down the development team, allows for easy+quality hires, and so on - even if it's not the best for when you get huge, is a benefit. That hardly requires Lisp.
That myth came from a few posts by a person who was a huge proponent of Lisp, in an era were web (front and backend) development was much easier, and web-related libraries were lacking in all languages (so it wasn't like Lisp's smaller popularity would hurt you there).
It's an example of one.
If they didn't get lucky and Yahoo! bought some other company in the same domain in their place (they already have a few competitors) the whole "Lisp as a secret weapon" would be moot.
Statistically speaking, 99.9% of succesful startups were written not in any Lisp, and 0.1% of them in Lisp. Not the biggest 0.1% either, more towards the bottom, in the context of web company deals during the ramp up to the dot-com bust. Viaweb was sold for $50 million. For context, Yahoo! bought Broadcast.com a year later for 5 billion and doc.com startups of the era routinely raising and burning $50-$100 million for fun - heck, Razorfish (a design consultancy) built websites for clients like Levis, Sony, Mercedes Benz, etc. for $20-$40 million (and that's late-90s era websites, nothing especially fancy).
Of course all that's irrelevant to whether Viaweb was written in Lisp. Which is my point, exactly. What's more, Yahoo! even scrapped all their Lisp code soon after.
Now, writing your startup code in a productive language, that doesn't slow down the development team, allows for easy+quality hires, and so on - even if it's not the best for when you get huge, is a benefit. That hardly requires Lisp.
> Statistically speaking, 99.9% of succesful startups were written not in any Lisp, and 0.1% of them in Lisp.
I think the point that PG was making wasn't that P(lisp|success) > P(blub|success) is what's key but that P(success|lisp) > P(success|blub)
> What's more, Yahoo! even scrapped all their Lisp code soon after.
I'm hardly a PG fanboy, but I have to say this is exactly what's expected in PG's original post on the subject[0]. It's precisely because big companies will only stick to blub languages that Lisp becomes an advantage for a startup.
The point of "Beating the Averages" isn't that you must use Lisp but that small groups of elite hackers can use more powerful tools than larger organizations can't get away with because they generally have lower skilled, only comfortable with blub, programmers.
The reason we don't see many startups following this advice now isn't because it's wrong (necessarily), but because since that essay was written, the vast majority of the startup ecosystem has come to resemble large companies. We've had a pipeline churning out devs for many years now. The ones that are great at leetcode and playing the game go to FAANG, the ones that aren't go to startups. It's not even about where the higher skilled devs are because that hasn't played a factor in most companies success in a long time.
With rare exceptions, startups are no longer created by brilliant hackers looking to change how things are, but by business minded people looking to take advantage of the easy VC money in the last decade. Interviewing at startups used to be exciting, now the vast majority feel like they're run by people who couldn't manage to get a director role at a FAANG.
0. http://www.paulgraham.com/avg.html
I think the point that PG was making wasn't that P(lisp|success) > P(blub|success) is what's key but that P(success|lisp) > P(success|blub)
> What's more, Yahoo! even scrapped all their Lisp code soon after.
I'm hardly a PG fanboy, but I have to say this is exactly what's expected in PG's original post on the subject[0]. It's precisely because big companies will only stick to blub languages that Lisp becomes an advantage for a startup.
The point of "Beating the Averages" isn't that you must use Lisp but that small groups of elite hackers can use more powerful tools than larger organizations can't get away with because they generally have lower skilled, only comfortable with blub, programmers.
The reason we don't see many startups following this advice now isn't because it's wrong (necessarily), but because since that essay was written, the vast majority of the startup ecosystem has come to resemble large companies. We've had a pipeline churning out devs for many years now. The ones that are great at leetcode and playing the game go to FAANG, the ones that aren't go to startups. It's not even about where the higher skilled devs are because that hasn't played a factor in most companies success in a long time.
With rare exceptions, startups are no longer created by brilliant hackers looking to change how things are, but by business minded people looking to take advantage of the easy VC money in the last decade. Interviewing at startups used to be exciting, now the vast majority feel like they're run by people who couldn't manage to get a director role at a FAANG.
0. http://www.paulgraham.com/avg.html
I think the real reason startups are using blub is something else entirely, tooling. The reason lisp was cool in the early days were because it allows you to easily build abstractions to improve productivity, but these days any language comes with a decent web framework. You also get fancy IDE's and language servers and SDK's from your cloud provider. The productivity a large ecosystem gives you IMO is much larger than what a language can provide.
>I think the point that PG was making wasn't that P(lisp|success) > P(blub|success) is what's key but that P(success|lisp) > P(success|blub)
Perhaps, but that was handwaving from an example of one.
Would Lisp startups dominate if more people did Lisp startups?
I seriously doubt so, but in any case, the examples we see are people not doing Lisp startups and succeeding just fine - with the language used hardly being a serious factor (compared to timing, feature set, user adoption, VC interest, and mere dumb luck).
>Interviewing at startups used to be exciting, now the vast majority feel like they're run by people who couldn't manage to get a director role at a FAANG.
Isn't the latter true for most real hacker types?
Perhaps, but that was handwaving from an example of one.
Would Lisp startups dominate if more people did Lisp startups?
I seriously doubt so, but in any case, the examples we see are people not doing Lisp startups and succeeding just fine - with the language used hardly being a serious factor (compared to timing, feature set, user adoption, VC interest, and mere dumb luck).
>Interviewing at startups used to be exciting, now the vast majority feel like they're run by people who couldn't manage to get a director role at a FAANG.
Isn't the latter true for most real hacker types?
> We've had a pipeline churning out devs for many years now. The ones that are great at leetcode and playing the game go to FAANG, the ones that aren't go to startups.
What about companies with a technology department that aren’t FAANG or a startup? Isn’t that where most devs go?
What about companies with a technology department that aren’t FAANG or a startup? Isn’t that where most devs go?
I think that difference is that now every mainstream programming language is so good that there is not that much difference outside very specific niches. That was not case in 2003.
Lisp-vs-mainstream was just a very different comparison in the 90s versus now. Most of Lisp's advantages have been incorporated into its big competitors. At the time the advantages were nothing to sneer at, even though there were counterpoints then too.
As someone who’s been involved in both, it’s not like the language you pick doesn’t make a difference, but I’d say that there are other factors that vastly overshadow it.
For example, one thing we were told when choosing to write in a lisp was that we’d have problems finding developers. It turned out to be true in that we received fewer applications, but compared to the same process with a more mainstream language, I’d say the applicants were on average of higher quality. So by the final step, we were approximately considering the same number of résumés in both cases.
But in the end, the language is not what has made the biggest impact. It’s been other, more universally applicable things, such as team dynamics and communication, clarity of vision and consistency in execution, process and planning, and a ton of other things where the particular language is not a factor.
For example, one thing we were told when choosing to write in a lisp was that we’d have problems finding developers. It turned out to be true in that we received fewer applications, but compared to the same process with a more mainstream language, I’d say the applicants were on average of higher quality. So by the final step, we were approximately considering the same number of résumés in both cases.
But in the end, the language is not what has made the biggest impact. It’s been other, more universally applicable things, such as team dynamics and communication, clarity of vision and consistency in execution, process and planning, and a ton of other things where the particular language is not a factor.
Many of the most famous "done in Lisp" products I know of have the delightful feature that it's not Lisp itself that provides a solution that other languages can't. Rather:
1) Developers who like Lisp tend to be really smart, well-read, and well-educated.
2) Lisp streamlines these developers' thought processes, allowing them to get more done with less, fast.
For example, the early Naughty Dog games written in GOOL/GOAL take clever advantage of asset data to produce effects close to the very edge of what the PlayStation/PlayStation 2 can do. No feature in these Lisp languages made this possible where C or C++ could not, but working in Lisp enabled the developers to iterate quickly, experiment, and arrive at interesting innovative solutions within the required time frame.
1) Developers who like Lisp tend to be really smart, well-read, and well-educated.
2) Lisp streamlines these developers' thought processes, allowing them to get more done with less, fast.
For example, the early Naughty Dog games written in GOOL/GOAL take clever advantage of asset data to produce effects close to the very edge of what the PlayStation/PlayStation 2 can do. No feature in these Lisp languages made this possible where C or C++ could not, but working in Lisp enabled the developers to iterate quickly, experiment, and arrive at interesting innovative solutions within the required time frame.
I'm sure the commenter above has seen this but for anyone who hasn't: Andy Gavin talking about pushing the envelope of the Playstation for Crash Bandicoot.
https://www.youtube.com/watch?v=izxXGuVL21o
https://www.youtube.com/watch?v=izxXGuVL21o
> No feature in these Lisp languages made this possible where C or C++ could not, but working in Lisp enabled the developers to iterate quickly, experiment, and arrive at interesting innovative solutions within the required time frame.
You just described key Lisp features that aren't in C++. It's almost the entire point of using Lisp.
You just described key Lisp features that aren't in C++. It's almost the entire point of using Lisp.
I worked at a game studio who used a lisp like that had morphed into a pythonic c with parens instead of curleys.
I asked the dev who made the engine why lisp.
"I had a month to build a new scripting engine. Lisp is the easiest to implement in c/c++ or in assembly and I had done it in college. So I made a compiler /emulator in like a week. If I had more time I would have made a dumbed down python with parallelism"
The parallelism was cool because you could have multiple conditions running at once like 5 ai walking 5 simple action loops until one of 3 conditioned collapsed them all into attack mode. Easy to model as threads with joins in this language. Could have been done with other languages but harder to do the memory management.
So not really cause it was a lisp
I asked the dev who made the engine why lisp.
"I had a month to build a new scripting engine. Lisp is the easiest to implement in c/c++ or in assembly and I had done it in college. So I made a compiler /emulator in like a week. If I had more time I would have made a dumbed down python with parallelism"
The parallelism was cool because you could have multiple conditions running at once like 5 ai walking 5 simple action loops until one of 3 conditioned collapsed them all into attack mode. Easy to model as threads with joins in this language. Could have been done with other languages but harder to do the memory management.
So not really cause it was a lisp
> For example, one thing we were told when choosing to write in a lisp was that we’d have problems finding developers. It turned out to be true in that we received fewer applications, but compared to the same process with a more mainstream language, I’d say the applicants were on average of higher quality. So by the final step, we were approximately considering the same number of résumés in both cases.
I can echo this statement with Rust. Same story. Our comparison was Python to Rust. Same app. Big leap, i know.
I can echo this statement with Rust. Same story. Our comparison was Python to Rust. Same app. Big leap, i know.
I think requiring your startup in lisp also keeps the type of engineers away you don’t want.
In all of my career, it’s not those who know who have been good. It’s those who know how to learn new things fast that are good. It’s people who problem solve that make good engineers.
In the inverse. If somebody has a negative reaction to lisp, it’s a sign of a poor engineer.
In all of my career, it’s not those who know who have been good. It’s those who know how to learn new things fast that are good. It’s people who problem solve that make good engineers.
In the inverse. If somebody has a negative reaction to lisp, it’s a sign of a poor engineer.
It is not poor engineering to prefer a statically typed language. If you want something exotic I would take Haskell over lisp any day.
Thanks for the insight. I've had a similar experience with using Haskell. It is a smaller pool of developers, but they tend to be higher quality on average.
It's probably not the way you're making it sound. This was in the early early days of YC, when there were like eight companies per batch, and at the peak of PG's influence as an online essayist. He wrote a lot about the benefits of Lisp and how Lisp was a competitive advantage for Viaweb. So it was probably less a case of trying to win favor with a seed investor and more a case of actually being convinced it was a good business decision.
Arguably the value of Lisps has been diminished quite a lot with the advent of highly dynamic "multi-paradigm" scripting languages like Ruby and Python. The "Lisp superpower" has traditionally been that small, tightly coupled groups (quite often n=1, the most tightly coupled group) can achieve a lot very quickly without wasting a lot of time on "boilerplate". Those scripting languages enable largely the same thing.
While I'm sure there are startups using Lisp, Clojure might be the most widely used List dialect amongst startup.
i have very very limited experience, but at the time lisp still had some values that weren't mainstream
that said a language can push you to the moon or help you crash faster, team culture and fit will matter more IMO
that said a language can push you to the moon or help you crash faster, team culture and fit will matter more IMO
I was pleasantly surprised when I stumbled on the Abuse source code a while ago and started reading through it, I've seen other people talk about it here but here's a link:
https://github.com/videogamepreservation/abuse
https://github.com/videogamepreservation/abuse
Time flies. I still remember when there were quite some discussions in the communities when the initial reddit team made it public that they were switching from Lisp to Python for development. It created a lot of controversy, especially the specific reasons they named. I think quite some of them happened here on hacker news. Back then of course reddit was just a tiny startup as many.
So it is funny to see this now popping up as a news item. Shows, how long this is in the past and how much reddit has grown in size.
So it is funny to see this now popping up as a news item. Shows, how long this is in the past and how much reddit has grown in size.
jychang(1)
MathB.in, a simple pastebin for mathematical snippets, went in the opposite direction. It was first written in PHP and then later rewritten in Common Lisp. See https://susam.net/blog/mathbin-turns-ten.html for the full story.
I would argue that for such a small website (in the sense of both codebase size and traffic), which language is used does not matter, and is almost purely a personal preference.
Yes, it was personal preference indeed. I have written a little more about that here: https://news.ycombinator.com/item?id=32980708
I've tried mathb and it feels amazing and blazing fast. Perhaps the best way to go for typing in quick snippets and sharing with others.
I checked out MathB.in. Neat stuff!! I've started and stopped with the Lisp/Schema family several times. When other languages frustrate me, I go back to lisp for some therapy.
It was announced in comp.lang.lisp most early adopters were lisp users.
The first Reddit drama was when Paul Graham was caught having a sock puppet account to defend his arguments :)
The first Reddit drama was when Paul Graham was caught having a sock puppet account to defend his arguments :)
> The first Reddit drama was when Paul Graham was caught having a sock puppet account to defend his arguments :)
Do you have a link to this thread?
Do you have a link to this thread?
wouldn’t surprise me if he has like 8 alt accounts here and on Twitter
How was he caught?
What language is HN written in ?
This appears to be a mirror of HN as it was 11 years ago. The site could've been entirely rewritten in something else since then.
Arc, which is a Lisp dialect.
Too obscure and weird for me, but quite interesting. :)
Too obscure and weird for me, but quite interesting. :)
Arc I believe.
OMG, Paul Graham actually released Arc in some form? I thought he never finished it.
I was going to make snarky comment about how between spez and PG, Lisp must actually be extended-release brain poison for people in executive management positions; now I wonder if spez just knows PG personally and takes his advice.
I was going to make snarky comment about how between spez and PG, Lisp must actually be extended-release brain poison for people in executive management positions; now I wonder if spez just knows PG personally and takes his advice.
Arc's Out - https://news.ycombinator.com/item?id=106398 - Jan 2008 (124 comments)
The current reddit seems to be written in js/python. I can see the logic behind js since its a lot easier to find devs and thanks to all the hard work of the engineers behind v8 its not a slow language. I do not understand why python was chosen though. If you need more speed you would go with c++ and not python which is the slowest popular language.
Anyone got any insight into why python was chosen?.
Anyone got any insight into why python was chosen?.
Because Spez can't read it, he can only parse shapes.
https://www.youtube.com/watch?v=oq7DEUhr7o0
https://www.youtube.com/watch?v=oq7DEUhr7o0
That’s not what he said. He’s actually making an interesting point about readable code. I’m not defending his current behavior, just these comments about readable code.
I can’t tell if this is an AI fake or not, and that scares me.
It was uploaded 9 years ago. Definitely real.
It was chosen because at the time because python was much more mature and productive. They didnt use Django, but as an example Django is still more productive than any Node framework I've tried thus far, too, and we have types now. :)
There's a post explaining exactly why they moved from CL to Python, and it was basically because of library availability.
https://web.archive.org/web/20060206185841/http://reddit.com...
Why they moved away from CL:
"If Lisp is so great, why did we stop using it? One of the biggest issues was the lack of widely used and tested libraries. Sure, there is a CL library for basically any task, but there is rarely more than one, and often the libraries are not widely used or well documented. Since we're building a site largely by standing on the shoulders of others, this made things a little tougher. There just aren't as many shoulders on which to stand."
And why Python was chosen:
"So why Python?
We were already familiar with Python. It's fast, development in Python is fast, and the code is clear. In most cases, the Lisp code translated very easily into Python. Lots of people have written web applications in Python, and there's plenty of code from which to learn. It's been fun so far, so we'll see where it takes us."
So, basically, they were happy with CL except for lack of libraries, and because they also knew Python already, and they knew there were more libraries for Python, they picked it. Simple as that.
https://web.archive.org/web/20060206185841/http://reddit.com...
Why they moved away from CL:
"If Lisp is so great, why did we stop using it? One of the biggest issues was the lack of widely used and tested libraries. Sure, there is a CL library for basically any task, but there is rarely more than one, and often the libraries are not widely used or well documented. Since we're building a site largely by standing on the shoulders of others, this made things a little tougher. There just aren't as many shoulders on which to stand."
And why Python was chosen:
"So why Python?
We were already familiar with Python. It's fast, development in Python is fast, and the code is clear. In most cases, the Lisp code translated very easily into Python. Lots of people have written web applications in Python, and there's plenty of code from which to learn. It's been fun so far, so we'll see where it takes us."
So, basically, they were happy with CL except for lack of libraries, and because they also knew Python already, and they knew there were more libraries for Python, they picked it. Simple as that.
NodeJS didn't even exist when Reddit started using Python, as far as I can tell.
Initial release of NodeJS was May 27, 2009; 14 years ago.
Swartz was involved in the development of Reddit until he departed from the company in 2007.
And we know that parts of the Python code for Reddit was written by Swartz.
Initial release of NodeJS was May 27, 2009; 14 years ago.
Swartz was involved in the development of Reddit until he departed from the company in 2007.
And we know that parts of the Python code for Reddit was written by Swartz.
Wow, that sounds so early to adopt Python!
Python was released in 1991.
And Ruby in 1995, but it wasn't before 2005 (and Rails) that it reached traction. In 2007, albeit 16 years old, Python was still far from mainstream, especially out of its scripting niche.
That doesn’t match my recollection. I’m not sure how to prove anything, but I’ll note that Python 3.0 was released in 2008. If Python 2 wasn’t already extremely popular and well-established at that point, I doubt we’d have seen the brutal 10+ish year migration from it.
It depends how you define "mainstream".
Python was definitely around, had a very strong community, and a number of very significant programs written in it. (Among them: the first versions of BitTorrent and Google.) However, if you were a random dev applying for a random job, you would most likely have to work in C++, Java, or C#. It was relatively difficult getting a job where you would write the majority of your code in Python. Certainly they existed, and many jobs would have you write small one-off scripts in Python, but the actual core product would usually be one of Java/C#/C++.
Remember that in 2009, Google had 20K employees while IBM (which was a major booster of Java) had 400K. And at Google, Python wasn't really allowed for major projects (Search had been rewritten from Python to C++ when it got big in 2000), so you'd be using Java or C++ anyway. Data science wasn't really a thing, and if you did machine learning you used frameworks like Weka or home-grown stuff.
Python was definitely around, had a very strong community, and a number of very significant programs written in it. (Among them: the first versions of BitTorrent and Google.) However, if you were a random dev applying for a random job, you would most likely have to work in C++, Java, or C#. It was relatively difficult getting a job where you would write the majority of your code in Python. Certainly they existed, and many jobs would have you write small one-off scripts in Python, but the actual core product would usually be one of Java/C#/C++.
Remember that in 2009, Google had 20K employees while IBM (which was a major booster of Java) had 400K. And at Google, Python wasn't really allowed for major projects (Search had been rewritten from Python to C++ when it got big in 2000), so you'd be using Java or C++ anyway. Data science wasn't really a thing, and if you did machine learning you used frameworks like Weka or home-grown stuff.
> I’m not sure how to prove anything, but I’ll note that Python 3.0 was released in 2008. If Python 2 wasn’t already extremely popular and well-established at that point, I doubt we’d have seen the brutal 10+ish year migration from it.
This is about the “scripting niche” I was talking about: in 2008 every Linux distro included Python and lots of build scripts were written in Python 2, but Python for the back-end was really rare back then.
This is about the “scripting niche” I was talking about: in 2008 every Linux distro included Python and lots of build scripts were written in Python 2, but Python for the back-end was really rare back then.
Yeah python for the backend started taking off with Django, which only released 1.0 and picked up steam in 2008.
Django was already very good well before 1.0, and had at a minimum some of the best documentation out there. I seem to recall a decent community, video tutorials, etc.
That's not true. We were using Python 1.5.2 at a well-funded startup I was at from 2000-2007. We were an early cloud provider and maybe 1/3rd of our customers were using Python. This was in the 3-tier stack days and Zope was fairly popular for the application server.
The internal tooling we built was mostly in Python.
The internal tooling we built was mostly in Python.
I'd used python on 3 commercial apps by 2007. One for Boeing, one for the navy one for an insurance company. Ruby was a little different. Ruby without rails had no real standards for making big apps. And I'm not sure when gems was made at rails con by Seattle ruby. Rails was for a long time a mega library for ruby... Ruby was missing a ton of features without rails. Similar but worse than a node web app versus working in a react app. No hot reload, no lib directory, many other features.
Aaron Swartz replaced the Lisp with web.py
Yes, web.py was his baby. He also had some markdown parser stuff.
Baby boomers always tell me about seeing The Doors live in dive bar in NYC. I can say I used Reddit when it was written in Lisp. :-)
Baby boomers always tell me about seeing The Doors live in dive bar in NYC. I can say I used Reddit when it was written in Lisp. :-)
Back then to us smaller forums goers, Reddit was the new kid in the playground that we didn't like because somehow he was viewed as "cooler" than the other kids.
Even years after, when I had stopped posting on those PHP powered smaller forums I resisted creating a reddit account just because it was "reddit".
I would say, the smaller forums sites were a more tightly knitted group that I attended several meetups with. Reddit just never felt the same.
Even years after, when I had stopped posting on those PHP powered smaller forums I resisted creating a reddit account just because it was "reddit".
I would say, the smaller forums sites were a more tightly knitted group that I attended several meetups with. Reddit just never felt the same.
I'm not sure why python was chosen.
But I contributed a tiny bit of code to reddit back when it was open source and chatted with reddit engineers on IRC, so I do know the official justification (as of 2010ish) for reddit sticking with python:
Websites are mostly IO bound anyway. The hot parts like markdown formatting and templating are already c modules. So the performances gain for rewriting in a compiled language aren't large enough to justify it.
But I contributed a tiny bit of code to reddit back when it was open source and chatted with reddit engineers on IRC, so I do know the official justification (as of 2010ish) for reddit sticking with python:
Websites are mostly IO bound anyway. The hot parts like markdown formatting and templating are already c modules. So the performances gain for rewriting in a compiled language aren't large enough to justify it.
That seems reasonable to me, even today. Am I off-base?
Incredibly reasonable. People love to trash on the performance of Python and JS saying that they're totally unsuitable for backend services with non-trivial amounts of traffic when they're usually the most cost effective solution for a business. These higher level languages are easier to hire for, much easier to prototype, allow for faster iteration, allow for substantially faster on-ramp time, and are fast enough to run these io bound workloads.
I have had the opposite experience with every python team I have interacted with.
On top of it being slow and brittle, dependencies break, needlessly destabilizing stuff.
Worse, the python team ends up having the hardest job, due to self-inflicted problems, so it either ends up with junior devs that don’t know better, or bitter senior devs that could be 10x more productive doing something else.
As always, it varies from company to company, but this is what I saw on four teams out of four at multiple companies.
On top of it being slow and brittle, dependencies break, needlessly destabilizing stuff.
Worse, the python team ends up having the hardest job, due to self-inflicted problems, so it either ends up with junior devs that don’t know better, or bitter senior devs that could be 10x more productive doing something else.
As always, it varies from company to company, but this is what I saw on four teams out of four at multiple companies.
> dependencies break
You can abuse dependencies in every language. This doesn't sound like a Python problem but a bad tech management problem (i.e. who signed off on allowing 'randomguy69/left-pad' as a dependency).
You can abuse dependencies in every language. This doesn't sound like a Python problem but a bad tech management problem (i.e. who signed off on allowing 'randomguy69/left-pad' as a dependency).
With Python it doesn't take abuse. You blink and the bloody thing rots away.
Relative to what other language?
we have account servers for a game in java that end up being cpu bound (we have to scale up based on cpu, not network), which is much much faster than python.
so i would be surprised if a server written in python could saturate the network, for a reddit-style workload, which i imagine would be similar.
any1 have relecent experiences to share?
so i would be surprised if a server written in python could saturate the network, for a reddit-style workload, which i imagine would be similar.
any1 have relecent experiences to share?
reddit never cared much about performance, they blame it on IO but they chose to use cassandra, they used it as a key value store, then they put python on top of it, with the result that most pages would take seconds to generate and the website would go down almost every day.
With the "new" reddit they replaced most of the frontend with javascript and it really shows. That's not to say I like sites that use too much javascript but python is slow enough just parsing and filling html templates, seemingly.
With the "new" reddit they replaced most of the frontend with javascript and it really shows. That's not to say I like sites that use too much javascript but python is slow enough just parsing and filling html templates, seemingly.
I have a theory (which nobody I ever met shares) that the visually most beautiful language is the one that long-term results in the fastest development of a project:
Visual noise is like inflation. A constant tax on what you do. But you barely notice it. But it accumulates because it is constantly leading to slightly worse and more complex code. Since it is harder to focus on a noisy something in front of you, you make slightly worse decisions. Over time this accumulates. And similar to how Warren Buffet noticed that a Dollar of his youth is not even worth a penny these days, over time, exponentially more effort is needed to develop the more and more complex system.
So over a longer timeframe, a visually more elegant language wins by a long shot.
Python is the visually most elegant language.
Visual noise is like inflation. A constant tax on what you do. But you barely notice it. But it accumulates because it is constantly leading to slightly worse and more complex code. Since it is harder to focus on a noisy something in front of you, you make slightly worse decisions. Over time this accumulates. And similar to how Warren Buffet noticed that a Dollar of his youth is not even worth a penny these days, over time, exponentially more effort is needed to develop the more and more complex system.
So over a longer timeframe, a visually more elegant language wins by a long shot.
Python is the visually most elegant language.
Those hypothetical tiny gains are instantly and completely eclipsed by the lack of static checking support.
__visuallyelegant__
That should trigger the good sense of any “Zen of Python” die-hard. Not saying I haven’t used/got access to double-underscore thingies in Python in my 15+ years of writing Python code, but I always felt dirty in doing that.
But isn’t that the point? The naming convention is for protocols that have syntax support
Scheme has entered the chat.
> Python is the visually most elegant language.
Having program login buried in a nauseating amount of boilerplate doesn’t feel very “visually elegant” :-P
Having program login buried in a nauseating amount of boilerplate doesn’t feel very “visually elegant” :-P
What you’re saying is only partially correct - Yes, for more exploratory programming for a new product with full US/EU/CANZUK based dev workforce it makes sense to prioritize maintainability.
For a company complaining about infrastructure costs for a product that has been cloned many times over, and which is large enough to hire offshore devs, the cost equation favours rewriting in something like Go, Rust or even Java. These are quite tailored towards Reddit’s current use case and many companies including Twitter did that rewrite.
For a company complaining about infrastructure costs for a product that has been cloned many times over, and which is large enough to hire offshore devs, the cost equation favours rewriting in something like Go, Rust or even Java. These are quite tailored towards Reddit’s current use case and many companies including Twitter did that rewrite.
Using a language with a faster runtime makes the system run faster. But it does not compound. So you are limited to a ceiling at about an order of magnitude faster execution.
Making better decisions on a daily basis compounds. You evolve towards a better architecture. Which can make the code multiple orders of magnitude faster.
One of many examples: In a complex system, developers often have a blurry vision of what can be cached, in which situations it can be cached, and how cache invalidation should be done. While in a more elegant system, the borders between different approaches to caching are clearer. Leading to the more elegant system being multiple orders of magnitude faster as it can leverage the different caching layers (in memory cache, on disk cache, the CDN ...) more effectively.
Making better decisions on a daily basis compounds. You evolve towards a better architecture. Which can make the code multiple orders of magnitude faster.
One of many examples: In a complex system, developers often have a blurry vision of what can be cached, in which situations it can be cached, and how cache invalidation should be done. While in a more elegant system, the borders between different approaches to caching are clearer. Leading to the more elegant system being multiple orders of magnitude faster as it can leverage the different caching layers (in memory cache, on disk cache, the CDN ...) more effectively.
This doesn't sound like argument in favor of Python, though. In my experience, Python code quickly reaches the level of an impenetrable word salad, at which point all the compounding stops.
This depends on your editor color settings though. C++ can look very handsome
C++ looks very handsome.
Usually someone will pop up showing crazy meta programming template code and use it a straw man to declare cpp=bad.
Usually someone will pop up showing crazy meta programming template code and use it a straw man to declare cpp=bad.
Which is why elegance argument would've favored staying with Lisp - not only it looks elegant, but also you can do all the "crazy metaprogramming" without the crazy, but rather in straightforward and elegant code.
And I say that as someone doing C++ for a living, and occasionally engaging in said crazy template metaprogramming stuff. Occasionally, because people tend to frown at it during code review; apparently, metaprogramming is reserved only to library authors...
And I say that as someone doing C++ for a living, and occasionally engaging in said crazy template metaprogramming stuff. Occasionally, because people tend to frown at it during code review; apparently, metaprogramming is reserved only to library authors...
I've bee using C++ since the late 90s, and namespaces are still annoying.
I agree with you. C++ and Python are great. Haskell looks the best but it’s way too tiring to write. Go is the worst offender.
Do you actually think CPU usage is a concern here? Database performance and optimization, horizontal scaling, cache optimization (and cache invalidation) are the big problems. Security is also a big problem.
The performance critical part is the database, and that at least used to be written in Java (cassandra db).
Writing the bit of glue code between your high performance database and your fronted in cpp would introduce all kinds of really bad potential bugs, scary remote code execution bugs.
CPU usage of that glue code might take up 10% of your hardware budget worst case, even with a "slow" language. It's just not a concern.
The performance critical part is the database, and that at least used to be written in Java (cassandra db).
Writing the bit of glue code between your high performance database and your fronted in cpp would introduce all kinds of really bad potential bugs, scary remote code execution bugs.
CPU usage of that glue code might take up 10% of your hardware budget worst case, even with a "slow" language. It's just not a concern.
We're not talking about 10% factors, but 10x factors between slow and fast languages. People say CPU usage doesn't matter when you're horizontally scaling, but horizontal scaling increases your ongoing operational costs, and you often are horizontally scaling precisely because your code is too slow.
What percentage of cpu time do you think would be spent in python land?
In my experience one of the limiting factors of a language as you scale is library support. You want to do more complex things (for better or worse) or integrations (monitoring, etc.) but the development cost is very high since you have to write the whole part yourself or make tradeoffs on the limited libraries available.
Ummm...same reason as JS? Why do you think they need more speed to the point they need c++? Do you think finding c++ devs easier than python devs?
> Ummm...same reason as JS? Why do you think they need more speed to the point they need c++?
Well, given complaints how much 3rd party apps cost them, they’d better off optimize their crap first.
Well, given complaints how much 3rd party apps cost them, they’d better off optimize their crap first.
It's not really about cost, it's about convincing investors CEO dude is in control and can make reddit profitable for their planned IPO. "We spent a shitton of money migrating to C++ and hiring expensive and harder to replace C++ talent to optimize shit" is not a good selling point.
I know what their plan is, just pointing out their hypocrisy.
I imagine because it's obviously "fast enough" performance-wise, and an order of magnitude faster dev-wise and hiring-wise.
Not fast enough to keep their operational at bay though.
Here is the twitter thread about it: https://twitter.com/ShriramKMurthi/status/166987750603426611...
The source code is much more interesting than some Twitter bro's hot take.
"We took a wrong turn w/ software."
... apparently.
Save your energy and check out the repo instead.
I don't totally blame the guy though. If you want to play the stupid Twitter game, you have to add a trite but provocative hot take to every post you make, lest your writings languish in obscurity. The only way to win is not to play, truly.
"We took a wrong turn w/ software."
... apparently.
Save your energy and check out the repo instead.
I don't totally blame the guy though. If you want to play the stupid Twitter game, you have to add a trite but provocative hot take to every post you make, lest your writings languish in obscurity. The only way to win is not to play, truly.
Takes me back .. once upon a time I was an admin on old OG Freenode + #C + #lisp and recall when that was floated to take potshots at and use as a beta test crowd.
The first non reddit team submissions were some very dry comp sci papers and tech manuals .. the cats came not long after.
The first non reddit team submissions were some very dry comp sci papers and tech manuals .. the cats came not long after.
He is not a Twitter bro though. He has developed software extensively. He is a PL researcher, a professor at BrownU, now working in computing education.
Thanks for the link. Among other things it made be feel old, as in I have a post on my blog from back then writing about the Lisp to Python switch that Reddit was carrying out and it’s interesting to see computer programmers that look like they’re in their early 30s or such and not knowing about it at all.
Those were more interesting and especially more vibrant times when it came to the web. Granted, the money was not as abundant as it is now.
Those were more interesting and especially more vibrant times when it came to the web. Granted, the money was not as abundant as it is now.
[deleted]
Would be fun to see what it takes to run it!
DB schema is missing. If there was an .sql script to create all the tables, I could run this reddit version.
Most of the information about columns and tables could be recovered from data.lisp and view-defs.lisp. Maybe also old.lisp
Most of the information about columns and tables could be recovered from data.lisp and view-defs.lisp. Maybe also old.lisp
You'd also have to find a copy of the web framework it uses because it got renamed and the author disabled the old download address.
The tbnl web server was indeed renamed to hunchentoot.
I found the old tbnl versions in the wayback machine. One of the snapshots: https://web.archive.org/web/20080131152054/http://www.weitz....
Most likely adapting to the current hunchentoot would not be difficult too.
I found the old tbnl versions in the wayback machine. One of the snapshots: https://web.archive.org/web/20080131152054/http://www.weitz....
Most likely adapting to the current hunchentoot would not be difficult too.
It has been done \o/ https://github.com/tamurashingo/reddit1.0/
Cool. So db schema is actually present, in the form of clsql clas definitions (the view-defs.lisp)
Here's an updated fork with DB migrations: https://github.com/tamurashingo/reddit1.0/
scraper.lisp contains parameters for a "delicious-scanner" to http://del.icio.us/popular/ ... also something which is gone / replaced
The original del.icio.us was pretty good, it got weird later on.
wow, now that's a website I haven't thought of in a while
> (defvar database-password "pgcwip42:")
Hope they changed that password!
Hope they changed that password!
[deleted]
>"written in Lisp"
And why does that matter? Whatever turns one on.
And why does that matter? Whatever turns one on.
It matters for the same reason that "X written in Rust" or "which companies are using Lisp?" matters: some people refuse to give Lisp a look unless every F500 company has at least 3 major projects written in it.
>"some people refuse to give Lisp a look"
Nothing is wrong with this. There are crapload of languages and very little benefit from a practical standpoint for a single person to use them all. They're just tools. Some are better, some are worse and some are highly specialized. What is being built is important, not how.
Nothing is wrong with this. There are crapload of languages and very little benefit from a practical standpoint for a single person to use them all. They're just tools. Some are better, some are worse and some are highly specialized. What is being built is important, not how.
hcks(1)
Another comment, from Paul Graham:
1. http://lemonodor.com/archives/001301.html