Python Developers Survey 2019 Results(jetbrains.com)
jetbrains.com
Python Developers Survey 2019 Results
https://www.jetbrains.com/lp/python-developers-survey-2019/
64 comments
Due to roughly exponential growth in the number of developers of all kind over last couple of decades it is true that at any point most developers have very little experience. I don't remember what was doubling time but I think it was more than two years but less than 5 years. It as also easy to see that the growth is already flattening out with some interesting repercussions in the near future (experienced/inexperienced developers ration finally getting better, etc.)
Doubling time shorter for python developers could be explained to be a result of python getting more popular (so even though programming gets more popular as a job, python grows even faster within developer community).
Doubling time shorter for python developers could be explained to be a result of python getting more popular (so even though programming gets more popular as a job, python grows even faster within developer community).
> Due to roughly exponential growth in the number of developers of all kind over last couple of decades
That is certainly true until ~2010, but I wonder if it still holds today. At some point it's got to taper off due to the global population being finite. Anyone got some data on this?
That is certainly true until ~2010, but I wonder if it still holds today. At some point it's got to taper off due to the global population being finite. Anyone got some data on this?
I interview a decent number. At least in my area, college grads have leveled off. There's still a ton of them, maybe 5% growth a year and a lot more applying for first jobs. But its a steady percentage.
There's been a huge jump in code camp grads. Gigantic. And unfortunately the vast majority are not great. We give them coding assignments first, otherwise there's just too many to even sort through. I would say over half are unable to complete the (very simple) coding assignment. Build 3 API endpoints in any language that can do a couple different sorts and limits of a static set of data in CSV.
There's been a huge jump in code camp grads. Gigantic. And unfortunately the vast majority are not great. We give them coding assignments first, otherwise there's just too many to even sort through. I would say over half are unable to complete the (very simple) coding assignment. Build 3 API endpoints in any language that can do a couple different sorts and limits of a static set of data in CSV.
I work in a large corp as a senior developer/tech lead. I interview 1-2 candidates every week. These are typically people who either passed screening or are already employed by an external company. Unfortunately, I have little influence on that process (remember... Big Co...)
Because I typically hire for more senior roles I get candidates with 5 years of experience at the minimum.
Roughly 4 out of 5 candidates do not know answer to ANY of these questions:
- What is virtual memory?
- Can you pass data to another process by passing a pointer to the data? (tricky, but generally processes have separate address spaces unless you put extra effort to have part of the space mapped to same address, using shared memory, mmap, etc.)
- Can you explain what is a breakpoint? (an instruction injected by debugging program that stops execution and causing an interrupt that causes OS to call debugger)
- Can a process allocate more memory than is physically available on the host?
- Is inserting random integers into a huge sorted array faster or slower than inserting into singly linked list (absent any indexes)? (Inserting to array list will require huge data copy but linear search over linked list is so much slower it will cause linked list to be slower than array every time).
- Please, write a program to group an input list of words into lists of anagrams.
Etc.
Because I typically hire for more senior roles I get candidates with 5 years of experience at the minimum.
Roughly 4 out of 5 candidates do not know answer to ANY of these questions:
- What is virtual memory?
- Can you pass data to another process by passing a pointer to the data? (tricky, but generally processes have separate address spaces unless you put extra effort to have part of the space mapped to same address, using shared memory, mmap, etc.)
- Can you explain what is a breakpoint? (an instruction injected by debugging program that stops execution and causing an interrupt that causes OS to call debugger)
- Can a process allocate more memory than is physically available on the host?
- Is inserting random integers into a huge sorted array faster or slower than inserting into singly linked list (absent any indexes)? (Inserting to array list will require huge data copy but linear search over linked list is so much slower it will cause linked list to be slower than array every time).
- Please, write a program to group an input list of words into lists of anagrams.
Etc.
HN community is quite self selected. The average engineer I've met chose their career for the money and isn't interested in much that's not right in front of them
No hard data to share but there are 2~3B people coming online still, so I doubt the S-curve is leveling off yet.
The results may include a lot of people who don't consider themselves to be professional developers at all, such as myself. While I've been programming for almost 40 years, I've never been employed as a programmer per se. I've always used programming as a problem solving tool, though some of my code has gone into production, usually by default because there was no coder available or willing to take it up.
Oddly enough I first tried Python because of the "buzz" that it was getting on HN.
Oddly enough I first tried Python because of the "buzz" that it was getting on HN.
Do people jaded by years of experience fill out as many surveys?
Sounds like they have less than 2 years professional experience, period. That might influence other results even more when it's not about experienced developers who have recently started python, but about developers who have recently started their job.
Python is the language of choice for schooling these days. As Java once was. One could argue that Python is much better suited for it.
This may mean that low level language skills become rare though. Besides memory management, Java is kindred of C. Many years ago schools started students in Perl and PHP, eventually giving way to Java. Now they're back on dynamic languages. Will the pendulum swing back the other way when schools realize that graduates are missing many skills needed to work with lower level statically typed languages? I guess time will tell..
This may mean that low level language skills become rare though. Besides memory management, Java is kindred of C. Many years ago schools started students in Perl and PHP, eventually giving way to Java. Now they're back on dynamic languages. Will the pendulum swing back the other way when schools realize that graduates are missing many skills needed to work with lower level statically typed languages? I guess time will tell..
You could download the raw data and look at the results of those with more experience ;)
Glad to see Flask surpass Django, it's always been my favorite web framework regardless of language and forces users to learn their style of software architecture. I personally prefer MVC: https://github.com/esteininger/flask-mvc-boilerplate
All large flask implementations eventually turn into a crappy version of Django.
I prefer Flask and I agree. However there is a big difference. If you learn how to turn flask into that crappy version of Django, you will thank yourself because you know what choices the Django Devs took. You will know what each section of a Django project is, and why it is that way, or how you can make it follow something nonstandard.
I have interviewed plenty of flask and Django Devs, and usually, the Django Devs reply to some of my questions is usually "that's how Django does it"
That is a poor, poor answer.
I have interviewed plenty of flask and Django Devs, and usually, the Django Devs reply to some of my questions is usually "that's how Django does it"
That is a poor, poor answer.
Oh god. I would not recommend that git repo to anyone. It is doing a lot wrong.
Look at exploreflask.com for a great example on how to structure production grade flask applications. Read the flask docs, the later sections deal with the create_app factory function and how to use it. It is really good in that sense.
Look at exploreflask.com for a great example on how to structure production grade flask applications. Read the flask docs, the later sections deal with the create_app factory function and how to use it. It is really good in that sense.
I was happy to see that Tornado is quite stable in the third place; it's probably the best Python framework currently in existence. Personally I have been moving to Starlette, but it's still far from Tornado's stability.
I have always enjoyed sanic for my python apis
https://sanic.readthedocs.io/en/latest/
https://sanic.readthedocs.io/en/latest/
Sure, just be sure to avoid threadlocals and pass them around explicitly!
threadlocals = app.context ?
Agreed, but that's the beauty of Flask you can just pass around the request context.
Agreed, but that's the beauty of Flask you can just pass around the request context.
flash, g, redirect, render_template, request, session etc.
Ideally request would be passed to view callables and would not be global. That is completely incompatible change so we will never see that I guess :(
Ideally request would be passed to view callables and would not be global. That is completely incompatible change so we will never see that I guess :(
Is it only my, or the color scheme for some plots (e.g. Python version) is atrocious from the UX perspective? (Small differences in shades, making it hard to read.)
good UX != pretty colors
good UX != pretty colors
[deleted]
Well... type hinting does not seem to be very popular
At 65% of developers using it at least sometimes, it seems surprisingly widespread.
Now obviously this survey is going to be somewhat self selected among devs that like to keep up to date and try new things, but 65% having used it in some capacity is pretty comparable to Typescript: https://2019.stateofjs.com/javascript-flavors/typescript/
Now obviously this survey is going to be somewhat self selected among devs that like to keep up to date and try new things, but 65% having used it in some capacity is pretty comparable to Typescript: https://2019.stateofjs.com/javascript-flavors/typescript/
The survey has 75% of users on python 3 in 2017, when a good half of popular libraries still didn't support python 3 at all.
It's clearly skewed toward devs on the bleeding edge.
It's clearly skewed toward devs on the bleeding edge.
I mostly use Python for web development and I've stopped using Python 2 for that with 3.5, so in early 2016, but I do remember that some machine learning or data analysis tools were stuck on Py2 for a little longer.
I definitely wouldn't say that half of popular libraries in general didn't support python 3 at all back then, but for some use cases that may have been true tho.
I definitely wouldn't say that half of popular libraries in general didn't support python 3 at all back then, but for some use cases that may have been true tho.
> good half of popular libraries still didn't support python 3
I don't know where you took your number from, but I don't think that's the case anymore.
Also, a lot of once-popular libraries were succeeded by now-popular libraries that run with Python 3.
I don't know where you took your number from, but I don't think that's the case anymore.
Also, a lot of once-popular libraries were succeeded by now-popular libraries that run with Python 3.
According to https://python3wos.appspot.com/ there were already 50%+ of the most popular packages compatible with Python 3 by the end of 2012
I treat it as a form of documentation, a step up from docstrings, with the nice touch some tools can actually test it.
I started adding type hints to some Python code the other day and something in my brain was like, "WTF!? Just use Haskell!"
When you add types to Python code it tends to destroy the elegance and readability of it, which is Python's main advantage over other languages.
If you're paying the overhead of adding type information already, you might as well use a language like Haskell that makes it first class and unlocks a ton of power and elegance (and convenient tooling.)
Python has a sweet spot but IMO explicit typing draws it away from that. Python is a very complicated and sophisticated semantics that acts like a simple and easy semantics. It's a delicate balance.
When you add types to Python code it tends to destroy the elegance and readability of it, which is Python's main advantage over other languages.
If you're paying the overhead of adding type information already, you might as well use a language like Haskell that makes it first class and unlocks a ton of power and elegance (and convenient tooling.)
Python has a sweet spot but IMO explicit typing draws it away from that. Python is a very complicated and sophisticated semantics that acts like a simple and easy semantics. It's a delicate balance.
I’m trying to use them more and they are helpful but they can be tedious because now I sometimes have to remember to import the right typing modules and I’m not very comfortable with typed sequences or union types. Anyone have a good cheat sheet or reference?
I’m adding them to code I’m refactoring and it definitely helps there. Instead of a random named variable I can now get completions for the thing I’m working on and dependencies are a lot easier to see and use.
I’m adding them to code I’m refactoring and it definitely helps there. Instead of a random named variable I can now get completions for the thing I’m working on and dependencies are a lot easier to see and use.
Happy to see Rust crawl in into top languages list :)
web development seems to be twice as popular as ML use. Anyone knows why?
Given node.js is known to be much faster while still offering the dev speed of a scripting language. I love python for algorithms and DevOps scripts but feel afraid of using for web as it might end up with huge cost for hosting (higher CPU power)
Given node.js is known to be much faster while still offering the dev speed of a scripting language. I love python for algorithms and DevOps scripts but feel afraid of using for web as it might end up with huge cost for hosting (higher CPU power)
It's usually not the language itself that makes dev speed faster. Of all the time I spend building a reasonably sane Java app, the time I spend recompiling/restarting it is a small fraction.
Rather, it's the programming models put in place by the initial framework choices and architectural decisions that will make the biggest difference. Prototyping a web app in Python isn't fast because Python as a language makes it fast. It's fast when you use Django to effectively wire a database model directly to a set of CRUD endpoints in less than a dozen lines of code, complete with validation. The same can be said about Ruby + Rails.
Node could, in theory, provide a similar capability, but the platform was popularized after large sections of the developer community seemed to demonize frameworks like Django and Rails. So, whether for this reason or another, Node doesn't really have anything quite like Django or Rails, and thus would likely not match the dev speed of those frameworks, all else being equal. Python is _fast enough_, and early stage companies rightfully tend to not worry about CPU usage or hosting costs, because they don't have the scale for it to matter.
------
With all that being said: pick an early tech stack that helps you optimize for iteration speed, not necessarily raw dev speed (although they often overlap). Importantly, you typically want to build a system that minimizes the cost of being wrong in a reasonable time frame. If that's Python? Awesome. If that's Node? Also awesome. If that's Java? Awesome. Pick what you/your team know best and can architect in a sane way to help you find the balance between time to market and ability to experiment.
Rather, it's the programming models put in place by the initial framework choices and architectural decisions that will make the biggest difference. Prototyping a web app in Python isn't fast because Python as a language makes it fast. It's fast when you use Django to effectively wire a database model directly to a set of CRUD endpoints in less than a dozen lines of code, complete with validation. The same can be said about Ruby + Rails.
Node could, in theory, provide a similar capability, but the platform was popularized after large sections of the developer community seemed to demonize frameworks like Django and Rails. So, whether for this reason or another, Node doesn't really have anything quite like Django or Rails, and thus would likely not match the dev speed of those frameworks, all else being equal. Python is _fast enough_, and early stage companies rightfully tend to not worry about CPU usage or hosting costs, because they don't have the scale for it to matter.
------
With all that being said: pick an early tech stack that helps you optimize for iteration speed, not necessarily raw dev speed (although they often overlap). Importantly, you typically want to build a system that minimizes the cost of being wrong in a reasonable time frame. If that's Python? Awesome. If that's Node? Also awesome. If that's Java? Awesome. Pick what you/your team know best and can architect in a sane way to help you find the balance between time to market and ability to experiment.
Javascript & co is not the most popular among backend developers.
I for example wouldn't even think about doing anything in it (or in Typescript).
Look at https://insights.stackoverflow.com/survey/2019#technology-_-... Python is not on the list of most dreaded languages, but Javascript is.
Look at https://insights.stackoverflow.com/survey/2019#technology-_-... Python is not on the list of most dreaded languages, but Javascript is.
Mention Python in a HN thread, and prepare for people to start moaning. IDK, maybe it's the SV filter bubble dealing with some mono-repo from google with a bajillion lines of legacy Python to support. Yeah, I can't imagine how unwieldy that might be. But for those of you that extol the virtues of the 'safety' a compiled language brings I say, try supporting a Java app that's been held together with shoe-strings and bubble-gum for 15+ years (in the enterprise world). Same nightmare, but much harder to re-architect.
Everybody's starting to figure out that to get affordable development you need:
1. fast development (both Python and node)
2. boring platform where it's usually OK to leave smth unupgraded for >5 yrs (Python wins here hands down)
3. cheap devs (good Node.js devs have a lot of option to "jump around" between front and back and frameworks so higher room for leverage ...with Python you can hire cheapest newly grads since it's now taught in most unis)
...on top of this:
- performance in terms of "requests served per CPU power" does NOT matter for most apps (and when it does caching saves most situations) - and if it does, modern Python can deliver now (async is mature)
- using same stack as ML and datasci comes as a bonus - you can have backend-full-stack-devs jumping between data pipelines, APIs/web, ML models, data analysis code etc. who can stay separate from frontend-full-stack-devs doing web frontends + mobile hybrid etc.
It would be great if we could also standardize on a static high performance language common to all (eg. usable for special-performance-sensitive-code either standalone services or Python-modules or node-modules) ...this area's a mess, too much overlap, duplicated work, and too hard for devs to jump from one ecosystem to another (Rust and C++ as lib/extension langs... Java/Kotlin or Go or C#/F# for standalone services... now Swift an option too...).
1. fast development (both Python and node)
2. boring platform where it's usually OK to leave smth unupgraded for >5 yrs (Python wins here hands down)
3. cheap devs (good Node.js devs have a lot of option to "jump around" between front and back and frameworks so higher room for leverage ...with Python you can hire cheapest newly grads since it's now taught in most unis)
...on top of this:
- performance in terms of "requests served per CPU power" does NOT matter for most apps (and when it does caching saves most situations) - and if it does, modern Python can deliver now (async is mature)
- using same stack as ML and datasci comes as a bonus - you can have backend-full-stack-devs jumping between data pipelines, APIs/web, ML models, data analysis code etc. who can stay separate from frontend-full-stack-devs doing web frontends + mobile hybrid etc.
It would be great if we could also standardize on a static high performance language common to all (eg. usable for special-performance-sensitive-code either standalone services or Python-modules or node-modules) ...this area's a mess, too much overlap, duplicated work, and too hard for devs to jump from one ecosystem to another (Rust and C++ as lib/extension langs... Java/Kotlin or Go or C#/F# for standalone services... now Swift an option too...).
>(async is mature)
is Django or Flask async yet?
is Django or Flask async yet?
Why do ppl like Django and Flask so much? They're heavy awkward dinosaurs with nasty codebases perpetuating really bad coding patterns (globals?! (multiple)inheritance-till-your-head-spins?! - that's what that soup of whateverMixins is, let's stop pretending... serializers that DE-serialize and different stuff depending on situation?! ...stuff you couldn't add type annotations too because the types would depend of calling contexts far away etc. etc. etc.).
If you code Python, at least take some time to read more modern code: see responder (not my favorite but cool), fastapi, starlette, asyncio http examples etc... Or see older examples (as old as Django) like web.py.
If you code Python, at least take some time to read more modern code: see responder (not my favorite but cool), fastapi, starlette, asyncio http examples etc... Or see older examples (as old as Django) like web.py.
Also, check out FastAPI, which is quite nice. Fully async with a lot of other nice features:
https://fastapi.tiangolo.com/
https://fastapi.tiangolo.com/
Does that have feature parity with DjangoREST?
...what would "feature parity" even mean? There's no universal checklist of api frameworks "required" features.
Short answer: no, bc FastAPI makes no assumption about your ORM (bring whatever, use none... your responsibility, your choice - I use sqlalchemy core + databases module for async support, no ORM, but but full use of Pydantic for enforcing schemas, just "divorced" from persistence layer, maybe you don't even need any), or your anything else.
I actively dislike DjangoREST and tbh Django too... but I admit that in combination they can save A TON of time when prototyping something.
Short answer: no, bc FastAPI makes no assumption about your ORM (bring whatever, use none... your responsibility, your choice - I use sqlalchemy core + databases module for async support, no ORM, but but full use of Pydantic for enforcing schemas, just "divorced" from persistence layer, maybe you don't even need any), or your anything else.
I actively dislike DjangoREST and tbh Django too... but I admit that in combination they can save A TON of time when prototyping something.
Potentially soon for Flask, https://github.com/pallets/flask/pull/3412. Also see Quart, https://gitlab.com/pgjones/quart
Not that I'm aware of, but Tornado is a great async web framework. We use MongoDB and an ODM called umongo (that uses Motor for async, pymongo-like connections). I'm not sure what good async ORMs are available.
Almost for django:
https://docs.djangoproject.com/en/3.0/topics/async/
https://docs.djangoproject.com/en/3.0/topics/async/
Use Quart in place of Flask
I think ML is a bit niche in terms of overall number of programmers doing it? Meanwhile, making websites has turned out to be very popular.
I suspect the response rate there is actually skewed high by the large chunk of students.
Regarding the CPU use and hosting cost, it's probably not an issue for most websites. Usually you end up database bottlenecked much sooner, and that doesn't kick in until you're on >100s of millions of rows.
I suspect the response rate there is actually skewed high by the large chunk of students.
Regarding the CPU use and hosting cost, it's probably not an issue for most websites. Usually you end up database bottlenecked much sooner, and that doesn't kick in until you're on >100s of millions of rows.
I'm more surprised that ML is so widely used. Still quite special area IMHO.
> Given node.js is known to be much faster while still offering the dev speed of a scripting language.
Is this so? I consider python as a language and ecosphere far above javascript and nodejs. And how well can nodejs be optimized with non-js-code?
But regarding the numbers, likely it's because a python-shop wouldn't use nodejs and introduce an additional liability. Everyone usually chooses their prefered language and environment for a task as long as there is no significant advantage in using something different.
> Given node.js is known to be much faster while still offering the dev speed of a scripting language.
Is this so? I consider python as a language and ecosphere far above javascript and nodejs. And how well can nodejs be optimized with non-js-code?
But regarding the numbers, likely it's because a python-shop wouldn't use nodejs and introduce an additional liability. Everyone usually chooses their prefered language and environment for a task as long as there is no significant advantage in using something different.
I'm no expert in the matter, but AFAIK, concurrency is becoming relevant in Python web dev.
One can already use stuff like FastAPI [1] and Django is supposed to be getting async/await support later this year [2].
So that should make Python more competitive when compared to node.
[1]: https://fastapi.tiangolo.com/
[2]: https://docs.djangoproject.com/en/dev/releases/3.1/
One can already use stuff like FastAPI [1] and Django is supposed to be getting async/await support later this year [2].
So that should make Python more competitive when compared to node.
[1]: https://fastapi.tiangolo.com/
[2]: https://docs.djangoproject.com/en/dev/releases/3.1/
Once upon a time, Python's most common use was in web dev where it was basically the middle ground where you wanted something more productive than Java but more explicit than Ruby.
While it get a big boost from the ML trend and these days has mostly finished displacing Perl in the "Non-bash operational scripting" department, there's still plenty of projects and users that have been happily or otherwise using it all that time.
While it get a big boost from the ML trend and these days has mostly finished displacing Perl in the "Non-bash operational scripting" department, there's still plenty of projects and users that have been happily or otherwise using it all that time.
I've started with Web Development in 2013. Back then Node.js was still in 0.x versions, Electron was basically non-existent and Django looked ten-times better than anything PHP was able to offer. I've dabbled with some PHP/Node/Java framework tutorials since then but haven't really had reason to fully switch.
But if I started today I would probably try to find some framework to do everything in JS/TS.
But if I started today I would probably try to find some framework to do everything in JS/TS.
I started web dev in 2012 with a MEAN stack. Now I work with a LAMP stack. I deal with far less migrations and dead packages, but that's probably because Node.js was not very mature back then.
The real question is why someone like me should pitch JavaScript to my Django/Flask shop instead of technology with clearer competitive advantages: Elixir/Rust/whatever.
If I want better CPU usage I can get halfway there it with FastAPI and other Python frameworks.
The real question is why someone like me should pitch JavaScript to my Django/Flask shop instead of technology with clearer competitive advantages: Elixir/Rust/whatever.
If I want better CPU usage I can get halfway there it with FastAPI and other Python frameworks.
python has a sane packaging ecosystem and type system.
CPU power is cheap and developers are not.
CPU power is cheap and developers are not.
TypeScript seems better (or at least on par) typing wise to Python.
I don't think anyone is picking Python for it's type system. It's still a second class citizen at best.
I don't think anyone is picking Python for it's type system. It's still a second class citizen at best.
Common misconception. You have startups choosing a language while trying to absorb unsustainable operations costs to justify using the wrong language instead of picking the right language from the start.
[deleted]
Man I can't believe BBEdit didn't make the editor list!
< 1 year: 29%
1-2 years: 20%
3-5 years: 20%
6-10 years: 14%
11+ years: 17%
--
That means almost half of the respondents have 2 years or less professional experience with python. Not sure how that influences all the other results.