Ask HN: Has web development become insane or am I stuck in my ways?
74 comments
I came late to the software and web development game. I just wanted to prototype a tool I needed and ended up becoming a full time developer.
After grasping HTML and CSS basics, prototyping my heavily interactive tool seemed impossible with JS alone, until I learned React. React solved real problems for me. Create-react-app with a firebase backend got me VERY far with zero formal education.
But I struggled with SEO, load times, and needed a sprinkle of server-side logic. Next.js solved those problems, and more.
As my prototypes grew into products, complexity was hard to manage. Refactors were impossible, even with small, modular components and well tested functions. Enter TypeScript. The single most important tool in my toolbelt. I refuse to work in plain JS now.
I'm several years in now, at my 2nd full-time dev gig and working on my 3rd side-business. My stack is Typescript, Next, vercel, firebase. Both at work and at home. Hardly any third-party dependencies needed beyond these (stripe, coingate, mapbox). I write most my CSS by hand with css grid because its the least verbose way I've found so far. I auto-format with Prettier.
I can use types and tests across my stack, I can enforce bespoke auth/permissions behavior, all kinds of queries, media upload, pretty much everything I'll ever need outside of real-time collaboration. I can build and maintain profitable products by myself. I've done it.
This also happens to be a very popular stack. What's insane about it? Every alternative I've looked at looks way less sane. Multiple languages and repos for one app? No thanks. Managing server instances and databases migrations myself? I hope I never have to. Software is fun, but I like making products more. JavaScript just happens to be the best way I've found so far.
After grasping HTML and CSS basics, prototyping my heavily interactive tool seemed impossible with JS alone, until I learned React. React solved real problems for me. Create-react-app with a firebase backend got me VERY far with zero formal education.
But I struggled with SEO, load times, and needed a sprinkle of server-side logic. Next.js solved those problems, and more.
As my prototypes grew into products, complexity was hard to manage. Refactors were impossible, even with small, modular components and well tested functions. Enter TypeScript. The single most important tool in my toolbelt. I refuse to work in plain JS now.
I'm several years in now, at my 2nd full-time dev gig and working on my 3rd side-business. My stack is Typescript, Next, vercel, firebase. Both at work and at home. Hardly any third-party dependencies needed beyond these (stripe, coingate, mapbox). I write most my CSS by hand with css grid because its the least verbose way I've found so far. I auto-format with Prettier.
I can use types and tests across my stack, I can enforce bespoke auth/permissions behavior, all kinds of queries, media upload, pretty much everything I'll ever need outside of real-time collaboration. I can build and maintain profitable products by myself. I've done it.
This also happens to be a very popular stack. What's insane about it? Every alternative I've looked at looks way less sane. Multiple languages and repos for one app? No thanks. Managing server instances and databases migrations myself? I hope I never have to. Software is fun, but I like making products more. JavaScript just happens to be the best way I've found so far.
It's very reasonable for someone new. It's batshit insane for anyone who's been in the industry for about 20 years. But that doesn't matter. The only thing that does matter is finding tech that you're productive with, and you've found it, so everyone else's input is pretty moot. In addition, it sounds like you've found the right idea anyway which is: avoid complicating work.
It's a real tangible accomplishment to find a tech stack you're productive with and don't have to change or update frequently. So, congratulations!
The last major real revolution on the backend in my eyes was Sinatra, not for Sinatra itself, but how it changed how everyone wrote web servers. The last real revolution on the front end was Angular.js (v1.x) and then shortly after, React.
There's a lot of change year after year in the real world, but a majority of it you can ignore.
It's a real tangible accomplishment to find a tech stack you're productive with and don't have to change or update frequently. So, congratulations!
The last major real revolution on the backend in my eyes was Sinatra, not for Sinatra itself, but how it changed how everyone wrote web servers. The last real revolution on the front end was Angular.js (v1.x) and then shortly after, React.
There's a lot of change year after year in the real world, but a majority of it you can ignore.
No, what matters is user experience & user satisfaction. That is why this frontend churn is useless because it doesn’t improve either.
It is really sad but in my experience the only improvement provided.bybthese tools are regarding developer experience (and some times, not even that!). We're trading our own dev experience and sacrificing our user's. For proof, look at the new reddit, and so many other react apps which even with modern macs feel terribly slow and clunky
I was finally able to load the modern web as quickly as the old web once I got a laptop with one of those newish Ryzen chips. I expect complexity to grow quickly past the performance gains, but it's a nice reprieve.
[deleted]
With firebase, do you call individual cloud functions or does an API sit in front?
I ditched firebase cloud functions for Next's /api/ functions. I put all the db stuff and checkout and webhook stuff behind those endpoints. At first I just loaded the entire firestore library on the client and did authenticated reads/writes directly... But I wouldn't do that again, made my app too heavily intertwined with firestore's api.
Luckily you only have to learn that lesson once.
My personal experience is this: I love building stuff. I love the web. For a long time I loved React/Vue but over time I began to worry about desktop, mobile, PWA, future-proofing and found myself way over-thinking MVP and proof-of-concepts stacks. I tried and tried to limit my scope but found it was really hard to have discipline. Should it be CRA or Next.js? They are both awesome so I can't really decide. Should I use react-native-web so that I can put out mobile apps? How important is being in the app store? Etc, etc, etc.
One weekend not too long ago, I started a Rails app with Hotwire and found myself wonderfully, nostalgically productive again. Maybe I'm old, maybe I'm burnt out but I can tell you one thing: I'm having fun and building neat, performant, and fast web apps. I hope that speaks for itself when I look for my next job.
One weekend not too long ago, I started a Rails app with Hotwire and found myself wonderfully, nostalgically productive again. Maybe I'm old, maybe I'm burnt out but I can tell you one thing: I'm having fun and building neat, performant, and fast web apps. I hope that speaks for itself when I look for my next job.
Same here. Nowadays I think Django and unpoly give me 99% of what I need. I always missed s way to do "components" server side instead of including chunks of templates, and recently found slippers, which for me completes tht 1% I wa smissing
https://unpoly.com
https://mitchel.me/slippers/
https://unpoly.com
https://mitchel.me/slippers/
I've gotten to the point in my life, where I'd much rather work with people with a good sense of humor over smart people. Smart people do really dumb things. At least funny people can laugh about the dumb things they do instead of hold a tribunal.
Completely agree. Also smart people complicate things extreamly, probably because they want to push their skills and constantly learn more.
Smart people are also better at rationalizing their behavior than they are at listening.
I'm on the other side of friendships like that. I like to think the quality of my puns that bounce off their explanations help them understand their own understanding better.
yea, big difference between smart and practical people. good lord.
Every happy web developer is happy in the same way, every unhappy web developer is unhappy in a different way.
Other people would tell a story similar to yours except it revolves around Ruby on Rails, React, Docker, CSS, npm, emotion, or some other tools or standards.
You either are in a state of peace with your tools or not, and that is something that can change over time. For my job I work on a React application that was started by other people that has some good ideas in it but has plenty of accidents built in.
I was doing a side project and was tempted to make a little application in Svelte, which would have been a perfect fit. I used React instead because I wanted to build on and extend my experience. I had a 60kb bundle which might have been 20kb in Svelte but the graphics files are way bigger than that and it loads faster than a speeding bullet as it is.
That's the way I attain peace.
Other people would tell a story similar to yours except it revolves around Ruby on Rails, React, Docker, CSS, npm, emotion, or some other tools or standards.
You either are in a state of peace with your tools or not, and that is something that can change over time. For my job I work on a React application that was started by other people that has some good ideas in it but has plenty of accidents built in.
I was doing a side project and was tempted to make a little application in Svelte, which would have been a perfect fit. I used React instead because I wanted to build on and extend my experience. I had a 60kb bundle which might have been 20kb in Svelte but the graphics files are way bigger than that and it loads faster than a speeding bullet as it is.
That's the way I attain peace.
Nice. I am at peace with the tools I use. I'm very confident that the tools, stack and setup I use, for my context (early stage startups) does provide them with real value. My concern is that the industry as a whole is moving away from seeing that as valuable based on the jobs I see advertised and I want to stay relevant so that if I ever want a job / contract in the future, I am seen as valuable to someone :)
If you are taking on a new job, take your peace with you.
To get a new job, signal that you will find that peace wherever you go with whatever tools they use. It might take a while (about a year), and sometimes it doesn't happen, but if you believe it you can usually make it so.
To get a new job, signal that you will find that peace wherever you go with whatever tools they use. It might take a while (about a year), and sometimes it doesn't happen, but if you believe it you can usually make it so.
dude, have you ever worked with someone not using linters, and code-golfing his way through life?
i fucking wonder everyday how he attained bliss in chaos, hahaha.
i fucking wonder everyday how he attained bliss in chaos, hahaha.
From my point of view, a lot of people want to use the big stick the FAANG guys use, but they haven't learned that the big stick is heavy, to stretch an analogy. I'm overall happy I have seen a bunch of different workflows.
For example, I've been on the operational side of a leading browser game in germany some 5 - 10 years ago. Obviously, it must have been an amazing CI/CD workflow with great automation. No. The global deployment was me. With SCP. And we made money. I'm not necessarily proud of that on a tech level, but it worked. The job after that... we made money by using chef to copy war-files into tomcats and then we had some application to sell. Again, it's not technologically amazing, but it has a huge advantage: It's simple to get right, and hard to get seriously wrong.
Now, still at the job, but with a changed scope, yes, we are embracing docker and a container orchestration. But we do so, because we have to scale from 10 developers to 100 - 150 developers and we cannot handle the complexity in a central config management. We cannot support the 3-5 languages and frameworks necessary centrally anymore, so it makes sense to introduce a framework that's easier to cooperate on, in the layer that needs cooperation. It increases the total tech stack complexity, but it simplifies the things the individual team has to deal with, so it's a net benefit for most people.
But all that is driven by necessity. If you tell me to deploy half a dozen of same-blueprint php/spring boot/.NET Core apps on 1-2 servers without any infrastructure, I'd look at some ansible and some cheap VMs. Or if there is a natural thing like capistrano or whatever Laravel Forge is, that. I like not doing work and making mistakes hard to make.
For example, I've been on the operational side of a leading browser game in germany some 5 - 10 years ago. Obviously, it must have been an amazing CI/CD workflow with great automation. No. The global deployment was me. With SCP. And we made money. I'm not necessarily proud of that on a tech level, but it worked. The job after that... we made money by using chef to copy war-files into tomcats and then we had some application to sell. Again, it's not technologically amazing, but it has a huge advantage: It's simple to get right, and hard to get seriously wrong.
Now, still at the job, but with a changed scope, yes, we are embracing docker and a container orchestration. But we do so, because we have to scale from 10 developers to 100 - 150 developers and we cannot handle the complexity in a central config management. We cannot support the 3-5 languages and frameworks necessary centrally anymore, so it makes sense to introduce a framework that's easier to cooperate on, in the layer that needs cooperation. It increases the total tech stack complexity, but it simplifies the things the individual team has to deal with, so it's a net benefit for most people.
But all that is driven by necessity. If you tell me to deploy half a dozen of same-blueprint php/spring boot/.NET Core apps on 1-2 servers without any infrastructure, I'd look at some ansible and some cheap VMs. Or if there is a natural thing like capistrano or whatever Laravel Forge is, that. I like not doing work and making mistakes hard to make.
I have seen many trends, frameworks and „best practices“ come an go. The app our company provides is based on very basic tech, with little js, and server-rendered pages. But you know what? Our clients love it, we are profitable, and we keep growing while being able to keep the dev team small and flexible. Just ignore the hype and focus on what your users actually need and want.
Yes it's insane. The javascript world in particular suffered greatly from people carving up a name and conferences for themselves by building frankly ridiculous tools that the vast majority of people don't even need to reach for. But FOMO takes over.
Then you have the extremely low barrier of entry, resulting in `is-odd` packages polluting the shit out of the ecosystem.
Then you have everyone and their mother becoming a "JS coder" and bombarding twitter with dev.to articles.
You end up in JS hell.
The solution is to eject and avoid javascript as much as you can. At this point it's already too late to reverse course, or worse the majority don't even want to reverse course.
I've know many people, including my younger brother who said "fuck this" and went full backend to avoid working with npm/node/js.
Then you have the extremely low barrier of entry, resulting in `is-odd` packages polluting the shit out of the ecosystem.
Then you have everyone and their mother becoming a "JS coder" and bombarding twitter with dev.to articles.
You end up in JS hell.
The solution is to eject and avoid javascript as much as you can. At this point it's already too late to reverse course, or worse the majority don't even want to reverse course.
I've know many people, including my younger brother who said "fuck this" and went full backend to avoid working with npm/node/js.
This is without joke what I'm actually thinking about doing. I just can't get on with Javascript, and that's with 4 / 5 years of Vue development. I hate that I can't just come back to a project 6 months later and build it. Something always breaks. Everything feels super fragile. So I might just learn Go and stick to backend only and embrace all the things I spoke about hating in my post just so I can avoid Javascript. You can't win
I was a web developer in the late 90s/early 2000s, then moved to low-level management...and over the years, i might touch a little script here or there, and maybe code up a boring website...but nothing needing such an expansive pipeline. Well, in the last 2 years or so, i've been interested (from an intellectual perspective) to start coding again...But, honestly, the cognitive load is enough that backend seems more straightforward for my brain now. (To clarify, i'm not s aying that backend work is easy, just that for me and my brain it seems more straightforward to tackle/learn.) Sure, nowadays browsers support css and javascriot far better than back in the day, but there's still seemingly so much to think about and do for front-end work that i'm moving more towards backend stuff...for now i'm just playing with stuff (sicne i do not do this anymore at my dayjob, and in fact haven't done it for over 15 yrs now), but if i lose my job or needed to drastically change my career (and head back to development), there's no way that i would enjoy the front-end...i would definitely choose the backend type of work. Maybe that's just me...but yeah, web dev (at least front-end) for me seems to have become insane.
I learned Go, switched to almost all backend work and infrastructure. Now I spend most of my time building infrastructure and tooling, and I couldn't be happier. I'm (or was?) a pretty competent front-end developer but I just can't do it anymore. I can still build a good front-end product if I need to, but I just can't get along with the new-school front-end guys that think they know everything and want to reinvent the wheel constantly.
I've been writing Javascript since it was in beta. I love Javascript as a language but, I almost completely agree with what you're saying for all the reasons you laid out.
> The javascript world in particular suffered greatly from people carving up a name and conferences for themselves by building frankly ridiculous tools that the vast majority of people don't even need to reach for. But FOMO takes over.
Then you realize your resume is getting screened out at half the jobs look look at because you "missed out". FWIW it happens on the backend too, see docker, k8s, and all the related container infra I “missed out on”
Then you realize your resume is getting screened out at half the jobs look look at because you "missed out". FWIW it happens on the backend too, see docker, k8s, and all the related container infra I “missed out on”
I appreciate your observations, but don't agree with your conclusion:
Javascript provides a decent API for interacting with the DOM dynamically. I understand you're frustrated with culture, community, and tooling, but you can still use Javascript without these to make web pages interactive in a way you can't do pure server-side. The MDN web docs are a great place to start.
Javascript provides a decent API for interacting with the DOM dynamically. I understand you're frustrated with culture, community, and tooling, but you can still use Javascript without these to make web pages interactive in a way you can't do pure server-side. The MDN web docs are a great place to start.
What stack do you use, to avoid JS? I was reading about Phoenix framework which promises to lessen the use of JS, I have no experience with it though.
Also, how easy (or hard) is it to get a backend only job? I'd like to avoid CSS/JS completely, but unable to find such jobs.
Also, how easy (or hard) is it to get a backend only job? I'd like to avoid CSS/JS completely, but unable to find such jobs.
[deleted]
> Why do we introduce so much complexity that needn't be to this entire thing?
Ignorance, fear, and our urge to pick The Best option.
If something is complex enough, at some point everyone is ignorant to some degree. That makes it easy to let fear get a hold of you. Still: whenever you invest, you want to invest in The Best Option since your resources are limited.
You're a professional photographer, and you need a new camera. Your old one lasted a decade, so anything you'll buy now will be better and cheaper than the old one. Guaranteed! It's unlikely you know everything about cameras, though. But since you're going to invest in one, you want to get The Best option for the price. You're reading blog posts, reviews, and random comments by people on the internet, and it's becoming harder and harder to decide. There are so many factors involved that you become paralized, you don't know how to decide. Also, you don't know exactly if you're going to be making pictures in the jungle or in the ocean, how many pictures you're going to have to be able to take and how long your battery should last.
Then you read that the world's most successful photographer uses a camera that is rumoured to be able to handle all those situations, and has auto-renewing battery and infinite storage. The pricing is a bit unclear, but at this point you're so fatigued that you decide to just get one and find out along the way. At least you know it has the potential to be The Best.
Ignorance, fear, and our urge to pick The Best option.
If something is complex enough, at some point everyone is ignorant to some degree. That makes it easy to let fear get a hold of you. Still: whenever you invest, you want to invest in The Best Option since your resources are limited.
You're a professional photographer, and you need a new camera. Your old one lasted a decade, so anything you'll buy now will be better and cheaper than the old one. Guaranteed! It's unlikely you know everything about cameras, though. But since you're going to invest in one, you want to get The Best option for the price. You're reading blog posts, reviews, and random comments by people on the internet, and it's becoming harder and harder to decide. There are so many factors involved that you become paralized, you don't know how to decide. Also, you don't know exactly if you're going to be making pictures in the jungle or in the ocean, how many pictures you're going to have to be able to take and how long your battery should last.
Then you read that the world's most successful photographer uses a camera that is rumoured to be able to handle all those situations, and has auto-renewing battery and infinite storage. The pricing is a bit unclear, but at this point you're so fatigued that you decide to just get one and find out along the way. At least you know it has the potential to be The Best.
> At some point, even though my speed of delivery with my framework and stack setup is frankly very fast and I can provide real value to people, it seems the perceived value of the above shennanigans is greater than the value of actually shipping stuff that matters if you look at the job boards, technologies used and pay given.
One dimension of this is that it isn't just about your own ability to be productive as a solo developer (unless you limit yourself to solo-developer jobs, which can be an option!), it's about the org's ability to be productive as a (constantly changing, mixed-experience-levels) group. It's frustrating to see one's own productivity suffer for the sake of aggregate productivity, but it's a reality of our industry.
I can't speak to the particular situations you described but I can speak to this being a real trade-off that often has to be made.
One dimension of this is that it isn't just about your own ability to be productive as a solo developer (unless you limit yourself to solo-developer jobs, which can be an option!), it's about the org's ability to be productive as a (constantly changing, mixed-experience-levels) group. It's frustrating to see one's own productivity suffer for the sake of aggregate productivity, but it's a reality of our industry.
I can't speak to the particular situations you described but I can speak to this being a real trade-off that often has to be made.
I'm not a web developer so maybe I'm missing something... What's your thesis here? That you're very efficient with a stack that you've been using for 13 years? That other people's stacks and architectures are overcomplicated?
If what you're doing works for you, just keep it up. I don't think that you have an obligation to add complexity to what you're doing just so you can stay relevant.
If what you're doing works for you, just keep it up. I don't think that you have an obligation to add complexity to what you're doing just so you can stay relevant.
> If what you're doing works for you, just keep it up. I don't think that you have an obligation to add complexity to what you're doing just so you can stay relevant.
I think this might be a recipe for being a person with 13-years of experience and yet unhirable at the same time.
I don't know if it is, but your advice sounds a little bit dangerous to me.
I think this might be a recipe for being a person with 13-years of experience and yet unhirable at the same time.
I don't know if it is, but your advice sounds a little bit dangerous to me.
I think that it comes down to the difference between not learning at all and not learning specific overcomplicated things when they're unnecessary for your use case.
If you look at OP's examples, I don't think that any company would disqualify an otherwise competent developer for having only worked on monoliths instead of microservices or for using an easy deploy set up instead of a something with Docker and a CI/CD system.
And of course OP still has tonnes of experience with their specific stack which is still quite popular - that's the opposite of being unhireable.
The other part is that you might be overestimating how difficult it is to pick up something new when it's necessary. Most of these aren't that big of a deal to learn for someone with experience.
If you look at OP's examples, I don't think that any company would disqualify an otherwise competent developer for having only worked on monoliths instead of microservices or for using an easy deploy set up instead of a something with Docker and a CI/CD system.
And of course OP still has tonnes of experience with their specific stack which is still quite popular - that's the opposite of being unhireable.
The other part is that you might be overestimating how difficult it is to pick up something new when it's necessary. Most of these aren't that big of a deal to learn for someone with experience.
Totally with you! We’re in your “Anecdote 1” situation at the moment, moving separate services back to a glorious Monolith & Monorepo…
Also just anecdotal, but I can confirm that jumping on the SPA/Vue/React bandwagon and the complexity that brings hugely decreased our productivity.
And I think if I’m really honest those decisions to move to JavaScript heavy apps were definitely a bit FOMO based in the sense that you don’t want to be left behind with a “gap” on your resume… makes me wonder how often decisions are truly based on “the right tool for the job” or maybe even more importantly “the right tool for the team”.
My biggest reason to use CI/CD after been at startups and big companies is to have a repeatable setup, build, and deploy. By taking the human element out of the machine setup and release/deployment phases a whole class of mistakes are mitigated.
Sure, you're fine with running and managing these servers, but what about the new hire who replaces you when you move on?
Specifically it forces most of the DevOps tribal knowledge to be coded/documented. Those "Oh, that's right I needed to recompile the mod_wsgi extension manually because this version of the CentOS didn't support python 3" moments go away. (This is a real example from the last server I needed to setup by hand.)
It conveniently it also makes setting it up on a new machine a piece of cake. Clone the repo, run the setup command/script and run the
It took me a while to grok, all of the pieces, but I'm glad to have it.
That said it can get extremely complex (and by extension fragile) for little benefit. My entire setup and deployment are controlled with two scripts one for creating a cluster (CDN, DB, message queue, app server, monitoring) and one GitHub action script for CI/CD.
It's all very understandable even for people who've never worked with these technologies before. Writing the scripts on the other hand take a lot of knowledge and it took me a month to really grok what I was doing.
Sure, you're fine with running and managing these servers, but what about the new hire who replaces you when you move on?
Specifically it forces most of the DevOps tribal knowledge to be coded/documented. Those "Oh, that's right I needed to recompile the mod_wsgi extension manually because this version of the CentOS didn't support python 3" moments go away. (This is a real example from the last server I needed to setup by hand.)
It conveniently it also makes setting it up on a new machine a piece of cake. Clone the repo, run the setup command/script and run the
It took me a while to grok, all of the pieces, but I'm glad to have it.
That said it can get extremely complex (and by extension fragile) for little benefit. My entire setup and deployment are controlled with two scripts one for creating a cluster (CDN, DB, message queue, app server, monitoring) and one GitHub action script for CI/CD.
It's all very understandable even for people who've never worked with these technologies before. Writing the scripts on the other hand take a lot of knowledge and it took me a month to really grok what I was doing.
Web development has become insane indeed. At work, I tell myself, technologies change and we need to adopt or we will be left behind.
But it is not as joyful as it used to be. For personal projects, I have been using Laravel, I run it on my local machine using Laravel Valet. I got a tiny CRM, chores manager, and various other little one off tools. Laravel is today's MS Access. In early 2000s, I built various applications for small businesses using MS Access.
But my skills in PHP/Laravel leads to web development jobs. Which means everything that you have mentioned.
So I am slowly exploring new options. Earlier I was playing with Python and Data Science. Jupyter Notebooks are fairly portable, and let you visualize data in fun ways. And I figured Laravel/Nodejs > Django > Data Engineer > Data Science should be easy. But it wasn't as enjoyable. I like building tools.
Now I am exploring desktop application development. I looked at Electronjs but that reminded me too much of work.
I have started playing with Swift/MacOS/Desktop app development. Apple's development ecosystem using Swift seems great. So far no real issue though I am not doing anything advanced. What I like about it is that I can actually build apps that I can share with my friends without having them install a bunch of dependencies on their machines. The debuggers are awesome. Also I am exploring artistic apps like AR with sound effects + visualizations. You are closer to hardware and can do some pretty cool things.
I am not sure how the job market for Swift macOS development is but my goal is to find some sane challenging but not overwhelming (CBNO) programming opportunities.
But it is not as joyful as it used to be. For personal projects, I have been using Laravel, I run it on my local machine using Laravel Valet. I got a tiny CRM, chores manager, and various other little one off tools. Laravel is today's MS Access. In early 2000s, I built various applications for small businesses using MS Access.
But my skills in PHP/Laravel leads to web development jobs. Which means everything that you have mentioned.
So I am slowly exploring new options. Earlier I was playing with Python and Data Science. Jupyter Notebooks are fairly portable, and let you visualize data in fun ways. And I figured Laravel/Nodejs > Django > Data Engineer > Data Science should be easy. But it wasn't as enjoyable. I like building tools.
Now I am exploring desktop application development. I looked at Electronjs but that reminded me too much of work.
I have started playing with Swift/MacOS/Desktop app development. Apple's development ecosystem using Swift seems great. So far no real issue though I am not doing anything advanced. What I like about it is that I can actually build apps that I can share with my friends without having them install a bunch of dependencies on their machines. The debuggers are awesome. Also I am exploring artistic apps like AR with sound effects + visualizations. You are closer to hardware and can do some pretty cool things.
I am not sure how the job market for Swift macOS development is but my goal is to find some sane challenging but not overwhelming (CBNO) programming opportunities.
Software development is all about picking the right tools for the job. The right tool is a function of problem complexity, team size, experience of the team, maintainability, and other factors.
There are two main types of tool selection failures I see:
1. Teams that try to choose the “best” tools according to what they perceive as the state of the art without regard to the problem their trying to solve. This tends to become a game of picking whatever they think FAANG might be using, even if their problem is orders of magnitude smaller and simpler. At the extremes, these are the teams that end up building a Kubernetes-deployed custom React-based CMS with server-side rendering when really they could have set up a Wordpress install with a custom theme in a couple of weeks.
2. Teams that embody the phrase “When all you have is a hammer, everything looks like a nail.” These teams have been using the same tech stack for a long time and they’re so good at it that they refuse to consider anything else. Instead of trying new tools, they try to brute-force their current stack to get the job done. Some times this works well if the problem is a good fit for the stack. Other times, they waste a ton of time and money doing things that could have been solved quickly by trying a new tool.
None of us are perfect, but most failures fall into one of the two categories above. Eager startups often struggle with #1 when they have a pseudo-CTO cofounder who wants to pick all of the technologies but hire someone else to do all of the work.
My approach is to draw up multiple proposals with different stacks and present them with an overview of possible risks, benefits, and timelines. Usually once it’s all laid out, reasonable engineers might recognize that the right tool for the job isn’t always the trendiest tool. It takes some good presentation skills to navigate this but it can be done.
At the same time, I see some opportunities for you to grow your experience a bit. A CI/CD system has real benefits for delivery, but it shouldn’t require 3-4 full-time engineers to get going. It’s worth learning how to do this right so you can actually evaluate it, rather than dismissing it as useless complication. Frankly, setting up a basic CI/CD workflow with modern tools like GitHub or Gitlab isn’t a big chore. You should be able to come up with something usable and a good understanding of the basics in a workweek by yourself.
There are two main types of tool selection failures I see:
1. Teams that try to choose the “best” tools according to what they perceive as the state of the art without regard to the problem their trying to solve. This tends to become a game of picking whatever they think FAANG might be using, even if their problem is orders of magnitude smaller and simpler. At the extremes, these are the teams that end up building a Kubernetes-deployed custom React-based CMS with server-side rendering when really they could have set up a Wordpress install with a custom theme in a couple of weeks.
2. Teams that embody the phrase “When all you have is a hammer, everything looks like a nail.” These teams have been using the same tech stack for a long time and they’re so good at it that they refuse to consider anything else. Instead of trying new tools, they try to brute-force their current stack to get the job done. Some times this works well if the problem is a good fit for the stack. Other times, they waste a ton of time and money doing things that could have been solved quickly by trying a new tool.
None of us are perfect, but most failures fall into one of the two categories above. Eager startups often struggle with #1 when they have a pseudo-CTO cofounder who wants to pick all of the technologies but hire someone else to do all of the work.
My approach is to draw up multiple proposals with different stacks and present them with an overview of possible risks, benefits, and timelines. Usually once it’s all laid out, reasonable engineers might recognize that the right tool for the job isn’t always the trendiest tool. It takes some good presentation skills to navigate this but it can be done.
At the same time, I see some opportunities for you to grow your experience a bit. A CI/CD system has real benefits for delivery, but it shouldn’t require 3-4 full-time engineers to get going. It’s worth learning how to do this right so you can actually evaluate it, rather than dismissing it as useless complication. Frankly, setting up a basic CI/CD workflow with modern tools like GitHub or Gitlab isn’t a big chore. You should be able to come up with something usable and a good understanding of the basics in a workweek by yourself.
Sure! I understand what you mean. I am always wary that I'm falling into point 2 which is why I posted this. I have build CI / CD systems before. It's just that it comes out-of-the-box with Forge in that some basic things run like pulling the latest code, migrating any database migrations, building any assets etc. It's not like deploys were manual etc. Just a push to master and the CI / CD would run.
1. masses of RDD: Resume-Driven Development
2. the delusion that the way google manages 1-3% of global computing power is appropriate for your company which bursts up to < 10 servers under "high" load
2a. though you can quickly get to a place where you do need more control over migrations once your db has a nontrivial size
3. early engineers and bad engineers love complexity. Love it.
ps -- lots of these choices come down to the choices of senior people. There are good shops out there! This may resonate with you. https://mcfunley.com/choose-boring-technology
2. the delusion that the way google manages 1-3% of global computing power is appropriate for your company which bursts up to < 10 servers under "high" load
2a. though you can quickly get to a place where you do need more control over migrations once your db has a nontrivial size
3. early engineers and bad engineers love complexity. Love it.
ps -- lots of these choices come down to the choices of senior people. There are good shops out there! This may resonate with you. https://mcfunley.com/choose-boring-technology
Yes - overengineering is a disease. As it is discussed extensively on here, most companies are not Google or Netflix.
Engineers usually opt for the most complex option, when there are multiple factors to consider - reliability, development time, is this massive setup cost worth the effort given our load, etc.
It's certifiably insane.
I think in some cases at least, this is an example of cargo culting ie. copying big tech without knowing why. Other reasons include resume driven development. Some devs, either inexperienced or plain poor experienced ones, like to entertain themselves by making things complex.
However, all of that said, in your particular case, a CI/CD pipeline using docker / EC2 doesn't really sound that mad. The patterns are super well understood there. Maybe it's just a case of getting used to it? Fwiw, we do pretty much that and we have fully automated deployments without much hassle. We use ECS / Fargate too which might sound like a other level of black magic but honestly, serverless is pretty amazing too. I don't even have to think about EC2 now. I'm not saying this to showoff, the point is that this new tech maybe takes some getting used to, but maybe it's just a lack of anyone experienced leading the way on it and helping you out?
As for microservices, that can be a nightmare when done wrong but it totally depends. If you just split things up without really understanding why, you're now going to have two problems as the old adage goes. The distributed monolith is the worst of all worlds, especially when the boundaries between services are poorly thought out. If in doubt, and unless you can articulate exactly why you're creating another service, build a well structured monolith imho.
However, all of that said, in your particular case, a CI/CD pipeline using docker / EC2 doesn't really sound that mad. The patterns are super well understood there. Maybe it's just a case of getting used to it? Fwiw, we do pretty much that and we have fully automated deployments without much hassle. We use ECS / Fargate too which might sound like a other level of black magic but honestly, serverless is pretty amazing too. I don't even have to think about EC2 now. I'm not saying this to showoff, the point is that this new tech maybe takes some getting used to, but maybe it's just a lack of anyone experienced leading the way on it and helping you out?
As for microservices, that can be a nightmare when done wrong but it totally depends. If you just split things up without really understanding why, you're now going to have two problems as the old adage goes. The distributed monolith is the worst of all worlds, especially when the boundaries between services are poorly thought out. If in doubt, and unless you can articulate exactly why you're creating another service, build a well structured monolith imho.
What a software engineer needed to know in the 90s to build an e-commerce website:
- PHP, HTML, CSS, JavaScript, Mysql, Apache, Linux, FTP
What a software engineer needs to know today to build an e-commerce website (let's say, we keep using PHP):
- A PHP framework, HTML, CSS, JavaScript, React, Typescript, scss, Mysql, an ORM mapper or similar, Nginx, Docker, K8s, GitLab (or GitHub) CI/CD, Git, Linux, be familiarized with some cloud provider (GCP/AWS/etc), oauth, DDD, SOLID, ...
The reason: user requirements today are way higher than in the past. And, we all are users. Personally, I, as a user of the internet, wouldn't mind not having banking apps, or SPAs, or 2FA, or "sign with Google", or push notifications, or all these "nice" things we have came up in the last 2 decades. I don't need it, I don't like it, and as a developer I would be even happier if we never introduced them in the first place.
Note: yes, a developer nowadays needs to know all the stuff I mentioned above. Just check out job descriptions out there, companies are no longer looking for simple individual contributors, they want engineers that own the whole product from conception to deployment. Very sad panorama, to be honest.
- PHP, HTML, CSS, JavaScript, Mysql, Apache, Linux, FTP
What a software engineer needs to know today to build an e-commerce website (let's say, we keep using PHP):
- A PHP framework, HTML, CSS, JavaScript, React, Typescript, scss, Mysql, an ORM mapper or similar, Nginx, Docker, K8s, GitLab (or GitHub) CI/CD, Git, Linux, be familiarized with some cloud provider (GCP/AWS/etc), oauth, DDD, SOLID, ...
The reason: user requirements today are way higher than in the past. And, we all are users. Personally, I, as a user of the internet, wouldn't mind not having banking apps, or SPAs, or 2FA, or "sign with Google", or push notifications, or all these "nice" things we have came up in the last 2 decades. I don't need it, I don't like it, and as a developer I would be even happier if we never introduced them in the first place.
Note: yes, a developer nowadays needs to know all the stuff I mentioned above. Just check out job descriptions out there, companies are no longer looking for simple individual contributors, they want engineers that own the whole product from conception to deployment. Very sad panorama, to be honest.
Did the microservices introduce so much complexity because they were built on some complex "microservice stack"?
How would you think about a simple approach to microservices based on http requests? Say we have an application that resembles Twitter and want to split the "messages" functionality off the main codebase. We could run all "messages" functionality on a different server. Served by a completely different codebase. When the user asks for a url below /messages/ we simply do a "GET messageserver.com/messages/...?uid=<userid>" get the result, put it into the template we are rendering at the {messages} placeholder and output the page to the user.
How would you think about a simple approach to microservices based on http requests? Say we have an application that resembles Twitter and want to split the "messages" functionality off the main codebase. We could run all "messages" functionality on a different server. Served by a completely different codebase. When the user asks for a url below /messages/ we simply do a "GET messageserver.com/messages/...?uid=<userid>" get the result, put it into the template we are rendering at the {messages} placeholder and output the page to the user.
I'd be going back to asking why you want to split it off though. If there's legitimate reasons for that then my other concern would be what about areas of the existing codebase that are intertwined with those messages' modelling. Let's say you use the messages as objects throughout the system and those objects have their own methods that give you some nice way to intepret / action those messages. Now what? Do you need to re-write all that, create a shared library of models etc? It does add complexity and I'd always be last-resorting it.
The motivation to split it off would be to make development easier. One developer can work on the messages codebase and the other one on the tweets codebase. Without much friction between them.
Looking around on the Twitter website, I would expect there to be rather little entanglement of the messages functionality with the rest of the site.
Looking around on the Twitter website, I would expect there to be rather little entanglement of the messages functionality with the rest of the site.
I agree, but I'm framing this entire talk about why people use microservices / these technologies prematurely. If you're at Twitter's scale, not only do you have a technical requirement to scale such an endpoint more than others, but also a people problem in that you want teams to be a bit more autonomous in the work that they do. None of that really applies to most companies. I just feel like most companies blindly model their developent practices off of FAANG when it actually hinders their development velocity instead of helps it.
As long as you don’t have any dependencies between the services.
But as soon as you do, which is probably the case, you end up doing method calls over HTTP and you need to have a strategy for refactoring two services at the same time that can be online at the same time but with different versions.
But as soon as you do, which is probably the case, you end up doing method calls over HTTP and you need to have a strategy for refactoring two services at the same time that can be online at the same time but with different versions.
It’s definitely insane, but you came into this part of the field when it was still fairly young.
13 years is a long time for technologies to build on top of each other, and while more things are possible now than ever, a trade off of that is that complexity is higher than ever, too.
Also, the boring middle road doesn’t get the press. No one is writing stories about all the little apps and services chugging along with their solid rails or laravel or whatever backends, serving basic html or API requests all day.
13 years is a long time for technologies to build on top of each other, and while more things are possible now than ever, a trade off of that is that complexity is higher than ever, too.
Also, the boring middle road doesn’t get the press. No one is writing stories about all the little apps and services chugging along with their solid rails or laravel or whatever backends, serving basic html or API requests all day.
Yes - it's insane.
HTML/CSS/Carefully-managed (modern) JS or Typescript is the way to go. A backend framework that supports DB migrations, authentication, templating, and email. This wouldn't have been reasonable 5-10 years ago, but is now. (Eg modern JS DOM API and language features, CSS grid and flexbox)
Maybe a declarative framework for a complex SPA.
Domain-specific languages, poorly-performing frameworks, and excessive use of dependencies is pervasive.
HTML/CSS/Carefully-managed (modern) JS or Typescript is the way to go. A backend framework that supports DB migrations, authentication, templating, and email. This wouldn't have been reasonable 5-10 years ago, but is now. (Eg modern JS DOM API and language features, CSS grid and flexbox)
Maybe a declarative framework for a complex SPA.
Domain-specific languages, poorly-performing frameworks, and excessive use of dependencies is pervasive.
Sometimes k8s, docker, and/or microservices are the right choice. Sometimes CI/CD is necessary and sometimes it's overkill. I think your best path forward is to learn some of the new hot technologies and recognize when they are valuable. If someone else forces you to overengineer or overcomplicate, the best you can do is advocate for something better and then be the guy who can do it anyway.
I have actually never worked in a team where CI was actually needed. If it hadn't been there, files would instead just be copied to production manually. One step.
Our industry is pretty crazy in the amount of tech it uses to move shit from A to B according to "best practices".
Not saying we shouldnt but I am saying that if we did start making things as simple as possible again, it would be extreamly productive and fun.
Our industry is pretty crazy in the amount of tech it uses to move shit from A to B according to "best practices".
Not saying we shouldnt but I am saying that if we did start making things as simple as possible again, it would be extreamly productive and fun.
Did you mean CD instead of CI?
FWIW, I have never worked in a team where CI was not needed.
FWIW, I have never worked in a team where CI was not needed.
Whenever this topic is brought up i wish people replied with concrete, specific situations rather than rationalizations that masquerade as abstractions for their choices. I 've had the luxury to be a solo dev , and every time i look up at a new shiny name that seems to be popular here (so many names! seemingly without hierarchy), i end up wondering "why isn't this thing useless"
Personally I always prefer Microservices and some sort of CI pipeline over a classic Monolith. Yes, it's painful sometimes but when using some best practices (e.g. one DB per service) it works surprisingly well and can offer benefits that monoliths just can't. But I think it's fair to write Monoliths for new projects - many people do.
I cannot possibly understand why you would do that.
You're dropping the ability to do trivial joins, trivial reporting, trivial SQL aggregates, over... what, exactly?
You're dropping the ability to do trivial joins, trivial reporting, trivial SQL aggregates, over... what, exactly?
Generally I believe it's done for encapsulation between teams. That way, each team can own their code without any fear of another team being able to break anything provided they all work to their contracts.
If you can perform a join to get the data yourself, nothing is stopping anyone from circumventing your API that is intended for fetching the data or by making breaking changes to the single database that your code relies on.
But yes, trivial joins is effectively the trade off for this encapsulation and can mean other services have to collect their own versions of data from services which means you now have eventual consistency.
If you can perform a join to get the data yourself, nothing is stopping anyone from circumventing your API that is intended for fetching the data or by making breaking changes to the single database that your code relies on.
But yes, trivial joins is effectively the trade off for this encapsulation and can mean other services have to collect their own versions of data from services which means you now have eventual consistency.
If you have decided to go microservices the most important part to have is good tooling to handle them.
Log tracing over multiple services, how to get an address of another service, deploy services independently from each other, track deployed versions of services, multi instance handling of same service, etc.
If you don’t have great tooling you will struggle.
Log tracing over multiple services, how to get an address of another service, deploy services independently from each other, track deployed versions of services, multi instance handling of same service, etc.
If you don’t have great tooling you will struggle.
Use https://redwoodjs.com/. My journey has been from no frontend framework -> Vue.js -> React -> Next.js -> Redwood, and I'm the most happy with the tooling at the moment.
Programming is like music and you are a Rush fan. Even Rush fans have their favourite periods of Rush. It wouldn't be a very fun world if we all had to like AC/DC :-)
it is insane, yes.
as much as possible i try to simplify everything in work, get rid of the extra complexities people introduce from these new hot things.
key questions
- can developers code and test on their own
- how much devs are needed to stage features
- do developers have autonomy on their own projects and assignments
- how much friction is there to onboard new developers
- how much mental bandwidth is required to keep track of the whole stack
like, we're not even a fortune 500 company serving to hundreds of thousands of people, keep it simple guys
as much as possible i try to simplify everything in work, get rid of the extra complexities people introduce from these new hot things.
key questions
- can developers code and test on their own
- how much devs are needed to stage features
- do developers have autonomy on their own projects and assignments
- how much friction is there to onboard new developers
- how much mental bandwidth is required to keep track of the whole stack
like, we're not even a fortune 500 company serving to hundreds of thousands of people, keep it simple guys
It's been insane for a long time.
Why do you use Laravel instead of just plain php files?
It is what all the cool kids are doing.
No, it's utterly insane.
No offense, but yes, you stuck in your ways and likely never shipped a real scalable web app at the scale that is expected today.
99% of software projects have, at most, a few hundred users for internal institutional things.
The vast majority of engineers have never built things to "scale", and don't need to.
Even in large companies the proportion of things that needs scale is very small.
The vast majority of engineers have never built things to "scale", and don't need to.
Even in large companies the proportion of things that needs scale is very small.
I work an old Zend App running in two nations that runs on a cheap VM. It's scales far beyond the market and makes real money.
We have docker now but that is only to manage legacy dependencies.
You don't always need complex infrastructure for scale.
Getting your indexes correct, keeping the number of calls to the db low, and tuning your db can do magic.
We have docker now but that is only to manage legacy dependencies.
You don't always need complex infrastructure for scale.
Getting your indexes correct, keeping the number of calls to the db low, and tuning your db can do magic.
At the digital agency I worked at, we moved to frameworks like Laravel and the frontend side of things was pretty static with custom CSS and some custom Javascript. Everything was hosted on a big dedicated server that would eventually become a VPS per client. The web was really simple then but understandly so were users' expectations of what the web could do and so there wasn't this mismatch that I feel I'm finding now.
Fast forward to today and I'm doing primarily product development for early stage startups.
CSS is a problem that is solved with Tailwind (for me at least anyway, I know it's a contentious topic). Backend is a problem solved with Laravel which truly does provide everything I've ever needed and in cases where it hasn't (for compute heavy services), I've pulled those areas out into a mini Go service and whacked it on a separate server. Javascript is just a problem...
Anecdote 1: I've worked at startups that had Laravel Monoliths where they tried to decouple it into microservices of their own, but this failed; it introduced so much complexity, mental concentration overhead of switching between code-bases and reduced developer autonomy: unless you knew how to deploy these new things and how code libraries should be shared amongst the new services you might as well pack up your toys and go home; you're not smart enough for this. Anyway, the services were bought back into the main repository because it was getting out-of-hand and frustrating to work with.
Anecdote 2: I recently deployed a Laravel app for a startup on an EC2 server. It worked great. I used Laravel Forge to do it and it took me half a day at most (once the dependancies I needed in PHP extensions were sorted). The startup wanted to move to Docker and have a CI / CD system. Previously I could just push changes to `master` and it would deploy using Forge. I watched as a CI / CD system was built that when production moved to it, it kept falling over and not really working very well. Additionally my autonomy to fix things was then taken away because there are layers or technologies stacks some of which I know but some of which I don't in the way. The whole thing became so frustrating to see. What was a perfectly working server that took me half a day to setup has now taken over 3 - 4 engineers full time work to end up with something that gives 0 benefit (except the idea that the server can never die, but that doesn't seem to be the case because it kept dying on any deployment and ended up with more downtime than the EC2 server ever had).
So why are companies with technology stacks that really arne't getting huge amounts of traffic or require huge amounts of complexity, choosing to use microservices based architectures or build their entire application in something like next.js which requires wiring together so much stuff (Prisma, Auth0) etc. Why do we introduce so much complexity that needn't be to this entire thing?
At some point, even though my speed of delivery with my framework and stack setup is frankly very fast and I can provide real value to people, it seems the perceived value of the above shennanigans is greater than the value of actually shipping stuff that matters if you look at the job boards, technologies used and pay given. I would love to be Bilbo Baggins in The Shire minding his own business, but I worry that if I don't jump on the bandwagon of overengineering everything to make myself feel smart then I'll quickly become irellevant in the job market. Am I completely missing the point of why these things are being done or is modern web development choices with regards to language and architecture just certifiably insane?