Why Invest in Tools?(medium.com)
medium.com
Why Invest in Tools?
https://medium.com/@leeb/why-invest-in-tools-3240ce289930
44 comments
I think that game studios have the right model: Hire full-time tools engineers and task them with making the tool-users more effective.
Stellar games simply can't exist without stellar tools, and since every game is a little different, everyone's tooling needs are a little different. So you have no choice but to build your own tools (to some extent). Unlike armchair architects designing wheel frameworks or factory factories, the guy building the level design tool has real users (in this example: level designers) he has to build for.
This same model can be applied to web or service development. You just need to involve users from the start, especially so if your users are themselves developers who are especially demanding and critical and whose tools are especially difficult to create.
Stellar games simply can't exist without stellar tools, and since every game is a little different, everyone's tooling needs are a little different. So you have no choice but to build your own tools (to some extent). Unlike armchair architects designing wheel frameworks or factory factories, the guy building the level design tool has real users (in this example: level designers) he has to build for.
This same model can be applied to web or service development. You just need to involve users from the start, especially so if your users are themselves developers who are especially demanding and critical and whose tools are especially difficult to create.
> I think that game studios have the right model: Hire full-time tools engineers and task them with making the tool-users more effective.
That's one approach, but it's not always the right one. It's only really necessary if your studio's selling point is pushing the limits, tech-wise. If your focus is the gameplay mechanics and engine and art are secondary to that, then focusing on tooling is more of a distraction than anything else. Hence why Unity and the now-quite-cheap UE are sane choices, since they let people focus on the gameplay.
General purpose software has the same pitfall that games do: people see the big studios/development companies spending a lot of time on tool-building, and they assume that's a prerequisite, which changes the mental model from "we need to start building the next cool _____" to "we need to build the tool(s) that will let us build the next cool _____". And that's a much more expensive model.
I think it's easy to get sucked into that trap for another reason too: developers know what developers want more than developers know what users want. That's why I agree with the last paragraph you wrote, that user involvement is key. I just think that companies need to be careful that they're not simply redefining users from "users of our core product" to "developers of the tools that will let us build our core product", since there's going to be some natural internal bias that way.
That's one approach, but it's not always the right one. It's only really necessary if your studio's selling point is pushing the limits, tech-wise. If your focus is the gameplay mechanics and engine and art are secondary to that, then focusing on tooling is more of a distraction than anything else. Hence why Unity and the now-quite-cheap UE are sane choices, since they let people focus on the gameplay.
General purpose software has the same pitfall that games do: people see the big studios/development companies spending a lot of time on tool-building, and they assume that's a prerequisite, which changes the mental model from "we need to start building the next cool _____" to "we need to build the tool(s) that will let us build the next cool _____". And that's a much more expensive model.
I think it's easy to get sucked into that trap for another reason too: developers know what developers want more than developers know what users want. That's why I agree with the last paragraph you wrote, that user involvement is key. I just think that companies need to be careful that they're not simply redefining users from "users of our core product" to "developers of the tools that will let us build our core product", since there's going to be some natural internal bias that way.
With Unity, that's a common misunderstanding. Unity provides nothing aside from an engine and a very open, flexible editor. The editor gives you a way to put game objects in the world and add script components to the game objects and that's pretty much it. If you don't build tools for the level designers you end up with a mess of bespoke game objects with random components. It's very easy to end up with messy Unity projects that are hard to debug and don't perform optimally (I've seen quite a few).
If you want to build something of a high quality in Unity you need structure and rigidity, and you need level designers and artists to follow guidelines and processes. So you still need tools and tools programmers. You might not have an entire team building an editor, but you need something (and it's very easy to build custom editor tools inside the Unity editor).
If you want to build something of a high quality in Unity you need structure and rigidity, and you need level designers and artists to follow guidelines and processes. So you still need tools and tools programmers. You might not have an entire team building an editor, but you need something (and it's very easy to build custom editor tools inside the Unity editor).
> So you still need tools and tools programmers. You might not have an entire team building an editor, but you need something (and it's very easy to build custom editor tools inside the Unity editor).
No disagreement there. I just meant that you don't need to start from a blank canvas and start in on engine, editor, etc. development just to get to the point where you can do good game mechanics. Obviously even if you use a canned engine you'll still need to do some work to get your asset pipeline squared away (just as how everyone needs to do a bit of scripting/config, etc. specific to their build process for example.)
I guess I didn't explain it well, but I was trying to contrast to the web world where people [seem to] very often look at places like Facebook rolling their own frameworks and saying "we need to do that to be good!" and thus literally go on to start from scratch.
No disagreement there. I just meant that you don't need to start from a blank canvas and start in on engine, editor, etc. development just to get to the point where you can do good game mechanics. Obviously even if you use a canned engine you'll still need to do some work to get your asset pipeline squared away (just as how everyone needs to do a bit of scripting/config, etc. specific to their build process for example.)
I guess I didn't explain it well, but I was trying to contrast to the web world where people [seem to] very often look at places like Facebook rolling their own frameworks and saying "we need to do that to be good!" and thus literally go on to start from scratch.
React as Relay/GraphQL aren't from a blank slate either. Hell, nothing really is. How much to invest in what level of tooling is, as it should be, a function of what already exists, what you're trying to build, and what resources you have at your disposal.
[deleted]
The way I read it, the advice is this: Allow and encourage your people to build tools to solve their own problems and if the projects are good, teams will form around them.
This answers the question from a management perspective. Facebook has a lot to gain from a small number of huge success stories like React. This makes up for the cost of all the failed little tooling projects.
This answers the question from a management perspective. Facebook has a lot to gain from a small number of huge success stories like React. This makes up for the cost of all the failed little tooling projects.
That's the way I read it, and no, it doesn't really answer the question -- it just begs the question. Most "good" projects are totally irrelevant to your business. Someone has to pick and choose, or you rapidly end up with the team situation I described: lots of people working on "their own problems", and only a few people working on the core problems of the business.
Your second paragraph is an assertion -- maybe it's true that one project like React makes up for all of the failed initiatives that the author describes (i.e. multiple person-years of effort), but that's a claim that deserves some evidence. Even if it's true, there's a level of management implied, or else everyone would spend their work hours chasing the same, captivating rabbits. My suspicion is that there's a lot more selection happening inside Facebook than the author lets on.
From experience, it's easy to get to a place where you have a lot of smart, well-intended people working on things that just don't matter. Saying "the best projects win" doesn't really solve anything, because you still have to define "best". Either that, or it's true that this is just a "champagne problem", and the Facebooks and Googles of the world can just throw dollars at anything, in the hope that something sticks.
Your second paragraph is an assertion -- maybe it's true that one project like React makes up for all of the failed initiatives that the author describes (i.e. multiple person-years of effort), but that's a claim that deserves some evidence. Even if it's true, there's a level of management implied, or else everyone would spend their work hours chasing the same, captivating rabbits. My suspicion is that there's a lot more selection happening inside Facebook than the author lets on.
From experience, it's easy to get to a place where you have a lot of smart, well-intended people working on things that just don't matter. Saying "the best projects win" doesn't really solve anything, because you still have to define "best". Either that, or it's true that this is just a "champagne problem", and the Facebooks and Googles of the world can just throw dollars at anything, in the hope that something sticks.
I call this the "spaghetti cannon" approach. Fire a cannon of spaghetti at a wall, see what sticks.
I think your last sentence is absolutely right. The author works on a team whose responsibility is to make programming easier for everyone in the world. That's a luxury most startups don't have.
[deleted]
> Rapidly growing companies tend to over-hire, and then have bored engineers that spend lots of time creating wheel-building framework frameworks instead of working on wheels. Sometimes good stuff comes out of that, but there's a ton of waste, too -- in the worst case, you end up in a company that has hundreds or thousands of people, but only a core team of a few dozen are doing all of the actual work. So you have to ask yourself: would it just be better not to hire hundreds of engineers, and cut away the complexity and inefficiency that comes with an organization of that size?
This paragraph is just a criticism of R&D, and when read that way, I think the counterarguments become obvious. Can large and/or rapidly growing technology companies (particularly Internet companies) afford to not invest heavily in R&D?
This paragraph is just a criticism of R&D, and when read that way, I think the counterarguments become obvious. Can large and/or rapidly growing technology companies (particularly Internet companies) afford to not invest heavily in R&D?
"Sometimes good stuff comes out of that, but there's a ton of waste, too"
If you have complex machines that are critical to the business and that take 6+ months before they are running at a reasonable capacity then it often makes perfect sense to have extra "dormant" capacity.
Probably not to the point where 90% of your capacity is "dormant" but IMO that is rare. A much bigger issue is all the companies running at over-capacity who are harming their long term success.
If you have complex machines that are critical to the business and that take 6+ months before they are running at a reasonable capacity then it often makes perfect sense to have extra "dormant" capacity.
Probably not to the point where 90% of your capacity is "dormant" but IMO that is rare. A much bigger issue is all the companies running at over-capacity who are harming their long term success.
Well our company believes in the power of tools to empower people and developers, and bring about social change. So I can definitely speak about this. And so can my cofounder: http://qbix.com/blog/index.php/2013/04/a-new-kind-of-platfor...
"Our job is not to just build Facebook, our job is to make the world more open and connected — and we in Product Infrastructure are tasked with giving the whole software industry the tools to help us accomplish this mission."
Then open source your social network and let it be hosted in a distributed manner around the world. Instead of shutting off access to APIs like this: http://techcrunch.com/2015/04/28/facebook-api-shut-down/
When I see companies like Twitter and Facebook become open source like Wordpress for blogs, then we can say they really care more about making the world more open and connected instead of building their own silo.
"Our job is not to just build Facebook, our job is to make the world more open and connected — and we in Product Infrastructure are tasked with giving the whole software industry the tools to help us accomplish this mission."
Then open source your social network and let it be hosted in a distributed manner around the world. Instead of shutting off access to APIs like this: http://techcrunch.com/2015/04/28/facebook-api-shut-down/
When I see companies like Twitter and Facebook become open source like Wordpress for blogs, then we can say they really care more about making the world more open and connected instead of building their own silo.
haha, no.
they want to open source their TOOLS and maybe even their model, but their data(ie their users)? gods no, that's their profit center, giving that up would be committing corporate suicide. Even if the engineers were willing & able to do that, the managers would kill it like a 1st trimester abortion.
they want to open source their TOOLS and maybe even their model, but their data(ie their users)? gods no, that's their profit center, giving that up would be committing corporate suicide. Even if the engineers were willing & able to do that, the managers would kill it like a 1st trimester abortion.
Bottom line: invest in good people, delegate responsibility to them, encourage experimentation, and focus on long-term. Facebook wasn't the first to try this model. They are one of its recent success stories, though. Most companies' IT departments won't have similar success due to control-freak management, conformance to established processes/agendas, or focus on quarterly-earnings. We'd all be better off if they switched to the model that actually produces innovation.
Even shifting a portion to it might send a hell of a shockwave through the industry.
Even shifting a portion to it might send a hell of a shockwave through the industry.
I'll state the obvious: FB has reached champagne problem status. It would be great if "most companies' IT departments" have the luxury of time to build non-core products but most companies don't have the bandwidth to "even shift a portion" of time to experiment like this.
Sure they do if they have their IT priorities right. There's a small business in my locale that I can't name that occupies one floor of an office, does all kinds of app development for big companies, and still allocates enough time to innovation that they regularly present clever stuff at conferences dedicated to what they do. That stuff also directly improves their competitiveness in most cases. People might say something similar about Jane St's Ocaml contributions that they turned into a side-effect of doing their business. There's been countless admins in companies with less vision that innovated plenty on semi-automating monitoring, management, and reliability just out of necessity to reduce stress on their overworked brains and hands.
There's tons of potential in companies for IT innovation, whether small- or large-scale. It's proven by those, even tiny firms, that act on it. Most companies act against it on top of under-funding, under-staffing, and over-stretching their IT staff. That combination causes the effect you describe. It's not inherent in any way. If it was, examples I described wouldn't exist.
There's tons of potential in companies for IT innovation, whether small- or large-scale. It's proven by those, even tiny firms, that act on it. Most companies act against it on top of under-funding, under-staffing, and over-stretching their IT staff. That combination causes the effect you describe. It's not inherent in any way. If it was, examples I described wouldn't exist.
I sort of agree with both of you. Yes, you can do it in a small shop, and my company does. A lot, actually. But I wouldn't say we exactly allocate time to innovation. It's more about taking a long view and integrating the long view mentality into projects with tight deadlines and immediate needs.
It takes a lot of work, and it feels quite different than the luxury of overt R&D and an emphasis on creating tools. We still mostly have to keep the development of tools to an as-needed basis and just take those opportunities to put an eye on the future. It's a delicate balance.
So to me the GP is not wrong when saying that these companies like Facebook have a serious luxury. That's the luxury to repeatedly experiment and fail with little relative consequence to the business. The rest of us can't afford to spend nearly as much time or make nearly as many mistakes.
It takes a lot of work, and it feels quite different than the luxury of overt R&D and an emphasis on creating tools. We still mostly have to keep the development of tools to an as-needed basis and just take those opportunities to put an eye on the future. It's a delicate balance.
So to me the GP is not wrong when saying that these companies like Facebook have a serious luxury. That's the luxury to repeatedly experiment and fail with little relative consequence to the business. The rest of us can't afford to spend nearly as much time or make nearly as many mistakes.
Good points. I agree with the point about luxury: some definitely have way more lee-way than others. Your company's method is one of the good methods I referred to. Another is Jane St's where they apparently dedicated time and labor to building better Ocaml tools (incl standard library). Almost every Global 2000 company can afford to do 100x that. Some, to their credit, are contributing things to IT community. Most just don't and even reinforce problems with their decisions.
So, there's certainly a huge gap between most companies and Facebook. Yet, even allocating one person, day, 15% time, etc. to try or build some new things might have quite some results over thousands to hundreds of thousands of companies. I'm just arguing companies could be putting in significantly more effort even without a true R&D department or luxurious business model. Past that, mileage varies from company to company esp with their circumstances.
So, there's certainly a huge gap between most companies and Facebook. Yet, even allocating one person, day, 15% time, etc. to try or build some new things might have quite some results over thousands to hundreds of thousands of companies. I'm just arguing companies could be putting in significantly more effort even without a true R&D department or luxurious business model. Past that, mileage varies from company to company esp with their circumstances.
[deleted]
Mostly because companies still look at software as being a line item on the budget rather than the foundation of their business.
The thing is whether they realize it or not every company is now has to be a software company, if they aren't then a software company will put them out of business within 10 years.
The thing is whether they realize it or not every company is now has to be a software company, if they aren't then a software company will put them out of business within 10 years.
"Our job is not to just build Facebook, our job is to make the world more open and connected — and we in Product Infrastructure are tasked with giving the whole software industry the tools to help us accomplish this mission."
If this is explicitly true, I've never heard of anything like it in a company that doesn't sell to engineering organizations, and it's pretty amazing.
If this is explicitly true, I've never heard of anything like it in a company that doesn't sell to engineering organizations, and it's pretty amazing.
They built their company on tools like this, so they do have a certain moral obligation to give back. I don't like a lot of things about Facebook, but good for them having this attitude.
Hey, original author here.
The moral obligation is real, and I personally feel quite strongly about it and I know most of my coworkers feel the same. Morality is relevant because it's a human feeling. It's easy to forget that even big companies like Facebook make decisions by the hands of single individuals.
Facebook was built on the LAMP stack >10 years ago. Without open source, Facebook wouldn't exist and we all know that.
I personally find satisfaction in sharing as much of the stuff we build as possible as open source in the hopes that others will be able to build something great.
The moral obligation is real, and I personally feel quite strongly about it and I know most of my coworkers feel the same. Morality is relevant because it's a human feeling. It's easy to forget that even big companies like Facebook make decisions by the hands of single individuals.
Facebook was built on the LAMP stack >10 years ago. Without open source, Facebook wouldn't exist and we all know that.
I personally find satisfaction in sharing as much of the stuff we build as possible as open source in the hopes that others will be able to build something great.
>so they do have a certain moral obligation to give back.
What moral obligation are you talking about? There's no moral obligation for anyone using open source software to give anything back. Are you aware of how software licenses work?
What moral obligation are you talking about? There's no moral obligation for anyone using open source software to give anything back. Are you aware of how software licenses work?
There's an interesting nuance in the phrase "a certain moral obligation." In my understanding, it's a fairly weak statement [1], somewhat akin to "People will think more highly of you if you do this thing, and less highly if you don't."
Contrast this with "they have a moral obligation." To me that is a much stronger statement: "People will think you are behaving in an immoral way if you do not do this thing."
It's funny how adding the word "certain" to the phrase actually makes it a less certain, gentler way of putting things - quite the opposite of the literal meaning of the word.
Put another way, I could imagine a law being passed to enforce "a moral obligation". But it seems much less likely to have a law enforcing something described as "a certain moral obligation."
[1] I mean "weak" in the sense of not being a strong claim of obligation, not "weak" in the sense that the phrase is a poor way to say something.
Contrast this with "they have a moral obligation." To me that is a much stronger statement: "People will think you are behaving in an immoral way if you do not do this thing."
It's funny how adding the word "certain" to the phrase actually makes it a less certain, gentler way of putting things - quite the opposite of the literal meaning of the word.
Put another way, I could imagine a law being passed to enforce "a moral obligation". But it seems much less likely to have a law enforcing something described as "a certain moral obligation."
[1] I mean "weak" in the sense of not being a strong claim of obligation, not "weak" in the sense that the phrase is a poor way to say something.
Right, I just meant in a "pay it forward" or "the decent thing to do" sense.
If you're referring to software license terms, you're making a legal, not a moral argument.
Open? Connected?
BS!
Or they mean "Connected to Facebook alone"?
BS!
Or they mean "Connected to Facebook alone"?
I did chuckle as it reminded me of the "curing cancer" talk Gavin Belson gives in the Silicon Valley series :D
Pretty fluffy piece, I'm not sure why it's so high. FB does have some pretty cool stuff though - I've been itching to play with React and GraphQL.
One tangential point I often think about is the importance of investing time in learning tools as a developer. What should the breakdown be? Should I spend 20% of my time "sharpening the ax" and the other 80% working on projects? Someone else in this thread pointed out that companies face some of the same questions. How many resources should be allocated to things like improving the build system or developing custom tools? Where is the break even point where time invested in tools starts saving time for developers working on core products?
One tangential point I often think about is the importance of investing time in learning tools as a developer. What should the breakdown be? Should I spend 20% of my time "sharpening the ax" and the other 80% working on projects? Someone else in this thread pointed out that companies face some of the same questions. How many resources should be allocated to things like improving the build system or developing custom tools? Where is the break even point where time invested in tools starts saving time for developers working on core products?
The importance of pushing the bar for tooling and infrastructure aside, this is still a laughably bombastic Facebook press release.
It's not a press release; its Lee's personal post.
I do think the author overstates their importance. All of the technologies he listed aren't exactly earth shattering; the web would be the same as it was yesterday if these technologies disappeared today. For example, react "native"; were apps not built before that? The only thing that did was let JavaScript devs write for iOS. Have apps increased in quality since react native? No. It just made it easier for JS devs to not learn Swift or Objective C. But it hasn't innovated much in terms of the user. Most of what they've done has been in Javascript frameworks; how innovative is that really? I actually like React, but it isn't like they've invented HTTP. It's just another framework. I can do some cool stuff, but nothing that could be done before. It all feels like a self congratulatory circle jerk with actual user-level improvements that would not be particularly noticeable by the average user; the Facebook UX I am presented on the web is still the same cluttered mess it has always been. So what magic is this innovation actually creating? Just because we like a particular framework doesn't elevate it to the level of earth shattering. It certainly hasn't had the effect on the web that Rails has.
I do think that Brian overstates Rails' importance. Rails and Ruby aren't exactly earth shattering; the web would be the same as it was yesterday if these technologies disappeared today. For example, RubyMotion; were apps not built before that? The only thing that did was let Ruby devs write for iOS. Have apps increased in quality since RubyMotion? No. It just made it easier for Ruby devs to not learn Objective C. But it hasn't innovated much in terms of the user. Most of what they've done has been in PHP frameworks; how innovative is that really? I actually like Rails, but it isn't like they've invented HTTP. It's just another framework. I can do some cool stuff, but nothing that could not be done before. It all feels like a self congratulatory circle jerk with actual user-level improvements that would not be particularly noticeable by the average user; the Basecamp UX I am presented on the web is still the same cluttered mess it has always been. So what magic is this innovation actually creating? Just because we like a particular framework doesn't elevate it to the level of earth shattering. It certainly hasn't had the effect on the web that PHP has.
Luckily not everybody shares your opinion, for we would still be writing assembly code if that was true.
You realize React Native has been out for about 3 months, right? Rails is nearly 10 years old. I almost think you must be joking.
There's some commentary here that's helpful, and some that's not. I don't think it's fair to say "Facebook sux lolz" when many of us work, have worked, or will work in companies that suck just as much if not more. Plus, this is just one engineer focused on a certain set of tools and products. Lee does not represent the history of Facebook's decisions. He is not Mark Zuckerberg.
It's totally fair to say, "thanks rich pplz, wut do WE do?" Though there's a ton of "rich pplz" out there that don't let engineers experiment the way that Facebook does. Perhaps Google. But the vast majority of these don't, I don't think:
http://fortune.com/2015/06/13/fortune-500-tech/
It's totally fair to say, "thanks rich pplz, wut do WE do?" Though there's a ton of "rich pplz" out there that don't let engineers experiment the way that Facebook does. Perhaps Google. But the vast majority of these don't, I don't think:
http://fortune.com/2015/06/13/fortune-500-tech/
What a ridiculous title. The reality is this is a backslapping Facebook post about bringing out new open source javascript libraries for nothing in particular with no particular benefit and a whole lot of fluff. Who were the 75 people who upvoted this? Probably Facebook employees.
Rapidly growing companies tend to over-hire, and then have bored engineers that spend lots of time creating wheel-building framework frameworks instead of working on wheels. Sometimes good stuff comes out of that, but there's a ton of waste, too -- in the worst case, you end up in a company that has hundreds or thousands of people, but only a core team of a few dozen are doing all of the actual work. So you have to ask yourself: would it just be better not to hire hundreds of engineers, and cut away the complexity and inefficiency that comes with an organization of that size?
I've seen it first-hand, and I've also seen friends go to big, famous unicorn companies and work on stuff that is far separated from the company's core business. It's sad, not invigorating.