Software is narrative(infiniteundo.com)
infiniteundo.com
Software is narrative
http://infiniteundo.com/post/158336863413/software-as-narrative-1n
42 comments
I think I'll open with a dubious metaphor I just came up with and a demand that the reader immediately and wholeheartedly accept that metaphor as literal truth. That's how you grab an audience!
Yes, I write intentionally click-baity blog posts. You got me.
ajarmst's point goes beyond click-baitiness -- indeed it is about the little quote-box on the first para rather than the title. You say: "Any software system begins as a shared narrative about a problem and the people who come together around solving that problem."
I can, in fact, bend my view of facts to fit this metaphor. And if am also dubious about doing so, that's OK as it makes it all the more likely that I will learn something new from your post.
So why spoil that with some the arrogant demand that: "If you don’t accept the above proposition completely then nothing I have to say about software is going to work for you."?
I can, in fact, bend my view of facts to fit this metaphor. And if am also dubious about doing so, that's OK as it makes it all the more likely that I will learn something new from your post.
So why spoil that with some the arrogant demand that: "If you don’t accept the above proposition completely then nothing I have to say about software is going to work for you."?
Just to advocate for the devil here, I think some advice is predicated on a shared model, between the giver and the taker, of how things work. If you don’t buy the model, then you probably won’t deeply understand or agree with the arguments that would get you from the model to the thesis.
I definitely appreciated this early warning to get off the ride.
While I appreciate the spirit of what you are saying, the average time on page of 3 minutes 45 seconds (across 2k uniques) indicates most visitors are continuing past the paragraph you claim would be problematic.
Devils advocate: Do you know what the average time is to read the entire thing is? If it's 7.5 minutes, then one possible explanation is that your audience is split (50/50). Median time would be a much better indicator.
Yep. I spend a lot of time qualifying and hedging on these ideas in the work I do for software organizations. These blog posts are my opportunity to just lay my true opinions out for people and see what happens :)
The fact you thought about it enough to write 3 paragraphs about it shows the stickiness of my click-bait technique. You may not like my post but you will remember it.
The contrast here is great.
Finding the problem and creating a solution along with focusing on the process of creating that solution in order to reduce risk of building the wrong solution. Contrasted to building a solution along with a sane dev and production environment for that solution to fit in, to increase possible iteration velocity by reducing deployment inertia.
Finding the problem and creating a solution along with focusing on the process of creating that solution in order to reduce risk of building the wrong solution. Contrasted to building a solution along with a sane dev and production environment for that solution to fit in, to increase possible iteration velocity by reducing deployment inertia.
I think the OP is on a path that will eventually lead them to discover or re-invent Conway's Law[0]
[0]https://en.wikipedia.org/wiki/Conway's_law
[0]https://en.wikipedia.org/wiki/Conway's_law
Nice one! Thanks for sharing the insight
Yeah, way ahead of you on that one. http://infiniteundo.com/post/87172935713/first-rule-of-hacke...
Just wondering if I'm the only one, but I like my logs to be the narrative of what's happening in the system.
Agreed. I think many people do a good job of logging when something goes wrong, and maybe they are good about logging inputs/outputs of a system, but I think that logging important decisions often falls off the table. Ideally I would like to take a request ID, grep the logs, and get an entire story of what happened to that request. In reality, this rarely happens!
> take a request ID, grep the logs, and get an entire story of what happened to that request. In reality, this rarely happens!
If that's the situation you find yourself in, I cannot praise centralized logging with a good frontend highly enough because I frequently find myself trying to figure out what happened to a request, and it's like night and day.
Needing to ssh anywhere and run grep against log files is functional if there's only one or a handful of VMs, but it gets complicated with a handful of machines, and even just SCP-ing the logs off becomes time consuming if there are a lot of machines. Then once the logs are off, 'grep' quickly becomes inadequate. (And I should know, I've built some truly horrible regexps to try and grep for dates because I didn't know any better.)
All that friction means that answering the original question; figuring out a detailed internal reason for why my customer received a 500 http status response error, is just too toilsome for all but the most (as you noted) doesn't happen in .
With centralized logging, I'm able to search for a request ID and see the logs, and this is a reality as often as I need, in order to debug complex multi-system issues.
If that's the situation you find yourself in, I cannot praise centralized logging with a good frontend highly enough because I frequently find myself trying to figure out what happened to a request, and it's like night and day.
Needing to ssh anywhere and run grep against log files is functional if there's only one or a handful of VMs, but it gets complicated with a handful of machines, and even just SCP-ing the logs off becomes time consuming if there are a lot of machines. Then once the logs are off, 'grep' quickly becomes inadequate. (And I should know, I've built some truly horrible regexps to try and grep for dates because I didn't know any better.)
All that friction means that answering the original question; figuring out a detailed internal reason for why my customer received a 500 http status response error, is just too toilsome for all but the most (as you noted) doesn't happen in .
With centralized logging, I'm able to search for a request ID and see the logs, and this is a reality as often as I need, in order to debug complex multi-system issues.
Word of advice: the 'jq' tool for handling JSON files (couple with a glob like '*.log' or something fancier with xargs or parallel) will absolutely save your bacon in those situations. It's way more powerful than it appears on the surface.
We had a series of Docker json-file driver log files. It's done as a raw list (no array around it) of JSON objects -- which is a bit annoying to sort and filter based on properties of the objects.
'jq '[inputs]' (asterisk).log > combined.json' was my favourite command today; it combines all the files inputs and wraps them in an array correctly. No awk needed!
Combine that with its cute:
jq '.someProp as $var | test("some search"; "gi") as $r | if $r then ($var + $__loc__) else null end' (asterisk).log | grep -v "^null$" > filtered.json
And you're away to the races. Can then load the file directly in and group_by(.somePath) and it will all magically work!
Edit: had to remove the actual asterix symbols as they screw with formatting but are used for globbing the file names. Replace with the real character
We had a series of Docker json-file driver log files. It's done as a raw list (no array around it) of JSON objects -- which is a bit annoying to sort and filter based on properties of the objects.
'jq '[inputs]' (asterisk).log > combined.json' was my favourite command today; it combines all the files inputs and wraps them in an array correctly. No awk needed!
Combine that with its cute:
jq '.someProp as $var | test("some search"; "gi") as $r | if $r then ($var + $__loc__) else null end' (asterisk).log | grep -v "^null$" > filtered.json
And you're away to the races. Can then load the file directly in and group_by(.somePath) and it will all magically work!
Edit: had to remove the actual asterix symbols as they screw with formatting but are used for globbing the file names. Replace with the real character
True but even with a centralized logging system, if the logs are not good enough you can find yourself still wondering what the hell happened. Grep here is just the tool to extract the "story".
Very good. I would add that I like my git commit messages to be narrative of the development process. Hmmm...I wonder if someone could write a science fiction short story using only git commit messages?
This is a good idea.
That's where I was going when I wrote this post. Logging and log analysis is a large part of the continuous delivery tooling that I've been working on for the past 7 years.
Sorry I just don't understand the article. Can anyone rephrase?
The main point of the article appears to be:
> You always have to design both “the product” your customers want and “the environment” in which your product will run in production. Thus any software product begins with two obvious categories of “work to be done.” People ignore this because it seems [counterintuitive.][going_there]
> You always have to design both “the product” your customers want and “the environment” in which your product will run in production. Thus any software product begins with two obvious categories of “work to be done.” People ignore this because it seems [counterintuitive.][going_there]
Hey you saved everyone a click!
I am the author of the post and I approve this summary :)
I am the author of the post and I approve this summary :)
I think I took the metaphor more to heart than others. I like to think of myself as a good storyteller. I can explain pretty in depth computer science topics to my business team who lack technical knowledge pretty well through stories. My problem seems to be translating this to software structures.
I wish I could be as good of a "writer" in my software, but I am always looking to others for the best practices. In this case, I am always in a constant state of doubt and "writers block", I need to figure out how to unleash my own creativity.
I wish I could be as good of a "writer" in my software, but I am always looking to others for the best practices. In this case, I am always in a constant state of doubt and "writers block", I need to figure out how to unleash my own creativity.
Your blog archive (where the title links to) seems to be broken.
Weird it just worked fine for me. http://infiniteundo.com/archive/filter-by/text
Oh, don't mind. My network connection was the one that had a problem.
what does [counterintuitive.][going_there] mean?
Based on the other usages in the article, it looks like some kind of link syntax like in markdown or wikitext, but it isn't being parsed/replaced. If so, then it just means "counterintuitive."
LOL I am the author and I didn't know what that comment meant when I saw it above!! Derp and I will go fix my broken links now :) Thanks for the heads up!
This is a good metanarrative for software authors. The more the product wants to be art, the more important this concept is. Humans are wired to spread stories so if a product doesn't communicate plenty of good ones, then there you have the anwer to why marketing budget isn't fixing the business.
Okay, I'll take it at face value. Can you name a few competent programmers who are also competent book / story writers ?
I create video training courses on tech topics and one of the metrics I use to determine "is this performing well?" is the completion rate of the courses and individual videos.
I started taking communication and story telling pretty seriously about 9 months ago and the data on my 2 latest courses (where I tried to create a cohesive narrative of the content) shows the completion rate being quite a bit higher than my older courses.
Are there other things at play? Maybe, but I'm 100% convinced it takes more than "raw programming talent" to be a good programmer in general.
When it comes to designing library code or public APIs, I would rather use code written by someone with 5 years of writing / story telling experience and 2 years of coding experience vs someone with 10 years of coding experience but can't communicate well at all.
Chances are the 2nd person wouldn't be able to see the "big picture" stuff when it comes to designing APIs, and when it comes to API design, the big picture is the most important thing. The implementation details of each function is the easy stuff (since you're a Google search away from solving most technical problems).
There's also many other factors that matter besides your code. For instance, if I can't even find your project, or your documentation is lack luster then it doesn't matter if you wrote the most elegant code in the universe, I'm not going to use it.
I started taking communication and story telling pretty seriously about 9 months ago and the data on my 2 latest courses (where I tried to create a cohesive narrative of the content) shows the completion rate being quite a bit higher than my older courses.
Are there other things at play? Maybe, but I'm 100% convinced it takes more than "raw programming talent" to be a good programmer in general.
When it comes to designing library code or public APIs, I would rather use code written by someone with 5 years of writing / story telling experience and 2 years of coding experience vs someone with 10 years of coding experience but can't communicate well at all.
Chances are the 2nd person wouldn't be able to see the "big picture" stuff when it comes to designing APIs, and when it comes to API design, the big picture is the most important thing. The implementation details of each function is the easy stuff (since you're a Google search away from solving most technical problems).
There's also many other factors that matter besides your code. For instance, if I can't even find your project, or your documentation is lack luster then it doesn't matter if you wrote the most elegant code in the universe, I'm not going to use it.
Coetzee comes to mind. Ted Chiang is a technical writer in the software industry, and while I don't know if that means he's a competent programmer, it seems logical that he can do some programming.
I believe Ken MacLeod was a programmer and he is an great writer (along with Stross and Vinge mentioned in a sibling comment)
https://en.wikipedia.org/wiki/Ken_MacLeod
https://en.wikipedia.org/wiki/Ken_MacLeod
Michał Cholewa - writes sci-fi book series as well as machine learning papers.
[deleted]
Zachary Mason
Ellen Ullman
Jon Evans
Ellen Ullman
Jon Evans
Vernor Vinge and Charles Stross spring immediately to mind. Also James Tiptree, Jr. was a CIA analyst in the 50s and 60s so probably was a regular user of the early government mainframe systems.
The Martian was fantastic and the author is a programmer. I recently got hooked on the Bobiverse books and their author is a programmer too. In fact, I wholly recommend anyone reading this to check both of those out :) Super fun books