Reducing Cognitive Load in Agile DevOps Teams Using Team Topologies(infoq.com)
infoq.com
Reducing Cognitive Load in Agile DevOps Teams Using Team Topologies
https://www.infoq.com/articles/reduce-cognitive-load-devops-teams/
31 comments
Keep your teams to max 10 people (counting the PO). Allow them to self-organize. Treat each other internal team's project the same way you'd treat an untrusted open source dependency. Ignore the article above. Thank me later.
Written by someone who's never worked for a large enterprise software company. You cannot apply simply blanket rules like this and expect them to fit every situation. Also, allowing all teams to simply "self organize" pre-supposes that these teams will make smart decisions in their organization, something in my experience which is anything but guaranteed.
Team A strongly depends on Team B's product to the point where their deliverables are impacted by Team B's, how do we collaborate in this case?
Team B depends on Team C's internal platform, how does this teams collaboration differ from Team A and Team B's situation?
Team D has a sub-team working on a highly complex sub-system (real-time ledger for example), this sub-team is growing, how should Team D interact and work with this sub-team?
Team E needs help from the SRE team with reliability and observability issues, how should these teams best work together and collaborate?
My point is, in all these situations, how these teams interact and collaborate will be subtly different based on the situation at hand. A "one size fits all" method of team dynamics simply won't work optimally in every situation. This is what books like Team Topologies are trying to help with.
Team A strongly depends on Team B's product to the point where their deliverables are impacted by Team B's, how do we collaborate in this case?
Team B depends on Team C's internal platform, how does this teams collaboration differ from Team A and Team B's situation?
Team D has a sub-team working on a highly complex sub-system (real-time ledger for example), this sub-team is growing, how should Team D interact and work with this sub-team?
Team E needs help from the SRE team with reliability and observability issues, how should these teams best work together and collaborate?
My point is, in all these situations, how these teams interact and collaborate will be subtly different based on the situation at hand. A "one size fits all" method of team dynamics simply won't work optimally in every situation. This is what books like Team Topologies are trying to help with.
You provided a lot of questions and no real answers. I already gave the answer: issue tracker, higher management prioritizes. For absolutely critical must do now changes, you ask for a direct line of communication, but only in that absolute immediate state where something is on fire.
A similar question arises from a team that depends on Qt for their GUI. How does a team solve any issues it might have coming from Qt? If you can answer that question, do the same internally.
If you are in a situation that does not work like this, you don't really have 2 teams, you have 1 dysfunctional team and no amount of buzzwords will help you.
EDIT: Also I work for a large enterprise company where these issues were identified :). If you have two teams, you have two projects, period, or you have 1 highly dysfunctional team. I've seen all the issues with Team A not being able to deliver because they depend on Team B which has 6 month release schedule.
The problem here is one of expectations. Team A should not be expected to deliver things that depend on things Team B has not yet delivered. How stupid is that even? The solution is to not ask for stupid things or realize that you're being stupid and adjust. "Hey higher management, we can't release because these guys have 6 month releases and we need their partial results now". Maybe higher management should listen?
A similar question arises from a team that depends on Qt for their GUI. How does a team solve any issues it might have coming from Qt? If you can answer that question, do the same internally.
If you are in a situation that does not work like this, you don't really have 2 teams, you have 1 dysfunctional team and no amount of buzzwords will help you.
EDIT: Also I work for a large enterprise company where these issues were identified :). If you have two teams, you have two projects, period, or you have 1 highly dysfunctional team. I've seen all the issues with Team A not being able to deliver because they depend on Team B which has 6 month release schedule.
The problem here is one of expectations. Team A should not be expected to deliver things that depend on things Team B has not yet delivered. How stupid is that even? The solution is to not ask for stupid things or realize that you're being stupid and adjust. "Hey higher management, we can't release because these guys have 6 month releases and we need their partial results now". Maybe higher management should listen?
I provided a lot of questions which you couldn't answer effectively, that's exactly my point ;)
Your "just let teams self-organize" is frankly too simplistic to cover the real-world complex environment of different team structures and roles.
How a data platform team works with stream-aligned teams is highly different from say how an enabling team, like a centrally reporting SRE team, works with stream-aligned teams. Or how a complex subsystem team (complex ml model) works with one specific team vs a team building something like "batch processing cluster self-serve/as-a-service" which can be used self serve via many teams.
Every situation requires different levels of communication, forms of communication, e.g. do you occasionally embed engineers in another team? Do you do some comms "in person"/sync versus all comms are async via Github Issues etc. Does the other team bring people to your standups or other similar meetings.
I would 100% that for certain combinations of teams, having people from one team embed in another team is massively, massively helpful, in other situations it sucks! For some teams, only relying on an API and async updates surrounding that API (requests to the api) is all the collaboration that is needed.
My point is that there is no "one size fits all situations" model here. And as someone with a lot of real-world experience in building teams and now running large Engineering orgs myself, I've seen this done well and badly many times.
Your "just let teams self-organize" is frankly too simplistic to cover the real-world complex environment of different team structures and roles.
How a data platform team works with stream-aligned teams is highly different from say how an enabling team, like a centrally reporting SRE team, works with stream-aligned teams. Or how a complex subsystem team (complex ml model) works with one specific team vs a team building something like "batch processing cluster self-serve/as-a-service" which can be used self serve via many teams.
Every situation requires different levels of communication, forms of communication, e.g. do you occasionally embed engineers in another team? Do you do some comms "in person"/sync versus all comms are async via Github Issues etc. Does the other team bring people to your standups or other similar meetings.
I would 100% that for certain combinations of teams, having people from one team embed in another team is massively, massively helpful, in other situations it sucks! For some teams, only relying on an API and async updates surrounding that API (requests to the api) is all the collaboration that is needed.
My point is that there is no "one size fits all situations" model here. And as someone with a lot of real-world experience in building teams and now running large Engineering orgs myself, I've seen this done well and badly many times.
I think I get what you mean now, you're not talking (just) about your typical overgrown software project where somehow 100 people ended up working on it (which always ends a mess, unless you split it into effectively 10 projects), but rather more multi-domain teams that do more than just code that gets reused.
That is a much harder problem, I agree. For that one I have no solution since it's not a problem I've tried to solve or seen solved.
Perhaps the article sheds some light on this more complex situation but unfortunately it's so drenched in confusing buzzwords that I cannot derive any knowledge from it. Maybe the book is a better source, I'll see if I can get my hands on it.
That is a much harder problem, I agree. For that one I have no solution since it's not a problem I've tried to solve or seen solved.
Perhaps the article sheds some light on this more complex situation but unfortunately it's so drenched in confusing buzzwords that I cannot derive any knowledge from it. Maybe the book is a better source, I'll see if I can get my hands on it.
Yeah, I just realised we might have been arguing based on a misunderstanding of each other's viewpoint! btw, I wouldn't say this book is a silver bullet by any means... it gives some ideas about how you might look at the various situations and think about how they affect comms and collaboration. I've mostly taken insight from the book and the bits they seem to make more sense, adapt them to my own org, and apply that way.
I think the best part to take away from the book are the pieces around cognitive load and how it can affect teams adversely if the team has too many "topics" or different major projects, and use that principle to break apart large teams. Then the remaining work is just looking at how those new teams communicate and collaborate in an effective manner.
I think the best part to take away from the book are the pieces around cognitive load and how it can affect teams adversely if the team has too many "topics" or different major projects, and use that principle to break apart large teams. Then the remaining work is just looking at how those new teams communicate and collaborate in an effective manner.
Regardless, this was a great conversation to witness and there was value generated by the misunderstanding - if it did occur.
Sounds like a recipe for getting a toxic Amazonian culture
I have no idea what Amazon's software development culture is like, only that they treat their warehouse employees like shit.
How would you coordinate work across teams?
The same way you "coordinate" work across your open source dependencies, you post an issue in their issue tracker. As far as those other teams are concerned, you are their customer, and they need to prioritize their work as usual.
If a team is consistently ignoring the issues posted by another and causing issues in the overall project, that's up to higher management to resolve and reprioritize things. Higher-management needs to collaborate and prioritize issues across teams, as their "own" team, since they should have a higher level overview of the project.
To make it a bit more concrete, lets say you have 10 teams of 10 people, each managed by a manager. Each of those teams is an independent "project". The 10 managers themselves form a team that has to ensure those independent projects form a coherent whole by organizing amongst themselves and prioritizing tasks across teams accordingly.
EDIT: this split can work (albeit poorly) at up to 3 levels deep, so ~1000 employees total on a "big project". If you can't make do with that many, you're screwing up somewhere, but 100 should be enough IMO.
If a team is consistently ignoring the issues posted by another and causing issues in the overall project, that's up to higher management to resolve and reprioritize things. Higher-management needs to collaborate and prioritize issues across teams, as their "own" team, since they should have a higher level overview of the project.
To make it a bit more concrete, lets say you have 10 teams of 10 people, each managed by a manager. Each of those teams is an independent "project". The 10 managers themselves form a team that has to ensure those independent projects form a coherent whole by organizing amongst themselves and prioritizing tasks across teams accordingly.
EDIT: this split can work (albeit poorly) at up to 3 levels deep, so ~1000 employees total on a "big project". If you can't make do with that many, you're screwing up somewhere, but 100 should be enough IMO.
in an organization that large, the 10 managers in your scenario would have to be some of the most selfless and honest humans, all able to communicate freely and openly without envy or ulterior motives.
In most companies those 10 managers are not a team, unfortunately. It's more like there's a "top" manager who has 10 separate sub managers of 10 teams. But those sub managers are mostly independent. At least that's how it is in most places, communication between middle managers goes up and down, not sideways.
What I'm talking about is different, it's not just 10 managers that "have to talk", it's 10 managers in the same team with the same goal. They just happen to each coordinate another 10 person team. I haven't really seen that done anywhere, but I've seen some chair shuffling that approximated it. Would love to see it taken to the logical conclusion.
What I'm talking about is different, it's not just 10 managers that "have to talk", it's 10 managers in the same team with the same goal. They just happen to each coordinate another 10 person team. I haven't really seen that done anywhere, but I've seen some chair shuffling that approximated it. Would love to see it taken to the logical conclusion.
This is a basic tenet of the book 5 Dysfunctions of a Team
> The same way you "coordinate" work across your open source dependencies, you post an issue in their issue tracker
Please let’s not confuse the record of a conversation with the conversation itself. I’ve seen tracking tools used in this way and it easily turns dysfunctional.
Please let’s not confuse the record of a conversation with the conversation itself. I’ve seen tracking tools used in this way and it easily turns dysfunctional.
Good point. Though I am suggesting a flip of the typical approach here. Rather than you first having a meeting, and then registering the task in the issue tracker, I'm suggesting the opposite. An issue is posted in the tracker, which may or may not trigger a meeting depending on severity or if more information is needed. The issue tracker is a starting point for discussion, rather than the end.
I find it exceedingly hard to tell whether headlines like these lead to articles that are in fact satirical, or not...
The use of the word 'Topology' is especially ... odd. To me, at least.
I understand that it is possible to use technical terms like this in a more casual sense. Somehow it makes me think it is an attempt to use adopt 'serious gloss' of another field.
I understand that it is possible to use technical terms like this in a more casual sense. Somehow it makes me think it is an attempt to use adopt 'serious gloss' of another field.
I've noticed people starting to use the term 'socialise' when they mean 'to share': 'And then we'll socialise these information products with stakeholders' instead of 'I'll email some documents to the managers'.
Because they think it sounds cleverer heh
Because they think it sounds cleverer heh
This isn't a new phenomenon, 70-years ago people thought serviette was posher than napkin:
https://en.wikipedia.org/wiki/U_and_non-U_English
https://en.wikipedia.org/wiki/U_and_non-U_English
If "organisation" was mainly about the relationships between things, that word would do. Unfortunately, it tends instead to refer to formal structure so another one is needed.
The article's introduction to Team Topologies is very dense and I'm not sure that it does it justice. The book is actually quite approachable – I've listened to the audiobook twice now.
The article's introduction to Team Topologies is very dense and I'm not sure that it does it justice. The book is actually quite approachable – I've listened to the audiobook twice now.
DevOps is now a parody of itself.
"Automate Ops and make sysadmins redundant"... by increasing complexity 10 times.
Now you have to hire a whole team of highly paid "cloud engineers" that spend hours at writing YAML files.
"Automate Ops and make sysadmins redundant"... by increasing complexity 10 times.
Now you have to hire a whole team of highly paid "cloud engineers" that spend hours at writing YAML files.
As a person that works in DevOps (but doesn't write YAML files all day), the problem is using the wrong solution for the scope.
I think DevOps groups a lot of times end up doing the work that the groups they support should have done for themselves. However, the working yourself out of a job is a very real thing. This is happening with DB Admins and QA primarily. Cloud technologies is displacing the rest.
If your company has 1 dev, 1 sys admin, 1 DB admin and 1 QA person, creating a DevOps team won't produce more efficiency, it will have the opposite effect.
On the 10x complexity comment, if DevOps is done correctly it's invisible to the user. In practice any decision made automatically has drawbacks. On the flip side, a lot of developers are unaware how things work or lack consistency that automation can put in place.
I think DevOps groups a lot of times end up doing the work that the groups they support should have done for themselves. However, the working yourself out of a job is a very real thing. This is happening with DB Admins and QA primarily. Cloud technologies is displacing the rest.
If your company has 1 dev, 1 sys admin, 1 DB admin and 1 QA person, creating a DevOps team won't produce more efficiency, it will have the opposite effect.
On the 10x complexity comment, if DevOps is done correctly it's invisible to the user. In practice any decision made automatically has drawbacks. On the flip side, a lot of developers are unaware how things work or lack consistency that automation can put in place.
The underlying "team topologies" idea [0] as I interpret it: Any work function evaluates to return a work product; one team's work product is another team's input to its function; the work plan for the organization looks like a distributed evaluation graph; design teams to minimize blocks on over all system throughput by factoring them into one of four functional patterns. If only value was immutable...
[0] https://teamtopologies.com/key-concepts
[0] https://teamtopologies.com/key-concepts
The sheer density of jargon and management buzzwords, none of which points to any actual argument validated by a shred of empirical evidence, is mind numbing.
Not using Agile reduces cognitive load perhaps too.
When i've seen agile in use, it's because incapable managers lack the insight or trust to let teams surface valuable work through emergent properties of what they're working on, so they, in politburo fashion, try controlling things, which just lessens the attention spent on work.
When i've seen agile in use, it's because incapable managers lack the insight or trust to let teams surface valuable work through emergent properties of what they're working on, so they, in politburo fashion, try controlling things, which just lessens the attention spent on work.
Stop using complex systems designed to solve problems you don’t and won’t have.
This has to be a joke. The sheer buzzword/BS density is just too high for this article to be real.
[deleted]