Ask HN: Is there a better way to document complex software architectures?
238 comments
Quick question that I only ask because of my 30-odd year history as a corporate contractor...
Does anyone here really use those UML-based docs to actually learn the systems? Or do they do what I always did and use the trusty step-debugger for a few days and come up with your own mental model of them?
Inevitably, unless someone is working full time on them, they are incomplete and several months out-of-date, and that makes them worthless really.
That's my anecdotal evidence against generating them. I'm at least somewhat sure someone has a conflicting story of how UML "saved the day", but it would be so rare to my experience that I would love to hear it.
Does anyone here really use those UML-based docs to actually learn the systems? Or do they do what I always did and use the trusty step-debugger for a few days and come up with your own mental model of them?
Inevitably, unless someone is working full time on them, they are incomplete and several months out-of-date, and that makes them worthless really.
That's my anecdotal evidence against generating them. I'm at least somewhat sure someone has a conflicting story of how UML "saved the day", but it would be so rare to my experience that I would love to hear it.
Completely agree. Systems change, people that work on them change. A diagram gets obsolete very quickly.
A diagram is mostly valuable in the design phase for me and to have faster less ambiguous discussion with people.
In the systems that do not drastically change, it is sometimes useful later to understand how the system was initially designed by new people.
But for specific part of the system I always have to look into the code on source control. Git blame and diffing commits have become the only good tool able to tell me what I need and also help me to realize what changed overtime, by who and when.
I remember seeing some version control tools for diagrams but unfortunately unless the design is updated as the code gets updated, these tools are worthless.
Comments that are in the code get outdated. There is little hope with diagrams.
The exceptions are tools that get their data from the system. In a company I was working long time ago, we had a GUI to connect parts of the systems. The changes in the UI became real changes in the system. That data was always updated.
For MySQL databases in the past I used a tool that was able to create the table diagram from the schema in MySQL but I had to spend quite some time to rearrange the tables with drag and drop. It worked of if people defined foreign keys and kept the naming consist.
A diagram is mostly valuable in the design phase for me and to have faster less ambiguous discussion with people.
In the systems that do not drastically change, it is sometimes useful later to understand how the system was initially designed by new people.
But for specific part of the system I always have to look into the code on source control. Git blame and diffing commits have become the only good tool able to tell me what I need and also help me to realize what changed overtime, by who and when.
I remember seeing some version control tools for diagrams but unfortunately unless the design is updated as the code gets updated, these tools are worthless.
Comments that are in the code get outdated. There is little hope with diagrams.
The exceptions are tools that get their data from the system. In a company I was working long time ago, we had a GUI to connect parts of the systems. The changes in the UI became real changes in the system. That data was always updated.
For MySQL databases in the past I used a tool that was able to create the table diagram from the schema in MySQL but I had to spend quite some time to rearrange the tables with drag and drop. It worked of if people defined foreign keys and kept the naming consist.
For groking relational databases and schemas, I find Dbeaver is good for producing an entity relation diagram in relation to a specific table (as well as an overall ERD)
I've been developing software professionally for 25 years. I've never found them to be useful. Even if you have them, they're usually out of date, and anyway they don't really help answering the questions you have. None of the places I have worked at have put a lot of emphasis on UML diagrams.
I tend to write up (not necessarily formal UML) diagrams in 2 cases:
1. I'm brainstorming with myself about how I want the software to behave
2. The software is basically done & tested, it's unlikely to change, and communicating the behavior to other people is of value, especially when describing a system that spans multiple components
1. I'm brainstorming with myself about how I want the software to behave
2. The software is basically done & tested, it's unlikely to change, and communicating the behavior to other people is of value, especially when describing a system that spans multiple components
I'll add (3) When I'm learning a new system, especially if it's big and/or old.
The idea is that if I go to the trouble to document it then the next person won't have to; this is seldom the case because as stated above the get out of date so fast. It's still a great way to learn and confirm your understanding though.
The idea is that if I go to the trouble to document it then the next person won't have to; this is seldom the case because as stated above the get out of date so fast. It's still a great way to learn and confirm your understanding though.
(4) Security audits
Just 15 years here. I've worked at somewhere where they did put a lot of emphasis on UML diagrams.
To the point they had a product where some of the code was generated from UML, not once moving from the initial design to code, but repeatedly every time the design became out of date. It became the project everyone tried to get away from and project all new joiners were thrown into the deep end on, many quitting as a result.
A couple of years ago I interviewed for a contract role with a sister company and they waxed lyrical about their love of "best practice" UML in the interview. I declined the offer, I'm too old for worrying about which arrow type to use when no one reading it's going to understand the significance anyway. I think there are a lot of attempts to move away from code because it's complex, but the bottom line is it's the best we have for now.
To the point they had a product where some of the code was generated from UML, not once moving from the initial design to code, but repeatedly every time the design became out of date. It became the project everyone tried to get away from and project all new joiners were thrown into the deep end on, many quitting as a result.
A couple of years ago I interviewed for a contract role with a sister company and they waxed lyrical about their love of "best practice" UML in the interview. I declined the offer, I'm too old for worrying about which arrow type to use when no one reading it's going to understand the significance anyway. I think there are a lot of attempts to move away from code because it's complex, but the bottom line is it's the best we have for now.
What would be useful is the ability to generate diagrams like UML from an AST. I'm sure theres already a Rust crate for it out there somewhere...
Then it could just be the case where you pick out a root struct / mod / something and a depth and let the software generate you a pretty graph to stick on a slide for the meeting in 15.
Then it could just be the case where you pick out a root struct / mod / something and a depth and let the software generate you a pretty graph to stick on a slide for the meeting in 15.
A lot of tools exist for generating UML from code (and some for generating code from UML).
I've never found any of the generated UML to be helpful (too big, too many details, and too much effort for me to simplify). Funny enough, Visual Studio cut out the builtin UML designer from VS 2017.
I've never found any of the generated UML to be helpful (too big, too many details, and too much effort for me to simplify). Funny enough, Visual Studio cut out the builtin UML designer from VS 2017.
Ditto, except that I have 30 years professional experience.
As someone who's been in software development for only a couple years by comparison, I don't see huge architectural changes happen too often for any of our products. This is something that I'm currently trying to improve upon within my company, but I've gotten pretty discouraged with all these comments lol. Is there any way to mitigate the obsoletion of these models?
Make a tool that generates architectural diagrams from the code?
This is why I dislike it when technical documentation itself becomes a goal. Detailed documentation never seems to be up to date in the real world. I've had far better experience with concise documention about high level concepts and structures and diving into the actual implementation for any details.
It's pretty straightforward - if it's pdf it's not versioned. If it's not versioned it's a fossil. The only thing I've ever done to reason about a new system is read the tests.
Ha-ha, the joke is on you, the systems I have inherited do not have tests.
Joke aside, tests are a great resource when they are present. The other technique I have used was to look at the system like a black box, and only look at the expected behavior at the boundary. Sometimes, it is the fastest way to clear old cruft.
Joke aside, tests are a great resource when they are present. The other technique I have used was to look at the system like a black box, and only look at the expected behavior at the boundary. Sometimes, it is the fastest way to clear old cruft.
These days I preach the gospel of the project README: One file, Markdown format, versioned with the code, that says what the project does with high-level concepts, and how to build it and run the tests.
I second that. Reading tests is often more efficient than trying to learn about a system through documentation.
maybe - but the codethat's the hardest to understand (age, size, immature) is often the least tested, or worse the tests have been gamed over time because the previous assumptions are no longer relvevant & the code has changed so much new developers just want them to pass.
related aside - In my experience developers are much more likely to cut out old code than they are to remove or refactor irrelevant tests
related aside - In my experience developers are much more likely to cut out old code than they are to remove or refactor irrelevant tests
I really appreciate having a high-level diagram of system architecture when joining a new project. It gives me an instant feel for whats going on and even if its slightly out of date it gives context to the evolution and design decisions of the codebase since then. That said, there's nothing that beats the good ol' step debugger and diving into the code first hand. They're complementary. Without the diagram its kind of like building a mental model of a room by feeling it out in the dark.
Edit: I'm not too strict on UML exactness, as long as it gives a feel for the components, data flow, etc.
Edit: I'm not too strict on UML exactness, as long as it gives a feel for the components, data flow, etc.
And when you can't debug... since no person in the company can build the code in Eclipse, or run it locally.. you look at the gigantic logs full of soap xml dumps. At least that's my experience.
I would use them if they existed. I usually spend the first couple months of my tenure with a new team documenting the product after I ask a few questions and inevitably people start giving me answers that are rarely consistent.
After those diagrams are created, usually everyone is SHOCKED that discussing the system becomes so much easier.
Unfortunately, modern software teams don't value documentation and that's the source of a lot of miscommunication, slowdowns and frustration.
After those diagrams are created, usually everyone is SHOCKED that discussing the system becomes so much easier.
Unfortunately, modern software teams don't value documentation and that's the source of a lot of miscommunication, slowdowns and frustration.
"WORKING PRODUCT, over comprehensive documentation"
Software changes too fast to make a static snapshot of it. Its the same with comments in the code, few months later what was there is no longer valid and only creates confusion.
But i can understand managers dont like that because it makes it harder to replace ppl.
Software changes too fast to make a static snapshot of it. Its the same with comments in the code, few months later what was there is no longer valid and only creates confusion.
But i can understand managers dont like that because it makes it harder to replace ppl.
> Software changes too fast to make a static snapshot of it.
Architectures don't really change that fast or often, except maybe in the initial phase. That's why most UML diagrams are a waste, they are concerned with low-level details like classes or internal interfaces.
Architectures don't really change that fast or often, except maybe in the initial phase. That's why most UML diagrams are a waste, they are concerned with low-level details like classes or internal interfaces.
Agreed, the architecture diagrams I usually look for are high level, mostly about how different systems are connected. These don't change that much.
When looking at architecture diagrams, usually you don't want them lower level details anyway, they would just obscure the big picture.
When looking at architecture diagrams, usually you don't want them lower level details anyway, they would just obscure the big picture.
I worked on a project that needed to implement regulations specified in legal documents. An analyst did work on reading hundreds of pages of legislation to extract UML diagram representing the objects and their relation sand also all constrains defined by the legislation. In this case, it was quite useful to have UML as an intermediary and not force developers to be jurist. The UML documentation was checked and validated by expert of the domain.
They are useful in articulating high-level insight on how systems integrate together.
A well done diagram would have an as of date, and some commentary along side boxes.
A well done diagram would have an as of date, and some commentary along side boxes.
>> A well done diagram would have an as of date
maybe we should promote a "best before" date for technical artifacts...
maybe we should promote a "best before" date for technical artifacts...
How would author anticipate the freshness of that diagram? Not possible I say
UML Sequence diagrams are useful, if they are kept up to date and reflect reality. I find they are great for the initial "what the hell is this doing?" stage of coming to something new.
The best way I like documentation is via a docs/ directory within the repo. Uing js-sequence-diagrams ( https://bramp.github.io/js-sequence-diagrams/ ) to create self-contained one page diagrams (like this https://jsbin.com/jusiyusowu/edit?output ) which go into code.
That allowed me to include documentation updates in the code review.
That allowed me to include documentation updates in the code review.
Yes.
- But is "out-of-date" the problem? "round-trip" UML tools, auto-generating UML from code, seemed like a great idea once (e.g. Rational, Together), auto-keeping UML up-to-date. They don't seem popular now.
- Is UML the problem? (IMHO: no. It's good enough to be useful; like, no matter how bad XML is, it does work).
- Or is it that the devil is in the details, and you just can't summarize code?
IMHO, docs of architecture would be the most useful of all docs. Design patterns, javadoc-like tools, type-systems and UML all try it.
In the end, understanding the domain problem is the key to understanding the solution approach, which is key to understanding its architecture. That's a lotta understanding.
In practice, I inspect source, stepping through manually - your way's better.
- But is "out-of-date" the problem? "round-trip" UML tools, auto-generating UML from code, seemed like a great idea once (e.g. Rational, Together), auto-keeping UML up-to-date. They don't seem popular now.
- Is UML the problem? (IMHO: no. It's good enough to be useful; like, no matter how bad XML is, it does work).
- Or is it that the devil is in the details, and you just can't summarize code?
IMHO, docs of architecture would be the most useful of all docs. Design patterns, javadoc-like tools, type-systems and UML all try it.
In the end, understanding the domain problem is the key to understanding the solution approach, which is key to understanding its architecture. That's a lotta understanding.
In practice, I inspect source, stepping through manually - your way's better.
I'd like to support this statement that I do not remember the last time I used them as well...
Most of the time, they are accompanied by company (or team) specific TLAs that I do not understand, at the time of starting, anyway.
For instance, 1 year ago I joined NXP, which is a HW oriented company. I tried to study the UMLs, which for instance, described how CMSIS packs are delivered and how they are split into DFPs, BSPs and SWPs. At that time I had no idea what the documentation is about and I've rather started incrementally building my mental image of the whole system. Looking at the UMLs now, I understand them, but I can still clearly see that they will only confuse a newcomer.
Most of the time, they are accompanied by company (or team) specific TLAs that I do not understand, at the time of starting, anyway.
For instance, 1 year ago I joined NXP, which is a HW oriented company. I tried to study the UMLs, which for instance, described how CMSIS packs are delivered and how they are split into DFPs, BSPs and SWPs. At that time I had no idea what the documentation is about and I've rather started incrementally building my mental image of the whole system. Looking at the UMLs now, I understand them, but I can still clearly see that they will only confuse a newcomer.
I have. When they are up to date they are useful and easy. There is only one time I've seem them up to date: the UML was used to generate the headers (we used C++ at the time).
If the build process doesn't force the UML to be kept up to date, then you should only do white board uml: draw the diagrams on a whiteboard, and erase the whiteboard two weeks latter.
If the build process doesn't force the UML to be kept up to date, then you should only do white board uml: draw the diagrams on a whiteboard, and erase the whiteboard two weeks latter.
In certain cases they are required. For example - compliance audits of varying flavor require architecture diagrams. To be clear, I expect that auditors understand less than 1% of what they look at in a diagram or why certain systems look the way they do, but it is a requirement none the less.
Planning is highly useful, plans are useless.
“Show me your flowcharts (source code), and conceal your tables (domain model), and I shall continue to be mystified; show me your tables (domain model) and I won’t usually need your flowcharts (source code): they’ll be obvious.”
~ Fred Brooks, “The Mythical Man Month”
Stuff that has improved or the potential to improve the documentability of modern systems, imo: postgREST, custom types (e.g. domains in postgresql), more expressive type systems e.g. rust, haskell, typescript perhaps (though I am not completely sold on typescript yet myself) nix/nixos (or more mainstream, hashicorp products/ansible/puppet/docker) kubernetes and co. (most people probably don't need)
~ Fred Brooks, “The Mythical Man Month”
Stuff that has improved or the potential to improve the documentability of modern systems, imo: postgREST, custom types (e.g. domains in postgresql), more expressive type systems e.g. rust, haskell, typescript perhaps (though I am not completely sold on typescript yet myself) nix/nixos (or more mainstream, hashicorp products/ansible/puppet/docker) kubernetes and co. (most people probably don't need)
But my domain model is
Store
=====
Id int
Col1 varchar
Col2 varchar
Col3 varchar
...
Col134 varchar
foreignKey1 varchar
foreignKey1Type varchar
foreignKey2 varchar
foreignKey2Type varchar
...
foreignKey10 varchar
foreignKey10Type varchar
validFrom varchar
validTo varchar
isActive boolean
deleted booleanI don't need your flowcharts, I'll be heading for the door!
Ah, it’s good they used varchar, at least now you can store everything in those columns.
And there’s a multitude of ways to soft-delete too!
This is much better than some schemas I’ve seen.
And there’s a multitude of ways to soft-delete too!
This is much better than some schemas I’ve seen.
Not 2 years ago, I saw an entire Joomla schema converted to Microsoft Excel tables which we should use to port the website. Our team just ignored the effort.
Probably the most important part of that effort was to keep the project managers and analysts busy fighting over that nonsense, providing aircover for the devs to actually do the work.
Good guess but this is third world country public school level. The friendly sextagenarian did his best but he was only comfortable with PHP and Excel.
Looks like a schema to allow the code to decide the real schema at runtime. A poor mans mongodb.
This is pretty common looking for things like CRMs or CMSes where customer specific custom fields rule the day.
The database was meant to be the CMS and SQL was designed to be human readable.
Counterpoint: If small design decisions require global SQL schema changes, it slows you down. And small barriers to change will, in aggregate, kill you.
Eg, over a month of iterations: "Can we add another field to user registration? It should be free text. Oh, they should select from a list. No wait, make it a date. Actually a date and a text field. No wait, dump the text field."
Sometimes slowing down these changes is a good thing. But often, being able to ship each iteration along the way will help your design team converge on the best solution. If you need to write a schema migration for every iteration, it kills the ability of your team to experiment and explore the space of "what should our product actually look like".
Faster iterations = more iterations = better product in the long term. Sometimes the best long term result requires some random stuff getting crammed into a big JSON field in your database in the short term.
Eg, over a month of iterations: "Can we add another field to user registration? It should be free text. Oh, they should select from a list. No wait, make it a date. Actually a date and a text field. No wait, dump the text field."
Sometimes slowing down these changes is a good thing. But often, being able to ship each iteration along the way will help your design team converge on the best solution. If you need to write a schema migration for every iteration, it kills the ability of your team to experiment and explore the space of "what should our product actually look like".
Faster iterations = more iterations = better product in the long term. Sometimes the best long term result requires some random stuff getting crammed into a big JSON field in your database in the short term.
Sounds like the real problem is that there's too much friction with schema migrations.
This is only a slight exaggeration of real things that exist in products like SharePoint or Visual Studio Team Services
That's scary to even read
> But my domain model is...
No it isn't. That's the model for the representation of the actual domain model. The actual domain model may well be "idk everything" (then you're SOL) or it just may not be formalized (then you might want to fix that).
No it isn't. That's the model for the representation of the actual domain model. The actual domain model may well be "idk everything" (then you're SOL) or it just may not be formalized (then you might want to fix that).
Absolutely this! No matter what the system is, the important part is being able to laser down to any lower levels of abstraction and reason about the specifics there. Higher level abstraction docs are summaries or getting started guides at best.
Agreed as well, though there is also the case of communicating the high level flows to others that are not software experts so that everyone has a common understanding and doesn’t have to rely on a few key developers with the knowledge. Most smart folk who aren’t developers reason pretty well about architectures so you want them to be self sufficient too.
Also as a user! User documentation should imho explain what are the datastructures that a piece of software is manipulating, and what the dependencies between them are.
The rest of the UI, I can figure out by myself.
The rest of the UI, I can figure out by myself.
There are exceptions though, where the domain model is trivial, but you will be mystified without the sequence diagrams. Paxos for example has a trivial domain model (a value that's supposed to be kept identical on all nodes in a distributed system) but the consensus algorithm is non-trivial enough that a lot of people claim you should not even try to implement it (and opt for Raft)
Well, you need to use an architecture modelling tool to work [efficiently] with architecture models.
Here are some I know of:
Sparx’s Enterprice Architect
NoMagic’s MagicDraw
Qualiware
Achimate (the tool, not the standard)
There are surely more out there. I know both Sparx and MagicDraw have the possibility to write your own custom plugins for the tool. If you are in a big enough shop that will become very handy at some point. Qualiware have better publication options than the others, as far as I know, but not completely sure. The Archimate tool probably only supports the Archimate notation standard, where as the others support BPMN, UML, DMN etc. So I’d go for those, unless you are really heavy into TOGAF, then Archimate may make sense.
Edit: these are “big” tools that can do alot of differnet stuff and you wont ever need all of the features in one of them. But also means you need to dedicate some time to learn how to use it (alot more time than you needed to learn PP or Visio)
Here are some I know of:
Sparx’s Enterprice Architect
NoMagic’s MagicDraw
Qualiware
Achimate (the tool, not the standard)
There are surely more out there. I know both Sparx and MagicDraw have the possibility to write your own custom plugins for the tool. If you are in a big enough shop that will become very handy at some point. Qualiware have better publication options than the others, as far as I know, but not completely sure. The Archimate tool probably only supports the Archimate notation standard, where as the others support BPMN, UML, DMN etc. So I’d go for those, unless you are really heavy into TOGAF, then Archimate may make sense.
Edit: these are “big” tools that can do alot of differnet stuff and you wont ever need all of the features in one of them. But also means you need to dedicate some time to learn how to use it (alot more time than you needed to learn PP or Visio)
If you want to keep everything up to date I recommend checking your diagrams in with your code. GitLab has support for plantUML https://docs.gitlab.com/ee/administration/integration/plantu... and Mermaid https://docs.gitlab.com/ee/user/markdown.html#mermaid
To add to this: I highly recommend reading into INCOSE's standards and recommendations on SysML.
Start by going through one of their last talks: https://sdincose.org/rsvpmaker/state-of-sysml-2018-09-12/
Start by going through one of their last talks: https://sdincose.org/rsvpmaker/state-of-sysml-2018-09-12/
EA is effectively a database of design artifacts, models and components so you can draw different viewpoints of a system and previously defined relationships are automatically displayed. Plantuml, which I love, can't do any of that.
EA also auto generates documentation in Office formats which is handy when submitting a design for review and comment.
A lot of other commentators will point out that UML is redundant, but in big enterprise system development it is still a useful design tool to start a discussion and not too onerous to keep up to date. Whilst even at enterprise scale you still want to evolve architecture as per Agile best practice, things take a little longer and there are more hoops to jump through and this is where well documented architecture always shines through.
EA also auto generates documentation in Office formats which is handy when submitting a design for review and comment.
A lot of other commentators will point out that UML is redundant, but in big enterprise system development it is still a useful design tool to start a discussion and not too onerous to keep up to date. Whilst even at enterprise scale you still want to evolve architecture as per Agile best practice, things take a little longer and there are more hoops to jump through and this is where well documented architecture always shines through.
I agree that the first important step is from drawing to modelling. Having a single model means that all your drawings are (and stay) consistent.
The next step is the integration with the surroundings to keep the architecture model consistent with the code and the requirements. For the code, the usual way is generate code from the model. That is tricky and has a bad reputation because it is hard to get right. For requirements, the question is whether you actually track them (probably no in SaaS/Web environment; probably yes in embedded).
What I'm not sure about: In theory the integration tests should be derived from the software architecture, but I have you to see even ideas for tool support there.
The next step is the integration with the surroundings to keep the architecture model consistent with the code and the requirements. For the code, the usual way is generate code from the model. That is tricky and has a bad reputation because it is hard to get right. For requirements, the question is whether you actually track them (probably no in SaaS/Web environment; probably yes in embedded).
What I'm not sure about: In theory the integration tests should be derived from the software architecture, but I have you to see even ideas for tool support there.
Nice thing about e.g. EA is that your aren't just drawing pictures. You're able to collaborate on models that are surfaced on views, so a change is global and you can see where else e.g. a component is used.
Dia/visio for general diagrams and ArgoUML/plantUML for sequence diagrams. I found that the only useful UML diagram is the sequence diagram, that really needs a dedicated tool to draw.
PlantUML has simple text markup to describe diagram. It's easier than drawing and it can be integrated in documentation and generated on the fly (sphinx, wiki, doxygen).
And last but not least, ascii diagrams with http://asciiflow.com/ . The ASCII art diagram is the only format that can be integrated flawlessly into any email or documentation.
https://en.wikipedia.org/wiki/ArgoUML https://en.wikipedia.org/wiki/Dia_(software) http://plantuml.com/
PlantUML has simple text markup to describe diagram. It's easier than drawing and it can be integrated in documentation and generated on the fly (sphinx, wiki, doxygen).
And last but not least, ascii diagrams with http://asciiflow.com/ . The ASCII art diagram is the only format that can be integrated flawlessly into any email or documentation.
https://en.wikipedia.org/wiki/ArgoUML https://en.wikipedia.org/wiki/Dia_(software) http://plantuml.com/
Monodraw is great for ASCII art https://monodraw.helftone.com/
> The ASCII art diagram is the only format that can be integrated flawlessly into any email or documentation.
Certain email viewers like to ... "re-format" even fixed-width plain text and thus mis-render even ASCII art (looking at you, Outlook). Of course, every time I've received such reports, I've been able to brush the problem aside by simply asking people to use a different email viewer, or view the docs in a browser proper.
Certain email viewers like to ... "re-format" even fixed-width plain text and thus mis-render even ASCII art (looking at you, Outlook). Of course, every time I've received such reports, I've been able to brush the problem aside by simply asking people to use a different email viewer, or view the docs in a browser proper.
I use Outlook all the time. Format the text with Courier New to have fixed width.
> Format the text with Courier New to have fixed width.
This succinctly captures a common problem with the MS ecosystem of products, from ages ago: the belief that "everybody uses MS". Yes, Courier New may be a fixed-width font, but I don't want to force the recipients of my emails to have Courier New, nor do I want the content I'm writing to show up in Courier New and Courier New alone, on someone else's computer. I want it to show in whatever font they have set as their monospace font. Same goes for when I view my own mails on my own computer.
This succinctly captures a common problem with the MS ecosystem of products, from ages ago: the belief that "everybody uses MS". Yes, Courier New may be a fixed-width font, but I don't want to force the recipients of my emails to have Courier New, nor do I want the content I'm writing to show up in Courier New and Courier New alone, on someone else's computer. I want it to show in whatever font they have set as their monospace font. Same goes for when I view my own mails on my own computer.
+1 for ArgoUML. It's been a solid modelling tool in my experience. By using it for capturing key abstractions and avoiding code generation/reverse engineering diagrams, keeping the diagrams up to date is made easier.
The Art of Visualising Software Architecture by Simon Brown is an interesting presentation on this topic:
https://www.youtube.com/watch?v=zcmU-OE452k
He references this other talk by Adam Tornhill on a similar topic:
https://www.youtube.com/watch?v=XzsXvsHcjc0
https://www.youtube.com/watch?v=zcmU-OE452k
He references this other talk by Adam Tornhill on a similar topic:
https://www.youtube.com/watch?v=XzsXvsHcjc0
Along with this, Simon has developed Structurizr, which uses code to describe architecture and then visualize it at different levels:
https://structurizr.com/
https://structurizr.com/
Informative and insightful videos. Thanks for sharing.
Full disclosure: I am one of the developers of a product which does the below, also known as model driven architecture.
For about 5 years now, the teams I have been on have found great success in the use of DDD. Towards this we have used a tool which not only allows us to visually model the domain using UML, but also performs what we call “code management” (smarter code-gen which doesn’t get in your way, or produce ugly code) which turns the UML into code for our ORM of choice.
All members of the team are required to use the tool and UML to create new domain entities which then automatically become (beautifully formatted) code.
The enormous advantage of this approach is that the documentation (in the form of UML in this case), never goes out of date and lives with the code, it’s literally checked in to our Git repo in an SCM friendly format next to the normal code.
We are always able to talk around these visual models when planning a new feature or strategising a technical solution for a requirement.
An architect can at any time reason about the domain without having to first build up a mental model of it from code.
When juniors ask questions of seniors, they can both talk around the UML so that they quickly get clarity on the best way forward.
I would never go back to building business software any other way.
For about 5 years now, the teams I have been on have found great success in the use of DDD. Towards this we have used a tool which not only allows us to visually model the domain using UML, but also performs what we call “code management” (smarter code-gen which doesn’t get in your way, or produce ugly code) which turns the UML into code for our ORM of choice.
All members of the team are required to use the tool and UML to create new domain entities which then automatically become (beautifully formatted) code.
The enormous advantage of this approach is that the documentation (in the form of UML in this case), never goes out of date and lives with the code, it’s literally checked in to our Git repo in an SCM friendly format next to the normal code.
We are always able to talk around these visual models when planning a new feature or strategising a technical solution for a requirement.
An architect can at any time reason about the domain without having to first build up a mental model of it from code.
When juniors ask questions of seniors, they can both talk around the UML so that they quickly get clarity on the best way forward.
I would never go back to building business software any other way.
If the code is generated from the diagrams, why check the code in at all? Feels akin to checking in the bytecode with your the java that generated it.
I can envision two reasons.
The first is that, in truth, the code generated from the UML, isn't good enough and people still actually work on the output.
The second guess is that humans occasionally rewrite the code for optimization. It might then be easier to source controll the code rather than patches.
The second guess is that humans occasionally rewrite the code for optimization. It might then be easier to source controll the code rather than patches.
Neither of those seem like compelling reasons, though. If the code isn't good enough, you are better incentivizing the team to improve the tool's generated code, than you are increasing the drift between what the tool generates and what is used.
Optimization can almost make sense, but maintaining a patch process is almost certainly the superior way to go. If only because it supports staying current on the generated code.
Optimization can almost make sense, but maintaining a patch process is almost certainly the superior way to go. If only because it supports staying current on the generated code.
There's also the reason of not wanting to generate all the code every single time someone new retrieves the repo or needs to revert back to a previous commit.
This reason doesn't make sense. Again, do you checkin the bytecode of your java? I'd expect you, instead, to recompile the java every time you do a build.
Now, I would expect you to have an artifact repository. To that end, you would be able to easily get the generated sources. But, they would decidedly not be in the source repository.
Now, I would expect you to have an artifact repository. To that end, you would be able to easily get the generated sources. But, they would decidedly not be in the source repository.
I don't feel the bytecode comparison makes sense, because the generated code is human readable, bytecode is (to almost all intents and purposes) not.
We feel that code generation is something one does explicitly while modelling, not something done every time you build.
We feel that code generation is something one does explicitly while modelling, not something done every time you build.
Completely our thinking too. Code generation should not be a build or run time dependency.
Our tool also allows developers to review proposed code changes before choosing to commit them to disk.
Code generation is something you choose to do explicitly as you model.
Our tool also allows developers to review proposed code changes before choosing to commit them to disk.
Code generation is something you choose to do explicitly as you model.
The main reason is that our "code management" (essentially code weaving) system allows coders to painlessly mix their own custom code in the same file as generated code.
This is very useful for cases where you want to model signatures of methods in our tool, but then leave it to the developer to implement the method which performs the business code. If you want to change the signature of the method, for example adding a parameter, then our code generation will add it in without messing up your custom code.
Our own templates generate code which is as beautiful looking and readable as code written by the best developers, and we encourage template writers to do the same. This makes working with the code in your IDE and reading it as much of a pleasure as any other well written code, having it in your SCM also gives you a history of how the code has changed over time.
Finally, use of the tool is optional, if you hand the the repository over to another team, or as a contracter you deliver it to a client, they don't need the tool, and just get code which looks as if it was written by a good developer.
This is very useful for cases where you want to model signatures of methods in our tool, but then leave it to the developer to implement the method which performs the business code. If you want to change the signature of the method, for example adding a parameter, then our code generation will add it in without messing up your custom code.
Our own templates generate code which is as beautiful looking and readable as code written by the best developers, and we encourage template writers to do the same. This makes working with the code in your IDE and reading it as much of a pleasure as any other well written code, having it in your SCM also gives you a history of how the code has changed over time.
Finally, use of the tool is optional, if you hand the the repository over to another team, or as a contracter you deliver it to a client, they don't need the tool, and just get code which looks as if it was written by a good developer.
This sounds like a large part of the reason is you deliver the code to others, but don't want to deliver the tool you used to make the code. This makes sense to me.
For the other cases you gave, that sounds like what Interfaces were made for. :) Granted, I think we are in a world where tooling not necessarily requiring language constructs is a better place.
Regardless, if you are seeing such success, I would be greatly interested in seeing you open up that tool.
For the other cases you gave, that sounds like what Interfaces were made for. :) Granted, I think we are in a world where tooling not necessarily requiring language constructs is a better place.
Regardless, if you are seeing such success, I would be greatly interested in seeing you open up that tool.
Here a recording of one of our guys on a webcast and it also shows the tool in action: https://www.youtube.com/watch?v=vA3L8_DaECQ
"No lock in" is an important aspect for us. We know of some other big players with "RAD" type tools which "generate" code, but unless one continues using their tool, the code they're left with is effectively unusable. Their business models are also often around runtime licensing.
As devs, building a tool for other devs, we want our tool to continue to be used because people clearly find the value in it, not because they feel locked in.
"No lock in" is an important aspect for us. We know of some other big players with "RAD" type tools which "generate" code, but unless one continues using their tool, the code they're left with is effectively unusable. Their business models are also often around runtime licensing.
As devs, building a tool for other devs, we want our tool to continue to be used because people clearly find the value in it, not because they feel locked in.
Speaking as someone who checks in other kinds of generated code. The tool, like any software, can have bugs/changes in behavior across versions. You don't want to introduce the chance of that happening without being audited in version control if you can easily avoid it.
The tool should be referenced as a versioned dependency at build time. When it gets a new release, you do a new build. Complete with new tests.
Anything you do that increases the time between the introduction of the bug and the finding of the bug will make it harder to actually fix it.
Anything you do that increases the time between the introduction of the bug and the finding of the bug will make it harder to actually fix it.
In addition to my reply above about code-weaving, rather than the all or nothing approach that some code generation techiniques suffer, our tool allows proposed changes to be reviewed before you commit with writing the generated code to your disk.
This allows developers to feel in control of what's changing in their code base during generation, rather than being worried that some custom code is going to get wiped out by code-gen without them realizing.
In a past project I was on, we had our solution build process run T4 templates to generate code, and it significantly slowed down the build process.
We feel code generation is something you explicitly do when you're "modelling", it's not something that's necessary to create a build nor should it create run-time dependency.
This allows developers to feel in control of what's changing in their code base during generation, rather than being worried that some custom code is going to get wiped out by code-gen without them realizing.
In a past project I was on, we had our solution build process run T4 templates to generate code, and it significantly slowed down the build process.
We feel code generation is something you explicitly do when you're "modelling", it's not something that's necessary to create a build nor should it create run-time dependency.
That attitude is ridiculously commendable, major kudos.
My fear is typically around having "mixed edited" code. That is, code that is edited by hand and generated. I greatly prefer being able to know that the generated stuff is extended by language mechanisms and I will never really have even the ability to edit something that the generator will do. I have seen bugs in generated code, but I have always pushed to fix the generator, not just fixed the generated code. (That make sense?)
And, reiterating because I feel it is important, your position is awesome. Thanks for sharing!
My fear is typically around having "mixed edited" code. That is, code that is edited by hand and generated. I greatly prefer being able to know that the generated stuff is extended by language mechanisms and I will never really have even the ability to edit something that the generator will do. I have seen bugs in generated code, but I have always pushed to fix the generator, not just fixed the generated code. (That make sense?)
And, reiterating because I feel it is important, your position is awesome. Thanks for sharing!
What product is it?
Intent Architect[1].
Our main development focus right now is on usability for new comers and we're improving it regularly.
[1] http://intentarchitect.com/
Our main development focus right now is on usability for new comers and we're improving it regularly.
[1] http://intentarchitect.com/
re: full disclosure,
I appreciate your reply and that it didn't end with "So, you should use program X (the one you work on) because it's the best software". Your objective response is great.
I appreciate your reply and that it didn't end with "So, you should use program X (the one you work on) because it's the best software". Your objective response is great.
Thank you, regardless of my interests in promoting the idea of using such a tool, we made it because we have found time and time again that always upto date visual documentation such as UML is invaluable, especially as projects get older and bigger.
Otherwise, in general we're far more interested in constructive criticism (or even just normal criticism) of our philosophy (and tool) from a place like HN, rather than trying to score a little free "advertising". When the time is right, we'll do proper promotion, and it won't be by spamming forums.
Otherwise, in general we're far more interested in constructive criticism (or even just normal criticism) of our philosophy (and tool) from a place like HN, rather than trying to score a little free "advertising". When the time is right, we'll do proper promotion, and it won't be by spamming forums.
What is the tool?
Intent Architect[1].
Our main development focus right now is on usability for new comers and we're improving it regularly.
[1] http://intentarchitect.com/
Our main development focus right now is on usability for new comers and we're improving it regularly.
[1] http://intentarchitect.com/
>where is the modern 'IDE' for the Software Architect
It's the same one the software developers use, 'architects' don't stop coding, the system architecture doesn't exist in Visio or PP. It's evident from the structure of the codebase and the accompanying documentation, your role is to collaborate and work with the senior devs to ensure this design vision is realised and to explain in documentation why this architecture solves the business requirement it pertains to.
It's not something you wash your hands of and walk away from, having handed it over to the dev team. You're in it for the long haul, same as them.
It's the same one the software developers use, 'architects' don't stop coding, the system architecture doesn't exist in Visio or PP. It's evident from the structure of the codebase and the accompanying documentation, your role is to collaborate and work with the senior devs to ensure this design vision is realised and to explain in documentation why this architecture solves the business requirement it pertains to.
It's not something you wash your hands of and walk away from, having handed it over to the dev team. You're in it for the long haul, same as them.
It's not something you wash your hands of and walk away from, having handed it over to the dev team
Unfortunately that is the reality of the vast majority of Enterprise Architects I have ever worked with, writing massive tomes completely disconnected from the actual systems. The could all be fired tomorrow and no one would even notice (in fact one company I worked for did exactly that after too many complaints about them from the devs! And since they sat in their own cordoned off section, really no one did notice until the meeting invites suddenly stopped!).
The only architecture that matters is a living document; for example the CloudFormation or Terraform that actually generates it for real.
Unfortunately that is the reality of the vast majority of Enterprise Architects I have ever worked with, writing massive tomes completely disconnected from the actual systems. The could all be fired tomorrow and no one would even notice (in fact one company I worked for did exactly that after too many complaints about them from the devs! And since they sat in their own cordoned off section, really no one did notice until the meeting invites suddenly stopped!).
The only architecture that matters is a living document; for example the CloudFormation or Terraform that actually generates it for real.
The Ballerina programming language (ballerina.io) has a syntax designed in such a way that any syntactically correct program can have its sequence diagram generated automatically. It's a type of self-documenting programming language. It only documents for the scope of the service that is being programmed, but eventually if many components and services are programmed with Ballerina, system wide architecture diagrams can also be constructed.
This is interesting. I ctrl-f'd for "self-doc" because I was wondering when we will start seeing Terraform, Puppet, etc., have options to generate a diagram automatically as part of their normal operation. Self-documenting infrastructure would be a godsend. My organization is challenged by the lack of documentation of our infrastructure, especially.
edit: Most of the responses to the original post seem to be addressing the idea of understanding code through diagramming. I am _much_ more interested in understanding complex infrastructure through diagrams. What queues do we have, what are the producers, what are the consumers? Where is the database? What writes to it? What do we shard on?
Code's a lot slipperier to autodoc, and the slipperiness is a function of number of contributors.
Infrastructure, however, we have been herding (pun intended) to a more declarative, predictable paradigm for years which is great. So let's start building the auto-documentation stuff into e.g. Terraform, since infrastructure-as-code seems like the natural place for such useful side effects of provisioning assets.
edit: Most of the responses to the original post seem to be addressing the idea of understanding code through diagramming. I am _much_ more interested in understanding complex infrastructure through diagrams. What queues do we have, what are the producers, what are the consumers? Where is the database? What writes to it? What do we shard on?
Code's a lot slipperier to autodoc, and the slipperiness is a function of number of contributors.
Infrastructure, however, we have been herding (pun intended) to a more declarative, predictable paradigm for years which is great. So let's start building the auto-documentation stuff into e.g. Terraform, since infrastructure-as-code seems like the natural place for such useful side effects of provisioning assets.
It's already there with Terraform but it's not that pretty yet:
https://www.terraform.io/docs/commands/graph.html
https://www.terraform.io/docs/commands/graph.html
More evidence that graphviz needs a successor or makeover. Some force-based post-processing with the additional goal of rectangular borders would be good. Also some tweaks like little arcs where lines cross and shadows for boxes.
I wonder if this needs its own language. It looks like c# from a quick glance. I wonder if static analysis on an existing language would be just as good. In any case I love this idea.
If the project is very large architects don't have time to code. Sometimes they can test something before taking decisions but all the production code is written by somebody else. Their role is more about meeting people and help the organization making the correct decisions.
I am always wary of architects that don't code or at least aren't involved in the whole life time of the project so they have to feel the consequences of their designs. Our IT department is full of "architects" whose designs then get outsourced to India. Either their designs are crap that doesn't survive the first contact with reality or they are so trivial that pretty much anybody could do them. They spend a lot of time in meetings with important people though.
I find it much better to empower the senior devs of different teams to duke it out among them. Maybe the process is not as clean as having some genius architect divine the perfect architecture but it reflects reality much better.
I find it much better to empower the senior devs of different teams to duke it out among them. Maybe the process is not as clean as having some genius architect divine the perfect architecture but it reflects reality much better.
I think this depends... I work at a Fortune 100 company, and I know architects that still code, despite being involved with large projects...
Currently my Favorite tool to Diagram Sofware Architectures is the C4-Model [1]. For the static view
and use a normal Sequence Diagram for the Dynamic View.
In both cases I like to use Plant UML, so that the text get's converted to the diagram[2]. That way the diagram can be stored together with the code.
[1] https://www.infoq.com/articles/C4-architecture-model [2] http://plantuml.com/sequence-diagram
[1] https://www.infoq.com/articles/C4-architecture-model [2] http://plantuml.com/sequence-diagram
C4 is my goto as well.
I’m hearing two things:
1. You want to build these diagrams faster and perhaps semi-automatically instead of dragging around boxes and lines.
2. You want them to look better and not so perfunctory. To communicate better.
There is kind of an oddball option here which I fell in love with. There’s a tool called Monodraw which is magic for making old school ASCII art diagrams. It’s surprisingly very visually compelling, easy to update and it’s just text so it can be embedddd in source code comments (no PDFs).
The best reason you might consider this over a “big” IDE-type solution is that ultimately good cogent communication requires making choices about what to highlight. An exhaustive auto generated architecture diagram in the same generic style of everything else can make one’s eyes glaze over. When you choose what to show and emphasize you can tell a better “story”.
Yes you have to keep the diagram up to date with major changes, but that’s not a problem if you lower the cost of updating and raise the visibility of the diagram by putting it in code. They even have a CLI now so you can export it in your build process.
1. You want to build these diagrams faster and perhaps semi-automatically instead of dragging around boxes and lines.
2. You want them to look better and not so perfunctory. To communicate better.
There is kind of an oddball option here which I fell in love with. There’s a tool called Monodraw which is magic for making old school ASCII art diagrams. It’s surprisingly very visually compelling, easy to update and it’s just text so it can be embedddd in source code comments (no PDFs).
The best reason you might consider this over a “big” IDE-type solution is that ultimately good cogent communication requires making choices about what to highlight. An exhaustive auto generated architecture diagram in the same generic style of everything else can make one’s eyes glaze over. When you choose what to show and emphasize you can tell a better “story”.
Yes you have to keep the diagram up to date with major changes, but that’s not a problem if you lower the cost of updating and raise the visibility of the diagram by putting it in code. They even have a CLI now so you can export it in your build process.
Try PlantUML. For system architecture, you can check this PlantUML extension https://github.com/RicardoNiepel/C4-PlantUML
Plant UML is good. If you find it a bit constricting at times, I believe you can drop DOT graphs into the PlantUML diagrams, too. DOT is the language used by graphviz. You can of course use graphviz directly, too!
> where is the modern 'IDE' for the Software Architect?
Generally architecture is about imposing constraints on the system. If you think about it that way, what you really want is not to be treating Visio and PowerPoint documents as the primary artifacts. You want to define the modules (in Parnas's sense) and their interfaces and executable specifications. You want tooling that lets you constrain what modules may use other modules, automatically test modules, automatically mock modules, and be alerted when the constraints are broken.
Now, most of the tooling for this probably doesn't exist, or exists in partial, widely scattered forms. For example, you can define a Java package with the interfaces of a module and then have some kind of hook in the build system that checks that it is not being imported in places it shouldn't be. You can mark the interfaces that should be considered architectural constraints with a custom annotation and build a tool to extract out all uses of them and plot connections.
But I am unaware of any toolkit that will do what you want.
Generally architecture is about imposing constraints on the system. If you think about it that way, what you really want is not to be treating Visio and PowerPoint documents as the primary artifacts. You want to define the modules (in Parnas's sense) and their interfaces and executable specifications. You want tooling that lets you constrain what modules may use other modules, automatically test modules, automatically mock modules, and be alerted when the constraints are broken.
Now, most of the tooling for this probably doesn't exist, or exists in partial, widely scattered forms. For example, you can define a Java package with the interfaces of a module and then have some kind of hook in the build system that checks that it is not being imported in places it shouldn't be. You can mark the interfaces that should be considered architectural constraints with a custom annotation and build a tool to extract out all uses of them and plot connections.
But I am unaware of any toolkit that will do what you want.
Know that the two dimensions of a diagram are insufficient to express the confluence of logical data flow, physical data flow, dependencies, the infrastructure stack, the users and their use cases, commercial arrangements, legal entity boundaries, concurrent change initiatives, and boxes for the systems themselves and their fractal decomposition of modules and submodules.
Basically, pick your battles and just draw what helps communicate something.
Basically, pick your battles and just draw what helps communicate something.
After the System Design Primer by Donne Martin got quite popular on Hacker News, I contacted Donne to find out which tools he used to draw those system architecture diagram and he was kind to respond my twitter message ( that is why twitter is so great ) to tell me it it is Omnigraffle ( only available in Mac and ios. )
10 years back I used to draw UML diagrams using Visio or one of the eclipse plugins for building systems diagram. However, the software landscape has changed quite a bit. On one of the podcasts of Martin Fowler recently, he also agreed that UML is now less favored standard in the software community.
Although I use Windows on my work but for the demo of the product I was working I built the system architecture diagram using my personal mac book and transfer to my work laptop to made a presentation. There was a good feedback and no-one raised any concern with UML or (any-)standards.
So, tool or IDE is less important these days, as we tend to use multiple tools/IDE based on the technology / framework / programming languages. I use intelliJ for Java, Sublime for Ruby / Rails + Angular + Others, Visual Studio for NodeJS, Vim for scripting, etc. etc....
10 years back I used to draw UML diagrams using Visio or one of the eclipse plugins for building systems diagram. However, the software landscape has changed quite a bit. On one of the podcasts of Martin Fowler recently, he also agreed that UML is now less favored standard in the software community.
Although I use Windows on my work but for the demo of the product I was working I built the system architecture diagram using my personal mac book and transfer to my work laptop to made a presentation. There was a good feedback and no-one raised any concern with UML or (any-)standards.
So, tool or IDE is less important these days, as we tend to use multiple tools/IDE based on the technology / framework / programming languages. I use intelliJ for Java, Sublime for Ruby / Rails + Angular + Others, Visual Studio for NodeJS, Vim for scripting, etc. etc....
I think the best tool is still the old fashioned well written technical reports.
This book has many examples: http://aosabook.org
This book has many examples: http://aosabook.org
It's not a tool (though there is the Structurizr tool), I like the c4model: http://c4model.com
C4 provides an approach with multiple layers, which is good for talking about architecture at different levels with different people. I also happen to like the Container diagram approach.
C4 provides an approach with multiple layers, which is good for talking about architecture at different levels with different people. I also happen to like the Container diagram approach.
Gaphor (https://github.com/gaphor/gaphor) is an open source UML tool written in Python. The goal is to create a simple, easy to use modeling tool not a big enterprise tool like the ones already mentioned. I would like to soon support SysML which is for modeling systems design and requirements. We are finishing up converting it to Python3 and Gtk+3 now. We would love more involvement or input if there are things you would like to see supported.
Side remark: A few screenshots and/or video would be a great intro to get a feel of the program
You can use TLA+ -- it's what Amazon and Microsoft uses.
TLA+ is a formal specification language, plain-old maths, and the tool suite includes a model checker and a pretty printer. The model checker verifies properties of your specification that you want to hold true such as liveness. And it also includes a pretty-printer for exporting your specifications into well-formatted PDF documentation.
The nice thing about writing specifications this way is that they are specific, at the correct level of abstraction for the problem, and can be verified.
I think Lamport's analogy is apt: specifications are like blueprints. If you're building a shed in your backyard then a sketch will be sufficient. There's no need to have a blueprint. If you're building a house you'll need some kind of blueprint. The software equivalent of a sufficient blueprint for building a house is a strong, static type system, unit, integration, and property-based tests. If you're building skyscrapers you need something much more formal and I think this is where TLA+ and Lean are useful.
TLA+ is a formal specification language, plain-old maths, and the tool suite includes a model checker and a pretty printer. The model checker verifies properties of your specification that you want to hold true such as liveness. And it also includes a pretty-printer for exporting your specifications into well-formatted PDF documentation.
The nice thing about writing specifications this way is that they are specific, at the correct level of abstraction for the problem, and can be verified.
I think Lamport's analogy is apt: specifications are like blueprints. If you're building a shed in your backyard then a sketch will be sufficient. There's no need to have a blueprint. If you're building a house you'll need some kind of blueprint. The software equivalent of a sufficient blueprint for building a house is a strong, static type system, unit, integration, and property-based tests. If you're building skyscrapers you need something much more formal and I think this is where TLA+ and Lean are useful.
> it's what Amazon and Microsoft uses
That's not true. TLA+ is very useful but only few services adopt and benefit from it. Also it's not very readable and cannot document many design aspects e.g. the reasons behind technical decisions.
That's not true. TLA+ is very useful but only few services adopt and benefit from it. Also it's not very readable and cannot document many design aspects e.g. the reasons behind technical decisions.
> TLA+ is very useful but only few services adopt and benefit from it.
It's true that formal specifications are rarely used, but I would bet that the ratio of those that benefit significantly from them to those that use them is far greater than many other, far vaguer tools.
> Also it's not very readable
That depends on what you mean by "readable." A prose description can appear readable in the sense that the reader may think they understand what the document says, but sometimes that's just because the text is vague enough to allow for conflicting interpretations. TLA+, on the other hand, is precise. True, it takes some learning, but it's easier to learn than a programming language, as it's much, much smaller (the reference documentation for the entire language and all of the standard library fits comfortably on 7 pages (https://lamport.azurewebsites.net/tla/summary.pdf)
> and cannot document many design aspects e.g. the reasons behind technical decisions
It can document some decisions very well. For one, you can state precisely your assumptions as well as the requirement. You then describe the desired operation of the system. You can then check that the design fits the requirements given the assumptions, and show that other designs don't. It is true that it's not intended to model the reasoning behind every decision, such as cost/time of implementation, but generic (i.e., non-software-specific) project management tools can help you with that.
Now, I am not saying that a formal specification is always required, and I agree that writing down an informal one is far better than not writing down anything at all, but it can really save a lot of time and trouble in subtle/complex/unclear cases.
It's true that formal specifications are rarely used, but I would bet that the ratio of those that benefit significantly from them to those that use them is far greater than many other, far vaguer tools.
> Also it's not very readable
That depends on what you mean by "readable." A prose description can appear readable in the sense that the reader may think they understand what the document says, but sometimes that's just because the text is vague enough to allow for conflicting interpretations. TLA+, on the other hand, is precise. True, it takes some learning, but it's easier to learn than a programming language, as it's much, much smaller (the reference documentation for the entire language and all of the standard library fits comfortably on 7 pages (https://lamport.azurewebsites.net/tla/summary.pdf)
> and cannot document many design aspects e.g. the reasons behind technical decisions
It can document some decisions very well. For one, you can state precisely your assumptions as well as the requirement. You then describe the desired operation of the system. You can then check that the design fits the requirements given the assumptions, and show that other designs don't. It is true that it's not intended to model the reasoning behind every decision, such as cost/time of implementation, but generic (i.e., non-software-specific) project management tools can help you with that.
Now, I am not saying that a formal specification is always required, and I agree that writing down an informal one is far better than not writing down anything at all, but it can really save a lot of time and trouble in subtle/complex/unclear cases.
Ok, some teams at Amazon: https://lamport.azurewebsites.net/tla/formal-methods-amazon....
You can write prose into your specifications and integrate the outputted PDF specifications with the rest of your documentation.
Also it's not very readable and cannot document many design aspects e.g. the reasons behind technical decisions.
You can write prose into your specifications and integrate the outputted PDF specifications with the rest of your documentation.
Have a look at the Arc42 template [0] which is a generic outline for software architectures. TLA does not make sense for most of the items there.
[0] https://arc42.org/overview/
[0] https://arc42.org/overview/
It makes a lot of sense for 1, 2, 3, 6, 7 and 12.
> Not very readable? How so? I'd rather read a concise mathematical definition rather than three pages of prose and diagrams. It is most definitely readable although it does require some training to understand the mathematics if you're not used to reading it. Just as reading a blueprint requires a bit of training.
You glibly toss off "a bit of training" as if it's an afternoon's work over a cup of coffee. Understanding, intuitively, the systems that mathematical models like TLA describe is _extraordinarily_ difficult. Reading that 1-page model description for comprehension could be the work of days. Reading a 10-page prose and illustration description of the same system is likely to be the work of 10 minutes, and result in a much more thorough practical understanding of the system.
You glibly toss off "a bit of training" as if it's an afternoon's work over a cup of coffee. Understanding, intuitively, the systems that mathematical models like TLA describe is _extraordinarily_ difficult. Reading that 1-page model description for comprehension could be the work of days. Reading a 10-page prose and illustration description of the same system is likely to be the work of 10 minutes, and result in a much more thorough practical understanding of the system.
> Understanding, intuitively, the systems that mathematical models like TLA describe is _extraordinarily_ difficult.
It is the opposite of intuitively extraordinarily difficult. It is much easier and more intuitive than understanding code. It is, however, different from code, so your coding skill do not automatically transfer to TLA+, but developers are generally able not only to read but to write TLA+ specifications of complex systems after a 3-day workshop or about 2 weeks of part-time self-study. Learning TLA+ is far easier than learning a new programming language, and it is much simpler than any programming language in existence. The difficulty is not at all with intuition, but with unfamiliarity. In any event, reading TLA+ is much, much, much easier than writing/understanding the systems for which you use TLA+ for.
It is the opposite of intuitively extraordinarily difficult. It is much easier and more intuitive than understanding code. It is, however, different from code, so your coding skill do not automatically transfer to TLA+, but developers are generally able not only to read but to write TLA+ specifications of complex systems after a 3-day workshop or about 2 weeks of part-time self-study. Learning TLA+ is far easier than learning a new programming language, and it is much simpler than any programming language in existence. The difficulty is not at all with intuition, but with unfamiliarity. In any event, reading TLA+ is much, much, much easier than writing/understanding the systems for which you use TLA+ for.
For some of the Amazon engineers mentioned in the paper that training took 1 to 2 weeks.
The trade off for that training is that you know the those properties it describes are correct. For some systems the trade off is worth it and for a few, required.
I didn’t say you should use TLA+ for simple projects. It is incredibly useful for specifying systems where correctness, liveness, etc matter greatly and the complexity of the project is sufficiently high that you’d be uncomfortable describing it with boxes and arrows.
I think it’s rather reckless to design a complex system of the sky-scraper magnitude without some sort of verification tool like TLA+.
It’s a matter of degrees.
The trade off for that training is that you know the those properties it describes are correct. For some systems the trade off is worth it and for a few, required.
I didn’t say you should use TLA+ for simple projects. It is incredibly useful for specifying systems where correctness, liveness, etc matter greatly and the complexity of the project is sufficiently high that you’d be uncomfortable describing it with boxes and arrows.
I think it’s rather reckless to design a complex system of the sky-scraper magnitude without some sort of verification tool like TLA+.
It’s a matter of degrees.
> For some of the Amazon engineers mentioned in the paper that training took 1 to 2 weeks.
That's obviously a lie.
That's obviously a lie.
I like Arc42. It is aimed at documenting technical architectural decisions, the whys and constraints (budgetary et c) that lead to a particular design.
It also outlines the main boundaries, the key interactions.
Architecture should not be about the details. Rather it should be like a guide so that the developer can look at the source and see for themselves.
It also outlines the main boundaries, the key interactions.
Architecture should not be about the details. Rather it should be like a guide so that the developer can look at the source and see for themselves.
GraphViz. Store the graphviz code in repo to track changes.
Pencil & Paper, take a picture scan it in. Then type up descriptions.
This is why I agree with other commenters who say that there's no good way to do this: with pencil and paper, you can't make changes / expand / remove parts so easily, it's harder to make big documents, it doesn't look that good, and yet... it still is one of the best ways to do it.
P.S. I don't have anything against paper, and in fact I think it should be used more frequently while designing / prototyping / trying to understand or come up with algorithms, etc, it's a very useful tool.
P.S. I don't have anything against paper, and in fact I think it should be used more frequently while designing / prototyping / trying to understand or come up with algorithms, etc, it's a very useful tool.
Is there any software that translates a picture or scanned doc into a powerpoint? Seems like that could be useful: quickly churn out lots of boxes/arrows/text by hand, scan them, modify as necessary when translated.
Great question. I too like diagraming and Im going to check out some of these tools.
Do you have any examples of your work? I can publish some of mine later today (have to remove specific details) if interested. I always love comparing and learning from different diagram styles.
I have found that spending a bit of time building some proficiency in Google Slides was well worth it. You have to do it without a real presentation in mind though, as it distracts from learning g the tool. Another big time saver came when I created a template presentation with colors, line weights, fonts and font sizes that were to my liking. The Polish, if you will, that we often don't want to go back and touch once the technicals are on the page.
Do you have any examples of your work? I can publish some of mine later today (have to remove specific details) if interested. I always love comparing and learning from different diagram styles.
I have found that spending a bit of time building some proficiency in Google Slides was well worth it. You have to do it without a real presentation in mind though, as it distracts from learning g the tool. Another big time saver came when I created a template presentation with colors, line weights, fonts and font sizes that were to my liking. The Polish, if you will, that we often don't want to go back and touch once the technicals are on the page.
This is a loaded question. Is there a better way? Maybe. It is almost certainly not any easier, though. Likely takes longer and requires as much investment from the reviewers as it does from the authors.
I confess I'm fond of Literate Programming for this, though, I have never tried introducing it in a company. My hunch is it would take too long. By far.
Even documented software often takes too long. Same reason you don't plot out how you are going to win a ballgame. Outside of "score more points than you allow", the plans are almost all exploratory and reactive. Solidifying that into a document is dangerously slow.
I confess I'm fond of Literate Programming for this, though, I have never tried introducing it in a company. My hunch is it would take too long. By far.
Even documented software often takes too long. Same reason you don't plot out how you are going to win a ballgame. Outside of "score more points than you allow", the plans are almost all exploratory and reactive. Solidifying that into a document is dangerously slow.
I've long dremead of a architecture view that was zoomable, zooming in enough would end up on the actual code, but zooming out a lot would show you app server +database basically.
This would be incredible. No reason zooming out can't also show you stuff like load-balancers, message queues, firewalls, and their config/infrastructure code.
I too have had this dream. It's like a Realtime stragedy game, you have a minimap where you make big movements around the map, zooming in to perform some micromanagement on individual units. Zooming in further to directly edit there AI.
If you're on AWS, consider CloudCraft. https://cloudcraft.co/
Disclaimer: not affiliated with and cannot vouch for them. I just think they look cool.
Disclaimer: not affiliated with and cannot vouch for them. I just think they look cool.
I use Cloudcraft to express the architecture of a number of my smaller projects. It's very easy to throw a few key parts together and get a basic idea out there. Exports in lots of resolutions, to SVG, and has shareable links.
Looks interesting. Anyone used it before and can share the experience?
It seems to fill a gap nicely. When there are more than a few apps up in AWS it’s easy to lose track.
It seems to fill a gap nicely. When there are more than a few apps up in AWS it’s easy to lose track.
If you use infrastructure-as-code like Terraform or Cloudformation you will find it harder to lose track, because the design, and hopefully some comments, are checked into source control.
Cloudcraft is good too, but not the whole answer.
Cloudcraft is good too, but not the whole answer.
Suggestion: There is none.
Start writing a blog in Git using Markdown and some basic images (export diagram as PNG). Use HTML where necessary, for instance, tables.
Use dotty/sphinx/doxygen for autogenerated stuff. If your architecture changes too often, you are doing something terribly wrong.
Do not worry about extraneous things such as mobile form factor, jazzy visuals, image DPI or things like that. Instead, focus on making the text more readable or the diagrams easier to understand.
If you want to not be disconnected, write/use a relatively simple script to export a document as PDF (RelaxedJS or Prince).
Start writing a blog in Git using Markdown and some basic images (export diagram as PNG). Use HTML where necessary, for instance, tables.
Use dotty/sphinx/doxygen for autogenerated stuff. If your architecture changes too often, you are doing something terribly wrong.
Do not worry about extraneous things such as mobile form factor, jazzy visuals, image DPI or things like that. Instead, focus on making the text more readable or the diagrams easier to understand.
If you want to not be disconnected, write/use a relatively simple script to export a document as PDF (RelaxedJS or Prince).
> "I still sit for hours in Visio & Powerpoint to painstakingly drag boxes and lines around to describe systems."
I much prefer to skip the painstaking messing with Visio and instead use tools like https://www.websequencediagrams.com
Simple to update and no faffing around with formatting.
I normally then do a screen/audio recording whilst talking about the diagram. These have received very positive feedback from developers and non-developers.
I much prefer to skip the painstaking messing with Visio and instead use tools like https://www.websequencediagrams.com
Simple to update and no faffing around with formatting.
I normally then do a screen/audio recording whilst talking about the diagram. These have received very positive feedback from developers and non-developers.
I've recently moved all my (new) diagrams to plantuml (my perso al workflow is actually orgmode + plantuml generation using babel).
I also asked my team if they are making new diagrams they should be made with plantuml.
The biggest benefit in my mind is clarity of design and most important of all: diagram as code, you can diff what's new in the latest version and everyone can comment/make pull requests to update the diagram if needed.
plantuml: http://plantuml.com
I also asked my team if they are making new diagrams they should be made with plantuml.
The biggest benefit in my mind is clarity of design and most important of all: diagram as code, you can diff what's new in the latest version and everyone can comment/make pull requests to update the diagram if needed.
plantuml: http://plantuml.com
I developed an interactive visualization just for that purpose a few years ago. It's called ArchitectureTree.
https://github.com/marmelab/ArchitectureTree
The tech stack is a bit outdated (it used Angular.js), but the idea (using JSON to store the architecture) and the UI (powerful search, "bush" look) are still a good answer to the requirement of vizualizing complex software architecture.
https://github.com/marmelab/ArchitectureTree
The tech stack is a bit outdated (it used Angular.js), but the idea (using JSON to store the architecture) and the UI (powerful search, "bush" look) are still a good answer to the requirement of vizualizing complex software architecture.
Take a look at cargo's contribution and architecture docs. [1] That project was hands down one of the easiest for me to get into and learn. I am a firm believer in a nice architecture document that points to interesting code sections. To date, I have never encountered a UML in the wild and been satisfied.
[1]: https://github.com/rust-lang/cargo/#contributing
[1]: https://github.com/rust-lang/cargo/#contributing
Wikipedia has page on Architecture Description Languages - https://en.wikipedia.org/wiki/Architecture_description_langu...
I've looked at Wright and Acme that are linked from that page. I came away with the conclusion that you need to use a formal language to describe relationships between the entities in your system. The language should be flexible enough to define new entities, and new kinds of relationships. All of that points to the need for an API/library to describe software architectures, probably using parameterized types in Scala/Java or relying on features in more dynamic languages like Ruby. The language need not be Turing complete, though.
An added benefit of using a formal language is that you can verify whether the invariants of the system are maintained when you make changes. For example: If you try to connect two components which support different protocols, the program won't validate.
These ideas may also have overlap with software specification languages - https://en.wikipedia.org/wiki/Specification_language . I didn't dig too much into that aspect.
Generating diagrams with suitable layouts from this description is a separate problem. Perhaps something from the field of graph drawing could help? https://en.wikipedia.org/wiki/Graph_drawing
I've looked at Wright and Acme that are linked from that page. I came away with the conclusion that you need to use a formal language to describe relationships between the entities in your system. The language should be flexible enough to define new entities, and new kinds of relationships. All of that points to the need for an API/library to describe software architectures, probably using parameterized types in Scala/Java or relying on features in more dynamic languages like Ruby. The language need not be Turing complete, though.
An added benefit of using a formal language is that you can verify whether the invariants of the system are maintained when you make changes. For example: If you try to connect two components which support different protocols, the program won't validate.
These ideas may also have overlap with software specification languages - https://en.wikipedia.org/wiki/Specification_language . I didn't dig too much into that aspect.
Generating diagrams with suitable layouts from this description is a separate problem. Perhaps something from the field of graph drawing could help? https://en.wikipedia.org/wiki/Graph_drawing
We plan to try Structure101 [0] next year to verify invariants. I would love to see comments from people who have experience with it.
[0] https://structure101.com/
[0] https://structure101.com/
I like to link our source code to enterprise architect so the sources you pull in a view are automatically populated based upon the source code. The eap can be checked in with the source to have an additional trigger to keep it updated. Also EA. can work with relative source code paths so others can use the same file. There is also a versioning system build in which works quite well. Some things I still dislike:
- Enterprise Architect is a great tool but produces diagrams that look dated. This means I sometimes copy in another tool for certain types of meetings that require fancy powerpoints (pitch like presentations, customers, ex. Management etc.
- if the source isn't there I create from scratch. I have to link post-source creation to update and link my diagrams to source code. I do not generate source code from enterprise architect because we do not work model based and usually my diagrams that I create in advance are not detailed enough to generate the source. (Nor should they be)
- Sometimes the right diagram type is just not there in EA. I created my own templates for our department with all AWS logos and services, Azure, docker, and many others. That's a pain..
* I work in a global enterprise. Size of source about 6 million LOC. EA does handle that without hickups.
- Enterprise Architect is a great tool but produces diagrams that look dated. This means I sometimes copy in another tool for certain types of meetings that require fancy powerpoints (pitch like presentations, customers, ex. Management etc.
- if the source isn't there I create from scratch. I have to link post-source creation to update and link my diagrams to source code. I do not generate source code from enterprise architect because we do not work model based and usually my diagrams that I create in advance are not detailed enough to generate the source. (Nor should they be)
- Sometimes the right diagram type is just not there in EA. I created my own templates for our department with all AWS logos and services, Azure, docker, and many others. That's a pain..
* I work in a global enterprise. Size of source about 6 million LOC. EA does handle that without hickups.
"Document" is a bit of a troublesome word. It implies that you can communicate out into the void, and whoever reads it will understand a series of perhaps complex choices.
You probably want a model, not a picture. Visio is useful for making pictures -- diagrams. You want something that is a purposefully-simplified version of a much more complex system for purposes of communication/decision-making. Something that can be manipulated. Something to help facilitate conversations around certain narrow issues in a complicated system. That's a model.
There is a great post right now on HN about MVC. In it, there are some UML models. Check it out. They're not documentation of a system. They're isolated "cut-outs" to facilitate discussion around one area.
There are a bunch of great modeling tools. Many are being mentioned here. Go check them out!
The big thing to understand is the difference between a diagram and a model -- and making deliberate decisions both about what to include and exclude in each thing you share.
BTW, you can also use physical materials to model. The secret to good modeling is the simplification-mapping, not necessarily the model substrate being used.
You probably want a model, not a picture. Visio is useful for making pictures -- diagrams. You want something that is a purposefully-simplified version of a much more complex system for purposes of communication/decision-making. Something that can be manipulated. Something to help facilitate conversations around certain narrow issues in a complicated system. That's a model.
There is a great post right now on HN about MVC. In it, there are some UML models. Check it out. They're not documentation of a system. They're isolated "cut-outs" to facilitate discussion around one area.
There are a bunch of great modeling tools. Many are being mentioned here. Go check them out!
The big thing to understand is the difference between a diagram and a model -- and making deliberate decisions both about what to include and exclude in each thing you share.
BTW, you can also use physical materials to model. The secret to good modeling is the simplification-mapping, not necessarily the model substrate being used.
I like to use DrawExpress when I'm on the go, have an idea, and a phone or tablet.
https://play.google.com/store/apps/details?id=com.drawexpres...
Gestures and drawing shapes are where the big speed improvements come from. Typing is eh since it's a touch screen but a keyboard fixes that.
https://play.google.com/store/apps/details?id=com.drawexpres...
Gestures and drawing shapes are where the big speed improvements come from. Typing is eh since it's a touch screen but a keyboard fixes that.
It does not exist. This isn't just for the software architect, but systems architects as a class don't have a generalized IDE to build diagrams (static or time-series) that allow them to think, demonstrate or iterate on systems in any meaningful way.
My workflow right now is hand-drawing diagrams in Paper and pulling them into a Notion page. Notion is amazing, the Paper workflow is not.
My workflow right now is hand-drawing diagrams in Paper and pulling them into a Notion page. Notion is amazing, the Paper workflow is not.
I used to “draw” ER/Flow/Activity/Sequence diagrams through various DSLs that would output PNG files with Graphviz but for quite a while now I like to marry “hand-drawn” ASCII diagrams with a README-driven-design-and-development process.
Embracing the constraints of a simple README for particular sub-systems lets me also put together general systems overview documents very easily after.
That said pencil & paper / whiteboard almost always comes first of course.
http://tom.preston-werner.com/2010/08/23/readme-driven-devel...
https://monodraw.helftone.com
http://asciiflow.com
Good non-ASCII alternative (for hyperlinked sharing in wikis etc):
https://www.lucidchart.com
Embracing the constraints of a simple README for particular sub-systems lets me also put together general systems overview documents very easily after.
That said pencil & paper / whiteboard almost always comes first of course.
http://tom.preston-werner.com/2010/08/23/readme-driven-devel...
https://monodraw.helftone.com
http://asciiflow.com
Good non-ASCII alternative (for hyperlinked sharing in wikis etc):
https://www.lucidchart.com
I've never been a system architect. But this reminds me of the principles of comments.
Obviously, one principle is "use as few comments as possible but no fewer".
But another principle is "don't spend extra-time formatting or otherwise prettifying a comment". Not only is that time not spent on development but it makes the comment seem more credible than it deserves and it can make other programmer looking at the comment hesitate to change it when they otherwise should.
Like comments, architecture draws may be fated to be often out-of-date. Like comments, that doesn't mean they're useless. Like comment architecture draws probably actually benefit from not looking more integrated into code than they actually are. They're a starting point, you have to read the code still but a starting point can be very important.
Perhaps your only real problem is finding a draw program you can use faster.
Obviously, one principle is "use as few comments as possible but no fewer".
But another principle is "don't spend extra-time formatting or otherwise prettifying a comment". Not only is that time not spent on development but it makes the comment seem more credible than it deserves and it can make other programmer looking at the comment hesitate to change it when they otherwise should.
Like comments, architecture draws may be fated to be often out-of-date. Like comments, that doesn't mean they're useless. Like comment architecture draws probably actually benefit from not looking more integrated into code than they actually are. They're a starting point, you have to read the code still but a starting point can be very important.
Perhaps your only real problem is finding a draw program you can use faster.
I've found that writing pseudocode with doxygen formatted comments to generate sharable diagrams is generally sufficient.
I actually got into that habit back when I was working with the insanely expensive suite of Rational IDEs. At that point, and later with the Eclipse Modeling Framework, the goal was to generate code from diagrams.... but that took more work than it was worth in my opinion.
With those tools, or with doxygen, you can get pretty far writing skeleton classes and method stubs, with exogenous systems represented the way you might mock them. Then just drop the generated SVGs into your slides, and profit.
This also answers your question about the lack of tooling: whenever a promising architecture/modeling product shows up they get acquired and then just fade away.
I actually got into that habit back when I was working with the insanely expensive suite of Rational IDEs. At that point, and later with the Eclipse Modeling Framework, the goal was to generate code from diagrams.... but that took more work than it was worth in my opinion.
With those tools, or with doxygen, you can get pretty far writing skeleton classes and method stubs, with exogenous systems represented the way you might mock them. Then just drop the generated SVGs into your slides, and profit.
This also answers your question about the lack of tooling: whenever a promising architecture/modeling product shows up they get acquired and then just fade away.
I work in between policy and the actual scientists writing code and running experiments. I use Visio and LucidChart to make sure I've figured out how the decision flow in policy space can map to some realizable version of meat space and network space, and demonstrate to actual policy makers where their policies may not work (e.g. there's a pocket of bureaucrats who we know will squeeze the time out of our clocks, here's some software or infrastructure that will require additional authorization because it exists inside or outside a policy boundary, etc).
Some of those documents I never look at again. Some of those documents I show to other people and it's like watching lightening strike: "Oh, I like this, I like this a lot."
Some of those documents I never look at again. Some of those documents I show to other people and it's like watching lightening strike: "Oh, I like this, I like this a lot."
Taking the Edward Tufte seminar will probably get you further faster than poking around in 5 different applications for the next few months.
https://www.edwardtufte.com/tufte/courses
https://www.edwardtufte.com/tufte/courses
Whoa thanks a lot for this suggestion. It looks amazing.
I used MagicDraw in a previous job, it's decent. We had it setup so that we actually generated C++ class's from parts of the UML. As well as generating powerpoint and word/pdf documents from it. MagicDraw also comes with its own version control system so that everything can live centrally on the companies server.
I do wish that it integrated with Git, and that the app was slightly faster / less bloated, but all in all; it was a much better experience than Visio & PowerPoint. These days I'm unfortunately back on PowerPoint architecture.
Enterprise Architect and MagicDraw are as far as I know the two most popular UML diagramming tools.
I do wish that it integrated with Git, and that the app was slightly faster / less bloated, but all in all; it was a much better experience than Visio & PowerPoint. These days I'm unfortunately back on PowerPoint architecture.
Enterprise Architect and MagicDraw are as far as I know the two most popular UML diagramming tools.
It would be nice to see graphics representation of the software architecture in a README.md
> It then gets saved as PDF and shelved as an artefact that is disconnected from all the other architectures, and the system boundaries are inevitably out of date by the time the next person looks at it.
Personally, I think documentation should be stored into source control like git. Any changes needs to go thru this document first. Specification by Example is one I think would be beneficial. However, all of these takes human willpower and disciple and business buy-in... and is probably why I'm stuck reading an ultra-slow confluence that is not updated.
> It then gets saved as PDF and shelved as an artefact that is disconnected from all the other architectures, and the system boundaries are inevitably out of date by the time the next person looks at it.
Personally, I think documentation should be stored into source control like git. Any changes needs to go thru this document first. Specification by Example is one I think would be beneficial. However, all of these takes human willpower and disciple and business buy-in... and is probably why I'm stuck reading an ultra-slow confluence that is not updated.
I do do feel that this is an unsolved problem, because capturing structure, behavior and temporarily in any sort of meaningful visual drawing is difficult. Even structure itself alone with its deep nesting is complex to depict.
As a tool, I find LucidChart fantastic for mapping architectures. It's a much easier to use, nice version of Visio, IMO. It is simple enough to use that other people adapt and maintain LucidCharts pretty well.
I had an idea and made a proof of concept for a framework based on ontology language that generated both cloud resources (AWS cloud formation, with the plan to target similar infrastructure description languages) as well as a set of code snippets for CRUD operations to be deployed to those resources. The nice part was that you'd have all your types and data flow modeled in a source-controlled set of repos, and you could generate and publish up-to-date system diagrams at any level of granularity you wanted. Seemed cool but I never got around to doing much with it.
I use LucidCharts and then export to SVG, pretty print, and embed that SVG in the repo. This gives versioning, use of an easy tool, and the ability for me to include it in markdown texts in my internal repo.
I spend a lot of time in python land and as of late I’ve been using Sphinx mixed with django-extensions graph_models and some hand edited graphviz. This allows the lion share of my documentation to live in actual code and for all documentation to be version controlled, architectural docs can reference real classes and code documentation can reference architecture docs, stir in the ability to hack simple Sphinx plugins in python and it’s been suiting my needs for everything from architectural overviews to operational “run books” and how-to guides.
I guess there are many suggestions and tool offering here, they are very helpful to software ARCH, I can offer one alternative option here,
which is mind map tools, it can help you make your thought of ARCH become very clear,
in some sense, it will save you a lot of effort afterward: xmind and mindmanager.
https://www.xmind.net/ https://www.mindjet.com/mindmanager/
https://www.xmind.net/ https://www.mindjet.com/mindmanager/
I personally found that the most useful diagram to help me understand a Rails project I don't know is the E/R automatically generated from this tool: https://github.com/voormedia/rails-erd
Being automatically generated, it's always up to date. As many others noted, I never had luck with written docs because it never happened to me to find good ones which were in sync with the project.
Being automatically generated, it's always up to date. As many others noted, I never had luck with written docs because it never happened to me to find good ones which were in sync with the project.
I have found that a modelling tool (such as Sparx Enterprise Architect or Archi) as opposed to a drawing tool is useful.
With a modelling tool there are increased opportunities for reuse and for the information to be presented in different ways (with differing levels of detail) depending on what needs to be communicated.
It's not ideal, but I do however often still use both Visio and Powerpoint to better communicate the information in a more visual manner.
> where is the modern 'IDE' for the Software Architect?
I have had success with http://staruml.io/
I have had success with http://staruml.io/
maybe have a look at https://mermaidjs.github.io. it's markdown for diagrams
I used mermaid in the past. Now I use nomnoml:
https://github.com/skanaar/nomnoml
Love mermaid. Very simple syntax for sequence diagrams and graphs.
This is a problem that I struggle with. I proposed what in my IMHO is a fairly comprehensive system for documenting architecture. It starts from stakeholders and goes on till it reaches instrumentation. Here is a video that explains this. Would appreciate comments on this. https://www.youtube.com/watch?v=UCoXgafOyMw
Piggy backing on your post, I have a very large and complex systems of architectures (both hardware and software). I'd love to take that documentation and run analyzation on the data/node and their relationships (maybe via a graph DB in the backend and some rich search capabilities). Anyone know of any tools that not only help you draw and document, but interface with the diagrams via API?
I wonder how useful it might be to try sketching cartoon technical explanations of the type that Julia Evans or Lin Clark do. Then again, how long does it take to develop the manual skill to produce these?
http://jvns.ca/zines/
https://code-cartoons.com/
http://jvns.ca/zines/
https://code-cartoons.com/
Try using a modelling tool rather than a drawing tool.
I have been a little out of the game lately but I remember that things like ArchiMate [1] try to solve or at least improve your situation. But this may very well be out of date. There is some research into improving IT architecture management going on all the time.
1: https://www.archimatetool.com/
1: https://www.archimatetool.com/
I tend to use Confluence with the Draw.io plugin. I have a custom template for specification and requirements along with a business process. Changes are captured in the history and highlighted as part of the business process.
It is a much better system for my team compared to getting changes by email or phone.
It does take some discipline to execute, but it is worth the effort in my opinion.
It is a much better system for my team compared to getting changes by email or phone.
It does take some discipline to execute, but it is worth the effort in my opinion.
Use mediawiki, the software that runs Wikipedia, to document. It doesn't easily go into book format, but hypertext is really powerful and easy to use. You can also embed powerpoints in it.
Best of all, updates and versioning are easy as it's a living breathing hyperlinkable system with a full change history built in.
Saved PDFs that nobody read anymore is one of the issue we are trying to address with https://mototomo.io
We want to tie your architecture visualization with your CI, making it an integral part of your daily workflow.
We focus on mobile apps at the moment but more stacks are in the pipeline.
We want to tie your architecture visualization with your CI, making it an integral part of your daily workflow.
We focus on mobile apps at the moment but more stacks are in the pipeline.
This reminds me of an idea I had: building service maps from logs.
If your logs have:
1) common format
2) service application names
3) tracer bullets
Then in theory it should be possible to create a crude visualization of a service mesh across your apps. It'd be a lightweight version (or extension) of Zipkin, I think. Although I'm sure there's a bunch of edge cases I'm not thinking of.
If your logs have:
1) common format
2) service application names
3) tracer bullets
Then in theory it should be possible to create a crude visualization of a service mesh across your apps. It'd be a lightweight version (or extension) of Zipkin, I think. Although I'm sure there's a bunch of edge cases I'm not thinking of.
We’ve at least come up with a way to programmaticaly declare links. We now can add comments above endpoint definitions in code and upload these “edges” to a central server that stores them to a graph. It has a web frontend that renders the graph (via tree.js I think). Not OSS though.
yEd from yWorks - https://www.yworks.com/products/yed has excellent features (with auto-arranging of nodes for better readability). Might be useful :)
ps - it's FREE!
ps - it's FREE!
I only abide with one rule - the documentation that you would create, whether diagrams, contextual model, or any artifact related to the software should be something worthwhile to read. Otherwise, it all becomes a meaningful assumption with zero audience.
Product Requirements Documents and Design Documents are a good start. At least then you have a 'human readable' understanding of what they were trying to accomplish. Note that diagrams are often part of a design document.
I'd like to understand if is there is some sort of "Literate Programming" approach to this problem? Interleaving the architecture through the (combined) code + architecture document?
Questions, isn't Terraform exactly suited for this? And doesnt it come with rudimentary charting features? I mean the charting feature is terrible by default but it's something, no?
We use the Flowchart Diagrams stencils in Moqups just for that. And, because it's collaborator friendly, your team can hold you accountable if the diagrams become out of date.
Enterprise Architect by Sparx is by far the best one I’ve used. At least as of a few years ago. The best user experience / convenience / features.
I've had tremendous success using a custom modeling language inspired by and older version of the Archimate approach and some simple diagramming tools (I typically use yEd, but you can use pretty much anything that supports the ability to layer sub-diagrams like you might in horizontal swim-lanes).
The approach is to use different stacked layers to represent different levels of abstraction or specificity and how they all align to a business goal at the top layer. The layers can change depending on the environment but typically are as follows:
The basic idea is to describe, as a stack, all the components that go into servicing a given business goal. Start with the hardware, and add boxes going up the stack until you've described all the components of the system. Connect them with arrows pointing up to the next layer. If you need to group things together at a layer, use a box to group them.
Some components might live in one lane or another (or both), depending on how you think about them.
Use colored arrows to show dataflow between components at the logical layer (you can infer the flow between all other lower layers from the stacks). I like to use red, blue and green to model the dataflow at different system states like red for deployment, green for operational and blue for configuration.
Add or remove layers as you deem fit. Urls, usernames/passwords, IP addresses, specific config options and so on fit in the descriptive components. For some smaller ER diagrams you can even fit them right in the lane or have a call-out to another diagram with the larger diagram.
Some people like to use different colors for the different layers (e.g. Archimate-style).
Print off the diagram on a plotter every so often so you can stick it on a wall and write on it as things develop. Adjust the diagram in a sync every week or two to keep it up to date.
The goal is to have a flexible framework and not to get locked into a modelling approach with such a high specificity that you're sweating over which kind of arrow represents exactly what thing, but if you need that for certain cases you can. The layered approach helps to decouple/deconvoluted some of the density of meaning in other languages that often has people having to look symbology up just to read the diagram. You can adjust/tweak as necessary and most projects have some different "dialect" of this language. Fill it out with just as much specificity as you need to answer any question you have. Often entire boxes just have a ??? for the description because it's not entirely necessary to know the details.
I've used this to describe fairly complex systems that serve millions of requests per day and after 3 or 4 iterations the diagrams were sufficient to answer any non-code-specific question we had. Multiple systems can be displayed next to each other and the dataflow arrows show interactions so you can even show very complex inter-service interactions as well.
I think the most important factor here is to use it as a mix of both documenting work that's been done (the existing system) and use different looking boxes to show aspiration or forward looking work to do (maybe dashed lines or a different color or whatever). Don't try to design the entire system at the start or you'll fall into disharmony with your agile developer teams.
I put a really trivial example here (made in yED) https://imgur.com/a/H8dCwoD I made this in just a couple minutes using mostly default yED pallet options if that helps you understand the level of effort (probably spent more time tweaking the colors to white and the font tbh).
The approach is to use different stacked layers to represent different levels of abstraction or specificity and how they all align to a business goal at the top layer. The layers can change depending on the environment but typically are as follows:
Top - Business Goals
(Top - 1) - Specific Software Implementation
(Top - 2) - Application and run-time environment
(Top - 3) - Containerization Layer
(Top - 4) - Virtualization Layer
(Top - 5) - Hardware Layer
The basic idea is to describe, as a stack, all the components that go into servicing a given business goal. Start with the hardware, and add boxes going up the stack until you've described all the components of the system. Connect them with arrows pointing up to the next layer. If you need to group things together at a layer, use a box to group them.
Some components might live in one lane or another (or both), depending on how you think about them.
Use colored arrows to show dataflow between components at the logical layer (you can infer the flow between all other lower layers from the stacks). I like to use red, blue and green to model the dataflow at different system states like red for deployment, green for operational and blue for configuration.
Add or remove layers as you deem fit. Urls, usernames/passwords, IP addresses, specific config options and so on fit in the descriptive components. For some smaller ER diagrams you can even fit them right in the lane or have a call-out to another diagram with the larger diagram.
Some people like to use different colors for the different layers (e.g. Archimate-style).
Print off the diagram on a plotter every so often so you can stick it on a wall and write on it as things develop. Adjust the diagram in a sync every week or two to keep it up to date.
The goal is to have a flexible framework and not to get locked into a modelling approach with such a high specificity that you're sweating over which kind of arrow represents exactly what thing, but if you need that for certain cases you can. The layered approach helps to decouple/deconvoluted some of the density of meaning in other languages that often has people having to look symbology up just to read the diagram. You can adjust/tweak as necessary and most projects have some different "dialect" of this language. Fill it out with just as much specificity as you need to answer any question you have. Often entire boxes just have a ??? for the description because it's not entirely necessary to know the details.
I've used this to describe fairly complex systems that serve millions of requests per day and after 3 or 4 iterations the diagrams were sufficient to answer any non-code-specific question we had. Multiple systems can be displayed next to each other and the dataflow arrows show interactions so you can even show very complex inter-service interactions as well.
I think the most important factor here is to use it as a mix of both documenting work that's been done (the existing system) and use different looking boxes to show aspiration or forward looking work to do (maybe dashed lines or a different color or whatever). Don't try to design the entire system at the start or you'll fall into disharmony with your agile developer teams.
I put a really trivial example here (made in yED) https://imgur.com/a/H8dCwoD I made this in just a couple minutes using mostly default yED pallet options if that helps you understand the level of effort (probably spent more time tweaking the colors to white and the font tbh).
After using Omnigraffle exclusively, I’ve taken a huge liking to PlantUML and alongside confluence... it serves 90% of my needs
ASCII art is the gold standard. You can still read those RFCs decades later. Add a little Markdown, and you're done!
Fire yourself up a Windows XP virtual machine, pour a glass of white zinfandel, and go to town with Rational Rose.
You could try https://archbee.io
Graphviz is surprisingly good for documenting architecture - bonus of bring easily versionable too!
[deleted]
There is a very good way to document software architectures:
I use BPMN 2.0 with DMN
Look it up, see what you think.
I use BPMN 2.0 with DMN
Look it up, see what you think.
Make documents live. Make them an organism. Swagger.io is the best idea ever.
You should instrument your systems to product the diagrams.
I manually write SVG, and maybe even animate them. Not the most productive way, but a picture can say more then thousand words. You can then put it into source control, web pages, or just have it as an image.
I would love a design IDE!
*Edit - off topic
*Edit - off topic
>It then gets saved as PDF and shelved as an artefact that is disconnected from all the other architectures
Sounds like architecture to me. You're doing it right already!
Sounds like architecture to me. You're doing it right already!
Draw.io has Amazon service icoms and GCP too. Did you try that?
I never knew architect was a thing in software.
The title is commonly found in heterogeneous projects where software isn't the solo driver. Ex: embedded systems
Pretty much the title just describes a senior dev., who owns the software aspect of the project.
Pretty much the title just describes a senior dev., who owns the software aspect of the project.
I'm working in automotive. The usual chain seems to be requirements engineer, system architect, software architect, software developer, integration engineer, tester.
Man, it seems like you totally lost connection with technology. There is no value in an architect who doesn’t code (none, if you think otherwise you’ve worked too long in enterprises that don’t deliver enough value to stay relevant the coming years).
Just use whatever drawing tools you know and draw up your diagrams. Should never take you hours to draw them up. How complex are they? If they take you so long, you probably are documenting way too much or are using way too complex architectures.
I feel the solution to your problem is more in the application than the tools. Try moving towards architectures like micro services. KISS. Then use the time you’re freeing up to keep connected to your developers and write code.
I feel the solution to your problem is more in the application than the tools. Try moving towards architectures like micro services. KISS. Then use the time you’re freeing up to keep connected to your developers and write code.
One thing I loved about Amazon was having no architects, visio, powerpoints, UML in any of the teams I worked in.
Each service is designed, documented, implemented, deployed, maintained and advertised to other teams by the same few engineers.
Most documentation and runbooks are just text in wikis. If your service cannot be documented without drawing 10 boxes and 20 arrows you should split it in smaller services.
Each service is designed, documented, implemented, deployed, maintained and advertised to other teams by the same few engineers.
Most documentation and runbooks are just text in wikis. If your service cannot be documented without drawing 10 boxes and 20 arrows you should split it in smaller services.
> "There is no value in an architect who doesn’t code"
This is debatable. The OP almost certainly knows how to code, and I think you can still provide quite a lot of value even if all you do is architecture.
It's much like saying a building's architect is worthless if they're not also doing construction work. I don't buy it.
This is debatable. The OP almost certainly knows how to code, and I think you can still provide quite a lot of value even if all you do is architecture.
It's much like saying a building's architect is worthless if they're not also doing construction work. I don't buy it.
Buildings are still built using stones, wood, cement. That doesn't change much.
In my experience as a DevOps transformation expert, 'Architects' are mostly old devs that have been kicked upstairs. The whole world changes each year to an extreme. You have to get your feet wet.
In my experience as a DevOps transformation expert, 'Architects' are mostly old devs that have been kicked upstairs. The whole world changes each year to an extreme. You have to get your feet wet.
Shack - doesn't need an architect.
House - some do, some don't.
Commercial Building - yep.
Factory, Bridge or Infrastructure - you're insane if you don't.
Just like in construction, it depends on what you are building/expanding. Not all software is the same.
House - some do, some don't.
Commercial Building - yep.
Factory, Bridge or Infrastructure - you're insane if you don't.
Just like in construction, it depends on what you are building/expanding. Not all software is the same.
So you're calling (f.e.) Amazon a shack?
And the Burj Khalifa doesn't need an architect?
Sometimes the system doesn't need it sometimes it does. Not all software systems are the same.
I personally think the architect role is to look at the big picture first rather than trying to dictate the minutiae.
Sometimes the system doesn't need it sometimes it does. Not all software systems are the same.
I personally think the architect role is to look at the big picture first rather than trying to dictate the minutiae.
The point is that an architect who doesn't 'code' (read: doesn't know how the practical implementation is done) doesn't know what he is designing. It's too abstract.
A structural architect is called the same, but is not really comparable. Apples and oranges. At the least because the Burj Khalifa is still built under the same physics model as they used 100 years ago.
A structural architect is called the same, but is not really comparable. Apples and oranges. At the least because the Burj Khalifa is still built under the same physics model as they used 100 years ago.
A shanty town?
“Try moving towards architectures like micro services. KISS.”
I don’t think you have worked with micro services, or more importantly have had to manage them.
I don’t think you have worked with micro services, or more importantly have had to manage them.
All the time.
My initial impression is the same as the parent. Juxtaposing microservices and KISS seems weird as the "architectural complexity" of microservices is several times higher than a monolith. You're not only multiplying the points of failure but also increasing the difficulty of translating the domain into code, especially when it comes to reading and writing from a db in a safe and efficient manner. Someone will need to come up with an answer to handling transactional operations that span multiple services sooner or later (or I guess the team can just accept some small percent of data corruption, which is what I'm guessing most companies that do microservices actually do, willingly or unwillingly). Ironically I think the benefit of an architect is a lot more evident in a microservice architecture than a monolith.
Edit: though maybe my view on what an architect should do is different? I think of a software architect as the person that lays out the skeleton and foundations of a project and has the answers to hard questions. Very likely someone who actively codes or has solved very similar challenges to the ones being solved.
Edit: though maybe my view on what an architect should do is different? I think of a software architect as the person that lays out the skeleton and foundations of a project and has the answers to hard questions. Very likely someone who actively codes or has solved very similar challenges to the ones being solved.
The problem with this approach is that it might be easier to build a monolith, maintaining it and the ops side of it is far more complex.
Nice answer! No one else is trying to help this guy turn his life around, they are just answer in his question like sheep. /S
Sometimes the answer to the wrong question helps you further down the wrong path.
You're going to get a fair amount of hate for mentioning the word "architect" here. They go into the same naughty list as manual testers, Scrum masters and release managers! Despite this, they can be very necessary for large orgs.
I've used Ardoq in the past, decent tool. All manual. Sparx EA looks very similar to visio and heavily TOGAF influenced. Both models are tough to create and maintain. A DSL in their own right.
No one really cares about architecture diagrams until they're really needed i.e platform migrations, rewrites, onboarding new staff, better reserve instance pricing. I've seen 6 different development teams all write architecture diagrams differently. All of them out of date and inconsistent.
I do think there is an opportunity here to build something more useful, the difficulty I see is that architecture is quite dependent on perspective. Security sees perimeters and firewalls, development sees microservices and DBs, DevOps see VPCs and networking.
I did come across weave works visualisation tools for Kubernetes that seem impressive and an OpenSource project but haven't had a chance to play with it yet.
You're right in your initial assumption that there is nothing out there to model architecture easily. Visio is as good as we have right now.
I've used Ardoq in the past, decent tool. All manual. Sparx EA looks very similar to visio and heavily TOGAF influenced. Both models are tough to create and maintain. A DSL in their own right.
No one really cares about architecture diagrams until they're really needed i.e platform migrations, rewrites, onboarding new staff, better reserve instance pricing. I've seen 6 different development teams all write architecture diagrams differently. All of them out of date and inconsistent.
I do think there is an opportunity here to build something more useful, the difficulty I see is that architecture is quite dependent on perspective. Security sees perimeters and firewalls, development sees microservices and DBs, DevOps see VPCs and networking.
I did come across weave works visualisation tools for Kubernetes that seem impressive and an OpenSource project but haven't had a chance to play with it yet.
You're right in your initial assumption that there is nothing out there to model architecture easily. Visio is as good as we have right now.
Do point towards the "architect" hate pages, please. You've piqued my interest.
Scan up and down this thread. Comments are saying he/she shouldn't have a job, he/she should be writing code instead, who is h/shee to dictate to delivery teams.
This was posted a few weeks back by a DRI in Stripe, https://hyperbo.la/w/aws-org-chart/, which sounds alot like what traditional architects do. Nobody showed any interest.
Anytime I've attempted to raise a discussion on this it gets shut down quickly.
Yeap, maybe hate is a strong term. Certainly bad feeling towards the practice.
This was posted a few weeks back by a DRI in Stripe, https://hyperbo.la/w/aws-org-chart/, which sounds alot like what traditional architects do. Nobody showed any interest.
Anytime I've attempted to raise a discussion on this it gets shut down quickly.
Yeap, maybe hate is a strong term. Certainly bad feeling towards the practice.
The analogy I sometimes use is that you don't need an architect to design a garden shed, but you do for a house which is much larger, has complex integrations with plumbing, electrics etc., plus building regulations to comply with, etc. A lot of dev work, especially in smaller companies, is building garden sheds, i.e. small single purpose self contained apps, but that doesn't mean every building is a garden shed.
I like that. The logic is often "I worked on a team that didn't require an architect, therefore no teams require architects".
I do readily admit there are quite a few hapless architects out there though.
I do readily admit there are quite a few hapless architects out there though.
If you need documentation for architecture, your architecture is complicated. Improve the architecture instead.
[deleted]
Sometimes the least complicated architecture is still complicated (by human standards).
What love about the Shinkansen, at least Tokyo <-> Osaka is that it leaves every 5-10 minutes so I never make reservations. I just casually go when I feel like it, shop outside for a snack, then go in and get a ticket at a machine and hop on the next train.
This is something I can't do with planes as well as you have to book in advance, get there early, check in, get inspected, line up to board. Filling plane seems to take 15 to 30 minutes. Filling the train takes 2-3 mins. And of course the train drops me off in the center of town, not 30 to 90 minutes outside of town like most airports.
There's a couple of seasons where getting a seat or getting 2 seats together might take more planning but most of the time taking the Shinkansen feels no different than taking any local train or subway in terms of prep which makes it super easy to use. It's a little expensive, $240-$350 round trip (free seating->reserved seating->1st class) but I've still made impulse trips for various events deciding on the same day that, yea, ok, let's go!
I will say China's bullet trains are amazing as well but booking/tickets are not nearly as smooth.
This is something I can't do with planes as well as you have to book in advance, get there early, check in, get inspected, line up to board. Filling plane seems to take 15 to 30 minutes. Filling the train takes 2-3 mins. And of course the train drops me off in the center of town, not 30 to 90 minutes outside of town like most airports.
There's a couple of seasons where getting a seat or getting 2 seats together might take more planning but most of the time taking the Shinkansen feels no different than taking any local train or subway in terms of prep which makes it super easy to use. It's a little expensive, $240-$350 round trip (free seating->reserved seating->1st class) but I've still made impulse trips for various events deciding on the same day that, yea, ok, let's go!
I will say China's bullet trains are amazing as well but booking/tickets are not nearly as smooth.
Wrong thread mate
While the system definitions themselves have arguably improved as my skill as an architect has increased, there's been no such improvement in the speed or method I use to describe them. My visuals are perfunctory, powerpoint-fu is lacking, and the end result always has plenty of room for improvement. It then gets saved as PDF and shelved as an artefact that is disconnected from all the other architectures, and the system boundaries are inevitably out of date by the time the next person looks at it.
So much craft has been put into better languages, better compilers, and better IDEs for the software developer, I'm absolutely confused - where is the modern 'IDE' for the Software Architect?