Show HN: Project Management as Code with Graphviz(zwischenzugs.com)
zwischenzugs.com
Show HN: Project Management as Code with Graphviz
https://zwischenzugs.com/2017/12/18/project-management-as-code-with-graphviz/
65 comments
I frequently make use of http://viz-js.com/ so I can generate dot graphs without needing to install GraphViz.
I built a tool for generating graphviz graphs at http://dottask.mattboehm.rocks .
Once you get used to the controls, it's very quick and has a ton of useful features (including export to dot so that you can further customize things.)
Once you get used to the controls, it's very quick and has a ton of useful features (including export to dot so that you can further customize things.)
Handy, thanks!
Glad to see somebody else trying something like this. I was manually maintaining some charts like this for a while at a prior job and was toying with productizing it, it's super powerful, especially if you've got a large backlog or a large feature list ahead of you.
I wanted to have another representation that generated the GraphViz files, though, so you could more easily do other work with the data. Some other features we thought we needed:
* Notation of whether a dependency was "can't start until it's done" or "can't finish until it's done" to designate whether or not an item and it's deps could be in progress concurrently or if the earlier project needed to be completed first.
* Attach rough time estimates to each node (person-week granularity)
* Generate a list of "eligible" projects, i.e. those things with no hard blockers
* Sort this list by how much downstream work it blocks. Probably by each of: number of work weeks blocks, number of projects it blocks, and depth of the tree it blocks. This would help identify the boring projects that stand in the way of everything.
* Generate a list of "up next" projects that had only one in-progress blocker and thus were eligible for moderate discussion, but couldn't be allowed to dominate meeting time.
* Generate another list of "off limits" projects that had at least one blocker that wasn't even in progress yet and thus were a waste of staff time to meet about. No point having design meetings when something is 3 nodes deep in the graph.
I ultimately left because the management was insane and irresponsible and undermined these efforts once they started to set in well (because they removed her absolute authority to monkey with priorities). But I think it's a system that works well with sane management. At present I don't have enough backlog to need this tool, but look forward to working on it again one day.
I wanted to have another representation that generated the GraphViz files, though, so you could more easily do other work with the data. Some other features we thought we needed:
* Notation of whether a dependency was "can't start until it's done" or "can't finish until it's done" to designate whether or not an item and it's deps could be in progress concurrently or if the earlier project needed to be completed first.
* Attach rough time estimates to each node (person-week granularity)
* Generate a list of "eligible" projects, i.e. those things with no hard blockers
* Sort this list by how much downstream work it blocks. Probably by each of: number of work weeks blocks, number of projects it blocks, and depth of the tree it blocks. This would help identify the boring projects that stand in the way of everything.
* Generate a list of "up next" projects that had only one in-progress blocker and thus were eligible for moderate discussion, but couldn't be allowed to dominate meeting time.
* Generate another list of "off limits" projects that had at least one blocker that wasn't even in progress yet and thus were a waste of staff time to meet about. No point having design meetings when something is 3 nodes deep in the graph.
I ultimately left because the management was insane and irresponsible and undermined these efforts once they started to set in well (because they removed her absolute authority to monkey with priorities). But I think it's a system that works well with sane management. At present I don't have enough backlog to need this tool, but look forward to working on it again one day.
We should talk. I was thinking size of node could == some kind of measure of size/risk/significance. There's lots of things one might want to visualise, so some collaborated standard would be good.
Don't forget other modalities. E.g.
Size: effort estimate
Number of edges to the shape: depth of dependencies.
Brightness: size of dependencies.
Hue: ...tbd...
Border (double/triple/dashed): ...tbd...
Font size: ...tbd...I like it. Another one for part 2.
Sounds good. I'm not in a position where I can spend a ton of time working on it, but I'm definitely game to be a sounding board for the design. I have the .com for my username, contact info is there.
Code here: https://github.com/ianmiell/pm-as-code
I'm not a GV expert, so contributions/suggestions/critiques welcomed.
I'm not a GV expert, so contributions/suggestions/critiques welcomed.
Nice stuff.
You might want to look at ordering things left to right with
Also subgraphs to group certain sections if you want: http://graphs.grevian.org/example
This may also be useful to see the building up of a bit more complex graph http://larsbarkman.com/2015/08/26/alignmentmaps-with-graphvi...
You might want to look at ordering things left to right with
rankdir=LR;
Also subgraphs to group certain sections if you want: http://graphs.grevian.org/example
This may also be useful to see the building up of a bit more complex graph http://larsbarkman.com/2015/08/26/alignmentmaps-with-graphvi...
Thanks, I may include these in a deeper dive.
nice one.
Dot (both the layout engine and the graph DSL) is used in OmniGraffle, so you can import the dot files like you made into OG and edit them further, either manually or via OG's scripting interface
and then for instance, quickly convert text inside a node to a link to a suboordinate graph (ie, the top-level graph might describe a large project, and each node in that graph, might represent a "sub-project"--hence you can represent a very large project as a hierarchy of graphs, eg, one per sub-project, but that still roll-up to a single top-level. This is a core feature in OG and trivial to do
Dot (both the layout engine and the graph DSL) is used in OmniGraffle, so you can import the dot files like you made into OG and edit them further, either manually or via OG's scripting interface
and then for instance, quickly convert text inside a node to a link to a suboordinate graph (ie, the top-level graph might describe a large project, and each node in that graph, might represent a "sub-project"--hence you can represent a very large project as a hierarchy of graphs, eg, one per sub-project, but that still roll-up to a single top-level. This is a core feature in OG and trivial to do
I'm interested in OmniGraffle and will check it out.
What a great idea. I really struggle to plan sprints and projects just using lists. Also MS Project etc are just far too complicated for my small projects.
I actually resorted to drawing dependency and flow diagrams by hand, then translating those into JIRA tickets and adding to Kanban style boards.
The ability to track changes using git is such a simple (and with hindsight, obvious) idea, but I should think very effective.
Nice one.
I actually resorted to drawing dependency and flow diagrams by hand, then translating those into JIRA tickets and adding to Kanban style boards.
The ability to track changes using git is such a simple (and with hindsight, obvious) idea, but I should think very effective.
Nice one.
Thanks - do get in touch with how to get on.
This is just a PERT chart, right? [1] Only difference is you don't appear to have any durations on the tasks. If you added that you would be able to calculate your critical path.
[1] https://en.wikipedia.org/wiki/Program_evaluation_and_review_...
[1] https://en.wikipedia.org/wiki/Program_evaluation_and_review_...
This is great, thanks for sharing. I stumbled across this idea myself since no todo manager really represented complex dependencies of tasks - the "list" format is just so prevalent and usually ends up increasing cognitive load as I can sometimes get frustrated trying to figure out what's "next" on a non-linear project.
Have you seen DropTask? It's the same idea, I just haven't dug too much into it yet: https://www.droptask.com/
The key feature for me is being able to have multiple ancestor/dependency tasks to a task node, instead of a hierarchical "next action" setup like most task managers.
Have you seen DropTask? It's the same idea, I just haven't dug too much into it yet: https://www.droptask.com/
The key feature for me is being able to have multiple ancestor/dependency tasks to a task node, instead of a hierarchical "next action" setup like most task managers.
Thanks for the kind words. I haven't seen droptask... I'll add it to the list. Although I have a prejudice for old and ugly tools like GV, Make et al.
Thanks! I never thought about applying this to my todo list, but its a great idea!
I have done something like this for creating "recursive recipes" [1] which generates similar looking dependency charts when you start probing how to make a recipe from scratch [2].
[1]: https://github.com/schollz/timetomakefood
[2]: https://user-images.githubusercontent.com/6550035/34167929-6...
I have done something like this for creating "recursive recipes" [1] which generates similar looking dependency charts when you start probing how to make a recipe from scratch [2].
[1]: https://github.com/schollz/timetomakefood
[2]: https://user-images.githubusercontent.com/6550035/34167929-6...
Thanks, I'll take a look.
a great gui editor for similar tree-node structures is freemind
a lot quicker than code dsl -> git -> graphviz
and freemind also has search, edit, history, links, export as HTML
http://freemind.sourceforge.net/wiki/index.php/Screenshots
a lot quicker than code dsl -> git -> graphviz
and freemind also has search, edit, history, links, export as HTML
http://freemind.sourceforge.net/wiki/index.php/Screenshots
A more active fork https://en.wikipedia.org/wiki/Freeplane
Can it export to code?
it is open source and you could write an extension to output whatever you like.
there is even the last resort of transforming the built in XML ouput ( yuck ). Several Perl examples are given.
http://freemind.sourceforge.net/wiki/index.php/Import_and_ex...
there is even the last resort of transforming the built in XML ouput ( yuck ). Several Perl examples are given.
http://freemind.sourceforge.net/wiki/index.php/Import_and_ex...
Really interesting and well written post. Certainly has started some cogs turning in my mind.
Just a note, I think there is some text missing or leftover at the end of the 'Surface Assumptions' paragraph.
Just a note, I think there is some text missing or leftover at the end of the 'Surface Assumptions' paragraph.
Cheers, have corrected.
This reminds me of an experiment I had going a year or so ago. Same basic idea, but far more automated: I'd been using taskwarrior for years, and had gotten into the habit of breaking what I was doing into multiple taskwarrior tasks with dependencies. Out of curiosity, I wrote a bash script that extracted those dependencies and created a graphviz dot file.
(I never really found much use for it though, and had forgotten about it until reading this. I now just prefer taskwarrior's urgency ranking in its regular output.)
(I never really found much use for it though, and had forgotten about it until reading this. I now just prefer taskwarrior's urgency ranking in its regular output.)
Gantt charts also represent dependencies among tasks.
They add more information as well: time estimation, milestones, asignees, etc.
Gantt charts receive some criticism since estimations are approximations and any precision issue starts compounding very quickly.
Issue trackers may also support dependencies between tasks. The problem is that issue trackers usually come with scrum/kanban boards that do not have a represention dependencies among tasks.
There are many applications to put together a gantt chart.
They add more information as well: time estimation, milestones, asignees, etc.
Gantt charts receive some criticism since estimations are approximations and any precision issue starts compounding very quickly.
Issue trackers may also support dependencies between tasks. The problem is that issue trackers usually come with scrum/kanban boards that do not have a represention dependencies among tasks.
There are many applications to put together a gantt chart.
Can I export it to code?
I am not sure a GraphViz digraph is the way to go.
If you want to keep track of what is done and what is remaining eventually you will need to update that file.
This will quickly lead to version control conflicts.
Issue trackers I think are the way to go. You can integrate with one using their API.
If you want to keep track of what is done and what is remaining eventually you will need to update that file.
This will quickly lead to version control conflicts.
Issue trackers I think are the way to go. You can integrate with one using their API.
I think version control conflicts are good in this case - they indicate the level to which there is conflict within the team around what the project consists of.
Version control conflicts in this case don't necessarily map to disagreement.
Imagine 2 team members that are in agreement add leaf 2 nodes. That would trigger a version control conflict.
Now imagine that situation in a team of 5, or 10. It becomes hard to maintain.
Imagine 2 team members that are in agreement add leaf 2 nodes. That would trigger a version control conflict.
Now imagine that situation in a team of 5, or 10. It becomes hard to maintain.
That is an odd question. There is data underlying any type of chart. You would work that data from your data layer, not your UI, and then code something new with it, or re-render a new visualization as you see fit.
Some data is easier to collaborate with as code than others. This is one reason why XML was superceded by JSON, and YAML exists as a rival.
GUI apps are great but less good for sharing. The conflicts are part of the point.
GUI apps are great but less good for sharing. The conflicts are part of the point.
plantuml is also text based and supports many types of diagrams:
http://plantuml.com/gantt-diagram
Perhaps someone will find this old-ish project of mine[1] useful. It automatically generates graphs using Graphviz based on Taskwarrior queries, which makes it considerably easier to manage and filter task lists.
[1]: https://github.com/shushcat/taskgv
[1]: https://github.com/shushcat/taskgv
I use this regularly.
Mixing this with http://orgmode.org/worg/org-contrib/babel/languages/ob-doc-d...
Would be pretty awesome. Your todo list could have graphs in them or dependancies.
Would be pretty awesome. Your todo list could have graphs in them or dependancies.
Great write-up! I tried something similar for modelling out the Snowplow release dependency chain (https://github.com/snowplow/) - I gave up and started looking at SaaS alternatives, but inspired by this I am going to try again.
For the record, I didn't find any good product management or project management tools (SaaS or desktop) which acknowledge that roadmapping any sufficiently complicated platform or product family is best expressed by a directed acyclic graph. Casual.pm had the right concept (https://casual.pm/) but the implementation was very primitive.
For the record, I didn't find any good product management or project management tools (SaaS or desktop) which acknowledge that roadmapping any sufficiently complicated platform or product family is best expressed by a directed acyclic graph. Casual.pm had the right concept (https://casual.pm/) but the implementation was very primitive.
I've found it quite hard to get non-technical people on board with this.
Had the same sentiment about casual.pm
Thanks for writing the post. Although I've heard of graphviz (and dot) before, I didn't take the time to read up on them since the docs were so dense. Thanks for making this accessible.
Thanks! Yes, I'd always shied away from GV and just copy/pasted examples until I used them at work and had to work through some problems.
Is there interest in a deeper post on graphviz from the POV of the casual user?
Is there interest in a deeper post on graphviz from the POV of the casual user?
How much do people use dependencies in jira etc? Seems that one ought to be able to create a plugin that would automatically generate and update this graph for any issue with dependencies.
Bugzilla does it out of the box, even using GraphViz.
However, Bugzilla only has one flavour of dependency ("blocks"), whereas JIRA implements the whole Allen Interval Algebra (which, however, only those people use who have ever wondered why the number of Allen relations is exactly 13).
However, Bugzilla only has one flavour of dependency ("blocks"), whereas JIRA implements the whole Allen Interval Algebra (which, however, only those people use who have ever wondered why the number of Allen relations is exactly 13).
I looked into this.
There are plugins, but I can't add plugins at will at work. People don't use deps properly though - GIGO.
There are plugins, but I can't add plugins at will at work. People don't use deps properly though - GIGO.
Well, yes both are then blockers to the success of this approach. And on the other side of the coin, knocking up one of these dependency graphs is reasonably quick and straightforward. Its a thought provoking article - thanks.
Thank you, very interesting example, but I don't necessarily see the added value in the graph form, for this particular example - it implies dependencies where there are none - for example buying GBP only after buying the guide book etc.
I think it is just the example that is wrong. What's typical for a holiday break, is a todo list, which is in no particular order.
Perhaps an actual project would be a better example?
The dependency order is the other way round:
Enjoy Holiday
depends on
Pack suitcase
depends on
Buy guide book
depends on
buy GBP
I could have used a 'real' example but then it would have got bogged down in the details of that example. Most of us have planned for a holiday, so I just used that example. Maybe 'moving house' would have been better, but that's more culture-specific.
I'd welcome a PR if you'd want to add a better example.
Enjoy Holiday
depends on
Pack suitcase
depends on
Buy guide book
depends on
buy GBP
I could have used a 'real' example but then it would have got bogged down in the details of that example. Most of us have planned for a holiday, so I just used that example. Maybe 'moving house' would have been better, but that's more culture-specific.
I'd welcome a PR if you'd want to add a better example.
Thanks for sharing!
I've been using the "same" technique with some projects I have but just to outline specific components of the system such that it becomes more clear what interacts with what.
For me, the best that I got out of it was having to think hard about those relations and improving the documentation of the subject.
Have a good one!
I've been using the "same" technique with some projects I have but just to outline specific components of the system such that it becomes more clear what interacts with what.
For me, the best that I got out of it was having to think hard about those relations and improving the documentation of the subject.
Have a good one!
Pragmatic approach, I like that and am willing to play with it. Combining it with https://en.wikipedia.org/wiki/Critical_path_method could yield interesting results.
Most of these graphs don't seem to have too many connections.
Wouldn't a checklist layout be much better?
Wouldn't a checklist layout be much better?
For simpler projects, yes. The more complex and shared the project the more value there is in documenting and sharing dependency relationships in code, as described in the article.
Seems really overkill
The toy example? Of course!
For an actual large, complex project with lots of inter-dependencies among sub-projects and tasks – arguably not.
For an actual large, complex project with lots of inter-dependencies among sub-projects and tasks – arguably not.
I love Graphviz but hate the Dot language with a passion. Has anyone tried to come up with a more consistent and user-friendly graph definition language?
Check out mermaid.
Thanks. That looks like just what I need to create class and sequence diagrams in a version control-friendly manner.