Launch HN: Mintlify (YC W22) – Maintainable documentation for software teams
58 comments
Hi I've been a technical writer for ~9 years: 3 at a startup and 6 at Google. I agree that documentation maintenance is probably the real hard problem. I think the winner of this space will be the one who solves the "connect code to documentation" aspect most effectively. Looking at your demo, I don't think the code/doc connection should be invisible metadata. In other words my hunch is that the code/doc connection needs to be obvious in code/doc themselves although I'm not sure why I feel this way at the moment. Also the root causes of stale documentation are usually a combination of 1) documentation not being incentivized by org leadership 2) the gradual but persistent proliferation of docs. I don't mean to say that these are people problems that are not solvable by technology. I do think they are solvable by technology but you'll need to get very creative and innovative to solve them. E.g. to address #1 maybe some kind of reporting system that makes leadership aware of which engineers are doing docs work that is truly creating value for the org. Happy to chat 1:1 and thank you for bringing innovations to my industry!
I've taken to supporting cross-references across the repo, connecting references of the form ^^{foo} to anchors of the form @@{foo}, wherever they appear (code, comment, docs).
Then I have CI check referential integrity and also surface connections in code review (adding annotations to all references when the corresponding anchor shows up in the diff in changed lines or context, and vice-versa).
In the idealized straight-forward elaboration, then, all docs would live in the repo, but I haven't worked with a team that was on board with that.
I'd also like to see a better tie between docs and tests.
Then I have CI check referential integrity and also surface connections in code review (adding annotations to all references when the corresponding anchor shows up in the diff in changed lines or context, and vice-versa).
In the idealized straight-forward elaboration, then, all docs would live in the repo, but I haven't worked with a team that was on board with that.
I'd also like to see a better tie between docs and tests.
> In the idealized straight-forward elaboration, then, all docs would live in the repo, but I haven't worked with a team that was on board with that
In case you're thinking about productizing your work I just want to give you a little encouragement that there are a lot of companies OK with having docs live alongside code. See Google's g3doc [1] for example. See also the docs as code movement.
[1] https://www.usenix.org/conference/srecon16europe/program/pre...
In case you're thinking about productizing your work I just want to give you a little encouragement that there are a lot of companies OK with having docs live alongside code. See Google's g3doc [1] for example. See also the docs as code movement.
[1] https://www.usenix.org/conference/srecon16europe/program/pre...
Thank you for sharing the article! We definitely don't see companies having all docs in their repos, but strongly believe that there should be stronger ties between docs and code (and more transparency in docs in general)
It's a really hard problem to solve, but we believe that there will be a product that will crack it (even if it's not what we have right now)
It's a really hard problem to solve, but we believe that there will be a product that will crack it (even if it's not what we have right now)
Thanks! At the moment it feels much too small to productize; virtually all of the complexity is the CI integration, and even that is small.
That's very interesting. How did you implement this? : "I've taken to supporting cross-references across the repo, connecting references of the form ^^{foo} to anchors of the form @@{foo}, wherever they appear (code, comment, docs)." Do you have a link? Would love to check it out myself. I'm also curious - why do you think most people are against docs living inside the repo?
> How did you implement this?
It wasn't public code, so no links handy, but the implementation was pretty much what you'd expect.
For checking referential integrity, make a list of all references and a list of all referents and diff the two. Trivial with git grep and a little bash, but pick your poison.
For surfacing connections, it was a matter of finding them in changes (which was approximately piping git diff to grep) , finding the corresponding locations in the whole repo, and then adding those locations to the PR, for which I used the github Checks API (you can add annotations in the output of a Check Run, see https://docs.github.com/en/rest/checks/runs#create-a-check-r...).
> why do you think most people are against docs living inside the repo?
For internal docs, I think it's an impression that Notion or Google Docs or Confluence or whatever is "easier", although it's usually been easier to get some measure of buy-in for at least some set of internal docs.
For external docs, it's "of course we can't teach git to anyone but engineers!" Which hasn't tracked my limited experience in trying to do so, but it can be hard to get them to try to learn in the first place.
It wasn't public code, so no links handy, but the implementation was pretty much what you'd expect.
For checking referential integrity, make a list of all references and a list of all referents and diff the two. Trivial with git grep and a little bash, but pick your poison.
For surfacing connections, it was a matter of finding them in changes (which was approximately piping git diff to grep) , finding the corresponding locations in the whole repo, and then adding those locations to the PR, for which I used the github Checks API (you can add annotations in the output of a Check Run, see https://docs.github.com/en/rest/checks/runs#create-a-check-r...).
> why do you think most people are against docs living inside the repo?
For internal docs, I think it's an impression that Notion or Google Docs or Confluence or whatever is "easier", although it's usually been easier to get some measure of buy-in for at least some set of internal docs.
For external docs, it's "of course we can't teach git to anyone but engineers!" Which hasn't tracked my limited experience in trying to do so, but it can be hard to get them to try to learn in the first place.
Gotcha, apologies for all the questions - did you implement this for your company?
That's interesting that the reception you got was primarily against documentation inside a repo. Maybe Gitbook could've been a good compromise? I recently made this post on reddit https://www.reddit.com/r/devops/comments/v6yuu5/whats_your_d... and many of the comments prefer to have documentation in the repo.
That's interesting that the reception you got was primarily against documentation inside a repo. Maybe Gitbook could've been a good compromise? I recently made this post on reddit https://www.reddit.com/r/devops/comments/v6yuu5/whats_your_d... and many of the comments prefer to have documentation in the repo.
> apologies for all the questions
No worries at all, it's half of what we're here for :)
> did you implement this for your company?
On a couple repos, yeah.
> many of the comments prefer to have documentation in the repo
I think there's a lot of us, but I also think we're more likely to want to talk about it, so it seems like there's more of us than there are.
I should say that I think most everyone's on board with a small amount of developer facing docs in the repo, especially a README and build instructions; whereas if I had my druthers, it'd even include pamphlets the sales team takes to conferences.
No worries at all, it's half of what we're here for :)
> did you implement this for your company?
On a couple repos, yeah.
> many of the comments prefer to have documentation in the repo
I think there's a lot of us, but I also think we're more likely to want to talk about it, so it seems like there's more of us than there are.
I should say that I think most everyone's on board with a small amount of developer facing docs in the repo, especially a README and build instructions; whereas if I had my druthers, it'd even include pamphlets the sales team takes to conferences.
Yeah, that honestly makes a lot of sense.
Non-engineers can edit Asciidoc or Markdown in the GitLab IDE.
True. Seems potentially very useful, but only relevant if you're using GitLab and want a relevant format (or something you can reasonably produce from a relevant format, although tooling questions then get potentially more complicated).
Thank you so much for the incredible feedback! We think you are absolutely SPOT ON. We would really love to chat more with you 1:1. Please reach out to us at [email protected]
I agree you need incentives for people to write documentation. Part of documentation is keeping it up to date (as shown in the demo video, keeping code in sync) but another important part is providing a way for newcomers to learn. Reference documentation is different to teaching documentation. Maybe a good platform can encourage this.
I can confirm the last point and want to support that with a taxonomy and talk that helped us a lot: https://documentation.divio.com/
Excellent suggestion. We're gonna work this into the product for sure
Maybe job rotation and showing people the difference between good and bad documents when they are on the receiving end helps?. My theory is that part of the problem is that effort and benefit are split across roles and the people thst should spend the effort never perceive that they get enough benefit back?
What do you think about documentation organization methodologies? And this one in particular: https://documentation.divio.com/
Love the Divio docs system. AKA https://diataxis.fr now. I regularly mention it when explaining basic docs strategy to stakeholders. Pretty much all professional technical writers subscribe to an organization methodology like this, they just may not use the Divio/Diataxis name. Also Divio/Diataxis did not invent this idea: they just did the best job explaining it.
Thanks for your input!
I agree with your sentiments have you looked at how Apple has integrated Doc-C with Xcode?
If anyone is wondering, Mintlify is not open source: https://github.com/mintlify/mintlify/blob/main/server/LICENS...
That is correct. The backend covers enterprise features that are only source-available, whereas the rest of the application is open source. We understand that this can be quite misleading, and will refrain from saying that we are open source in the future
I also suggest you add documentation to the github repo. The first thing I did after reading the first paragraph of this announcement was checking the source code. And when I saw no documentation at all - zero - nada - niente, I completely lost interest in looking more into the project (I was quite enthusiastic, given your description, so seeing no docs was an incredible turn-off).
Just a note based on the demo video above
1. it starts with the worst possible feature (if I chnage the docs then a slack notification is sent to everyone).
2. the code link feature is ... ok. It prevents a merge on certain code modules if the docs have not also been updated. Useful I guess.
I am missing the killer app here - surely going from the docs (ie this docentions this function) would automate the linkage but even so it seems a nice to have feature not the "aha" moment
Anyhow sorry for the usual negative HN tone - good luck
1. it starts with the worst possible feature (if I chnage the docs then a slack notification is sent to everyone).
2. the code link feature is ... ok. It prevents a merge on certain code modules if the docs have not also been updated. Useful I guess.
I am missing the killer app here - surely going from the docs (ie this docentions this function) would automate the linkage but even so it seems a nice to have feature not the "aha" moment
Anyhow sorry for the usual negative HN tone - good luck
> 1. it starts with the worst possible feature (if I chnage the docs then a slack notification is sent to everyone).
They show the update going to a #docs channel, that doesn't mean it notifies everyone. It notifies whoever wants to be notified of updates posted to the #docs channel. Currently using something similar for notion, channel membership is opt in, and most have notifications silenced
They show the update going to a #docs channel, that doesn't mean it notifies everyone. It notifies whoever wants to be notified of updates posted to the #docs channel. Currently using something similar for notion, channel membership is opt in, and most have notifications silenced
Fair enough (the words "notifies everyone" probably should be chanaged.)
But that's not really the point - it's that I don't get what they are doing that beats me coding something in an afternoon. (Inam trying really hard not to do the famous Dropbox comment)
But afaik the main feature here is you can add a piece of metadata in a comment that says "if this module is chnaged in a release branch, then some other documentation file must also be chnaged in the same branch.
Now, yeah that's kind of good as a policy. But if I am using docutils and my index.rst has a reference to "foo.bar.addmoney" then I can see how to set up a pre-commit hook and some worker that will do the same thing.
Now this is a pretty good feature to add to say Jenkins or GitHub Actions.
I could see that being a desirable feature for some set of teams.
I just don't like the "add extra meta data" thing, and I don't see what the win is outside of the CI tools
(not too sure how they are hooking into the CI ecosystems)
But hell good luck to you guys, you are in the arena. well done - good luck and I hope it works out.
Just please stop and work on that elevator pitch so that I do get it. I mean Inam interested enough in this area that i watched a video, read thirty comments and replied a couple of times.
I should be the perfect target customer.
But that's not really the point - it's that I don't get what they are doing that beats me coding something in an afternoon. (Inam trying really hard not to do the famous Dropbox comment)
But afaik the main feature here is you can add a piece of metadata in a comment that says "if this module is chnaged in a release branch, then some other documentation file must also be chnaged in the same branch.
Now, yeah that's kind of good as a policy. But if I am using docutils and my index.rst has a reference to "foo.bar.addmoney" then I can see how to set up a pre-commit hook and some worker that will do the same thing.
Now this is a pretty good feature to add to say Jenkins or GitHub Actions.
I could see that being a desirable feature for some set of teams.
I just don't like the "add extra meta data" thing, and I don't see what the win is outside of the CI tools
(not too sure how they are hooking into the CI ecosystems)
But hell good luck to you guys, you are in the arena. well done - good luck and I hope it works out.
Just please stop and work on that elevator pitch so that I do get it. I mean Inam interested enough in this area that i watched a video, read thirty comments and replied a couple of times.
I should be the perfect target customer.
Thank you, we really appreciate your feedback. We're both technical founders and we're very aware that we need to work on making the language clearer. Your feedback makes it crystal clear that we really need to work on this. Appreciate it.
No worries - don't take the feedback as a negative - I wish you all the best, this problem is a real issue (as is a lot around this point - see "your code as a crime scene" as a good book on this sort of automated analysis of likely areas code is falling down) and good luck for just being in the game - well done and wish you the best
Thanks for the feedback! We were planning on adding an ownership layer to it so that you can ping certain people who are the specific people who need to be notified.
Random thought: maybe you could connect a UI component (or State, or Service call) to a function chain and types used. Then reviewing a document could include links to code changes. It would be difficult to automatically determine which code changes matter. Programmers might be able to @ttribute functions, or include checking tags/links. Anyway, documents that are not one-to-one with a code file seem like the hardest to keep up to update.
Currently we use a repo's git information to create the link between code and documentation. We also support many-to-many connections so you could granularly choose lines of code to connect to documentation. But you make a good point! We could definitely do some work with AST parsers to select and connect specific functions/components to documentation.
The opening line says "Open source" , but they aren't. Source available is the correct term.
Totally agreed. While the frontend and core functions are open source, the backend under the `server` folder is source available. We made the mistake of referring to the entire repo as open source, which we will refrain from in the future.
We have a highly regulated environment and supply chain and have to go through a whole lot of GRC on a regular basis, and additionally one-off audits from some customers before they'll feel ok using our solution. So I will share here what I said to my folks:
"I looked further at the documentation solution I mentioned looked interesting. It is very nifty but it is pure SaaS, and I don't care how secure they are, having our source flow through another SaaS is not something I'm going to take on for any amount of pixy dust."
"I looked further at the documentation solution I mentioned looked interesting. It is very nifty but it is pure SaaS, and I don't care how secure they are, having our source flow through another SaaS is not something I'm going to take on for any amount of pixy dust."
We totally understand how important security is to companies. It the main reason why we are source available and self-hostable.
sounds interesting
i’m working on Typosaur, which is a spell-checker that can check websites (or any structured document really) for spelling mistakes, grammar and style issues and suggest corrections
link: https://typosaur.com/
maybe you’d be interested in integrating Typosaur-powered spelling suggestions into your offering?
if so my e-mail is linked in profile
i’m working on Typosaur, which is a spell-checker that can check websites (or any structured document really) for spelling mistakes, grammar and style issues and suggest corrections
link: https://typosaur.com/
maybe you’d be interested in integrating Typosaur-powered spelling suggestions into your offering?
if so my e-mail is linked in profile
We took a look around and am actually really interested! Sending you an email to discuss more.
Also, how did you manage to embed any website as an iframe without CORS or CSP issues?
Also, how did you manage to embed any website as an iframe without CORS or CSP issues?
cool!
the websites are pre-rendered using headless Chrome
the side-effect of this approach is that all the files (like images) are embedded into the HTML and the website also loses interactivity
works great for static documentation sites however!
the websites are pre-rendered using headless Chrome
the side-effect of this approach is that all the files (like images) are embedded into the HTML and the website also loses interactivity
works great for static documentation sites however!
It does indeed. Would love to jam more about this with you
Isn't the underlying problem that developers hate writing documentation for some reason? They will spend months writing some crazy audio framework in C++ but refuse to spend even a day writing a tutorial on how to install it.
Yeah. We started this whole endeavor because we, as developers, also hated writing documentation. We thought that if we made it easier - whether it's by automating it through AI or by bringing the documentation closer to the code - it would reduce the friction. We definitely think more can be done here to incentivize developers to update documentation.
I personally love writing, but it seems to me that if don't want to write, no amount of tooling will change that. Hope I'm wrong.
I find these solutions actually make documentation harder. They're binary and proprietary in nature, locked behind a paywall if I can no longer afford the service (and what's the exportable data going to look like? XML?), And the UI is often limited and leaves a lot to be desired.
My code is not binary, however, it's plain text. So I write my documentation in plain text too, such as Markdown. That can, should I so choose, be rendered to anything I like. It can be included alongside the code, which the developer will need a copy of anyway, and be easily reviewed (with "mkdocs serve" for example).
You're already committing code to a central repository and working in a decentralised manner. If you put your documentation in "./docs/" (I recommend MkDocs + the Material Theme) and commit it, then everyone has access to it immediately. It's free, close to the code and in context, everyone who needs access to it has it, and it's simple to use. It's also local, extremely fast and, if desired, can be easily embedded into a CI pipeline, compiled to anything you like (Markdown => *) and pushed to wherever you like.
I also don't understand the obsession with making documentation available online, 24/7, when we live in a cyber security nightmare right now (https://www.hertzbleed.com/; https://securityboulevard.com/2022/06/apple-m1-flaw-cant-be-...; https://www.darkreading.com/threat-intelligence/emotet-banki...; https://www.darkreading.com/edge-articles/turbulent-cyber-in....)
Just keep it within context, plain text, and easily convertible to other formats. Try not to over think things.
My code is not binary, however, it's plain text. So I write my documentation in plain text too, such as Markdown. That can, should I so choose, be rendered to anything I like. It can be included alongside the code, which the developer will need a copy of anyway, and be easily reviewed (with "mkdocs serve" for example).
You're already committing code to a central repository and working in a decentralised manner. If you put your documentation in "./docs/" (I recommend MkDocs + the Material Theme) and commit it, then everyone has access to it immediately. It's free, close to the code and in context, everyone who needs access to it has it, and it's simple to use. It's also local, extremely fast and, if desired, can be easily embedded into a CI pipeline, compiled to anything you like (Markdown => *) and pushed to wherever you like.
I also don't understand the obsession with making documentation available online, 24/7, when we live in a cyber security nightmare right now (https://www.hertzbleed.com/; https://securityboulevard.com/2022/06/apple-m1-flaw-cant-be-...; https://www.darkreading.com/threat-intelligence/emotet-banki...; https://www.darkreading.com/edge-articles/turbulent-cyber-in....)
Just keep it within context, plain text, and easily convertible to other formats. Try not to over think things.
We totally agree! We think it would be best when documentation is centralized, version controlled, and coupled with the code. Unfortunately, we realized that's just not the case for the vast majority of companies. One way to think about our product is just bringing the features of documentation on GitHub into existing documentation that lives outside of it.
A potentially valueable goal. I wish you luck.
> Unfortunately, we realized that's just not the case for the vast majority of companies.
Nor is DevOps, but that's changing. And as more things shift-left and get actioned in a CI/CD pipeline, companies are going to want more automation, checks, analysis, etc., of all their digital assets. That includes documentation - spell checks, auto-generated and static, checking for broken links, warnings about docs that haven't been reviewed or updated for X days, and more.
That's the real answer to this problem: how do we keep documentation close to the context to which it relates, AND allow it to be manipulated, tested, and more, all through automated business logic? Solve that.
> Unfortunately, we realized that's just not the case for the vast majority of companies.
Nor is DevOps, but that's changing. And as more things shift-left and get actioned in a CI/CD pipeline, companies are going to want more automation, checks, analysis, etc., of all their digital assets. That includes documentation - spell checks, auto-generated and static, checking for broken links, warnings about docs that haven't been reviewed or updated for X days, and more.
That's the real answer to this problem: how do we keep documentation close to the context to which it relates, AND allow it to be manipulated, tested, and more, all through automated business logic? Solve that.
> how do we keep documentation close to the context to which it relates, AND allow it to be manipulated, tested, and more, all through automated business logic? Solve that.
Well spoken.
Well spoken.
[deleted]
I think this is really just level 0 of what documentation is from everything I'm seeing. That sounds harsh! But what I mean additionally is that this is laying the necessary and important groundwork to increase the legitimacy of documentation efforts. Being a backbone/level 0 I would caution you to keep in mind incentives to try and avoid perverse incentives
Right now there are close to 0 incentives to either create or maintain documentation unless they're inherent in the team already. Or in the case of specific offerings business pressure (Example: API to external developers, but this also requires a business model that means lots of external developers that can't be interfaced with in a higher touch way).
The perverse incentives I can think of immediately which are by no means exhaustive: 1. Increased visibility means increased visibility by management: A double edged sword where management must pick the right metrics to incentivize the production of good documentation. And on the other hand, increased visibility(internally and externally) empowers them to buy into the value produced by documentation efforts and help their teams build documentation habits.
2. This system necessarily increases # of tasks to be completed by a team. Over-subscribing to new task notifications leads to notification blindness or even worse notification dread. I personally love batched tasks that are relevant and important. I do not like things that interrupt focus or even worse interrupt my focus with things that are irrelevant at the current point in time. Nailing this is one of many things to be nailed in order to make the difference between being happy with a tool and feeling overwhelmed.
1 and 2 above can be generalized a bit. With the additional tooling you build on top of your "backbone" you are setting cultural, management, implementation, and usage standards and habits!
So this leaves some core questions to inform the goals of this additional tooling to raise the profile of documentation. - What metrics should management focus on, how can they use this? Is it about keeping documentation updated as often as possible? Is it about producing documentation that the readers find informative? Is it about reducing errors in the documentation? Is it about X words per month? Is it about knowing which developers are the most likely best writers for a specific area of documentation? Does someone need help improving their documentation abilities? - Do documentation writers feel empowered rather than overloaded? Do they find value in the documentation that others are writing for them? (I.E. you notice them struggling implementing a feature, maybe there is relevant well written documentation to get them unstuck!) - And as readers of the documentation... it's not always just internal teams. What tasks are 3rd parties trying to solve with the documentation? Very granular documentation requires a "wide walk" of documentation. Very task focused documentation can mean missing additional information that allows the reader to generalize the task specific documentation to a broader use case. How are readers reading and engaging? I.E. They go to figure out how to do Task A and read section B on page 1, section A,C on page 3, section D on page 4. In this process they notice something slightly incorrect. Rather than just a thumbs down are they empowered to make changes or ask clarifying questions?
Edit: And to keep this from becoming much longer than it already is :) I have one final important question.
What is "bad" documentation and what is "good" documentation?
Right now there are close to 0 incentives to either create or maintain documentation unless they're inherent in the team already. Or in the case of specific offerings business pressure (Example: API to external developers, but this also requires a business model that means lots of external developers that can't be interfaced with in a higher touch way).
The perverse incentives I can think of immediately which are by no means exhaustive: 1. Increased visibility means increased visibility by management: A double edged sword where management must pick the right metrics to incentivize the production of good documentation. And on the other hand, increased visibility(internally and externally) empowers them to buy into the value produced by documentation efforts and help their teams build documentation habits.
2. This system necessarily increases # of tasks to be completed by a team. Over-subscribing to new task notifications leads to notification blindness or even worse notification dread. I personally love batched tasks that are relevant and important. I do not like things that interrupt focus or even worse interrupt my focus with things that are irrelevant at the current point in time. Nailing this is one of many things to be nailed in order to make the difference between being happy with a tool and feeling overwhelmed.
1 and 2 above can be generalized a bit. With the additional tooling you build on top of your "backbone" you are setting cultural, management, implementation, and usage standards and habits!
So this leaves some core questions to inform the goals of this additional tooling to raise the profile of documentation. - What metrics should management focus on, how can they use this? Is it about keeping documentation updated as often as possible? Is it about producing documentation that the readers find informative? Is it about reducing errors in the documentation? Is it about X words per month? Is it about knowing which developers are the most likely best writers for a specific area of documentation? Does someone need help improving their documentation abilities? - Do documentation writers feel empowered rather than overloaded? Do they find value in the documentation that others are writing for them? (I.E. you notice them struggling implementing a feature, maybe there is relevant well written documentation to get them unstuck!) - And as readers of the documentation... it's not always just internal teams. What tasks are 3rd parties trying to solve with the documentation? Very granular documentation requires a "wide walk" of documentation. Very task focused documentation can mean missing additional information that allows the reader to generalize the task specific documentation to a broader use case. How are readers reading and engaging? I.E. They go to figure out how to do Task A and read section B on page 1, section A,C on page 3, section D on page 4. In this process they notice something slightly incorrect. Rather than just a thumbs down are they empowered to make changes or ask clarifying questions?
Edit: And to keep this from becoming much longer than it already is :) I have one final important question.
What is "bad" documentation and what is "good" documentation?
Your point about incentivizing is very interesting. We mainly interviewed teams that were in our batch AKA early stage startups/small teams that didn't have much of a divide between management and developers - definitely a mistake on our part and we should definitely go back to interviewing larger teams. We knew that documentation is hard to maintain as changes are made and that a feature can be shipped without documentation having to be written/updated. We also agree that whatever our solution is - we need to be opinionated in terms of the "cultural, management, implementation, and usage standards and habits". I totally see where you're coming from about this being level 0 - at the very least we create visibility that something may or may not be updated. I guess the assumption we made was that a CI check would be enough (because the mentality is that you can't merge until you resolve everything) but there's definitely more incentives we can create.
Analytics about documentation is something we've talked about, but it's definitely something we should talk about more seriously.
> What is "bad" documentation and what is "good" documentation?
There are definitely varying levels of granularity I can go into answering this question. As a baseline I can definitely say that bad documentation is documentation that is out-of-date - a series of instructions that are not the correct steps to get to the end goal that it claims will get for you. And good documentation is one in which it does allow you to get to your end goal. However, there are definitely layers to it - because I know for myself I'm very opinionated on how verbose, clear, or straight-to-the-point documentation is. Outside of just the content, there's also a lot that surrounds the structure of documentation.
It's so clear that a problem exists here. We just have to figure out what the best solution looks like.
Analytics about documentation is something we've talked about, but it's definitely something we should talk about more seriously.
> What is "bad" documentation and what is "good" documentation?
There are definitely varying levels of granularity I can go into answering this question. As a baseline I can definitely say that bad documentation is documentation that is out-of-date - a series of instructions that are not the correct steps to get to the end goal that it claims will get for you. And good documentation is one in which it does allow you to get to your end goal. However, there are definitely layers to it - because I know for myself I'm very opinionated on how verbose, clear, or straight-to-the-point documentation is. Outside of just the content, there's also a lot that surrounds the structure of documentation.
It's so clear that a problem exists here. We just have to figure out what the best solution looks like.
I like the answers in the last part. I literally had an interview today where I quoted a professor of mine from about a decade ago. He was very upset with me about some incorrect types in my javadoc comments and told me "The only thing worse than no documentation is wrong documentation!" :)
For your answer on good documentation, that's a good general answer. As you say there are lots of layers to it. Some ideas just from my current viewpoint: - From my forays into foreign language learning I picked up the linguist's Stephen Krashen's research after learning about it from my language exchange partner. One of the main themes of his research is the idea of "Comprehensible Input"(CI). CI is the idea that you acquire knowledge via exposure to new language that is at a difficulty level where you understand most of what's being said. Then you acquire these new smaller pieces of knowledge via context. I would say this is most closely related to what you mean by structure of the documentation. Obviously with some overlap with the content itself. - There's also a stronger version of CI that adds an additional piece to this, the content must be of interest to the acquirer/learner. This property is I think the closest to what I think you mean in regards to content of documentation.
In my opinion reaching the first goal, Comprehensible Input, where knowledge is able to be acquired efficiently is step 1. Resources of this type can be created much more easily as there is less subjectivity about the content's style itself rather than simply the pacing of the concepts introduced. The second part is obviously the hard part :) Maybe style text style transfer will start to get really good without the possibility of it influencing the former?
[X] - Old video of Krashen explaining his central ideas https://www.youtube.com/watch?v=fnUc_W3xE1w
P.S. I specifically mentioned text style transfer because I noticed your team had an offer of bringing in trusted technical writers. What would be a very good deal for both you, doc writers(at the client side), doc consumers, and 3rd party writers is the ability for their style to be automatically transferred into anyone's technical writing. 3rd party writers can license their fingerprinted style, Client side writers get a neat assist on quickly changing their writing style, and readers get a choice of voice they identify with.
For your answer on good documentation, that's a good general answer. As you say there are lots of layers to it. Some ideas just from my current viewpoint: - From my forays into foreign language learning I picked up the linguist's Stephen Krashen's research after learning about it from my language exchange partner. One of the main themes of his research is the idea of "Comprehensible Input"(CI). CI is the idea that you acquire knowledge via exposure to new language that is at a difficulty level where you understand most of what's being said. Then you acquire these new smaller pieces of knowledge via context. I would say this is most closely related to what you mean by structure of the documentation. Obviously with some overlap with the content itself. - There's also a stronger version of CI that adds an additional piece to this, the content must be of interest to the acquirer/learner. This property is I think the closest to what I think you mean in regards to content of documentation.
In my opinion reaching the first goal, Comprehensible Input, where knowledge is able to be acquired efficiently is step 1. Resources of this type can be created much more easily as there is less subjectivity about the content's style itself rather than simply the pacing of the concepts introduced. The second part is obviously the hard part :) Maybe style text style transfer will start to get really good without the possibility of it influencing the former?
[X] - Old video of Krashen explaining his central ideas https://www.youtube.com/watch?v=fnUc_W3xE1w
P.S. I specifically mentioned text style transfer because I noticed your team had an offer of bringing in trusted technical writers. What would be a very good deal for both you, doc writers(at the client side), doc consumers, and 3rd party writers is the ability for their style to be automatically transferred into anyone's technical writing. 3rd party writers can license their fingerprinted style, Client side writers get a neat assist on quickly changing their writing style, and readers get a choice of voice they identify with.
We worked at software companies in all stages ranging from startups to big tech, and they all had bad documentation, if it even existed at all. We decided to work on this problem and created a VS Code extension called Doc Writer which generated in-line documentation for codebases using Codex from OpenAI. Doc Writer helped people document their code more frequently and still continues to, but there were limitations. We were highly reliant on OpenAI, people didn’t want their proprietary code to be sent into the cloud, and the AI was satisfactory 80% of the time. But after digging deeper into the documentation problem space, we realized that Doc Writer only solved a small part of it.
We quickly learned that (1) the debate about writing documentation vs. having self-documenting code was highly controversial, and (2) entire teams hated writing documentation—not only developers.
We proceeded to interview dozens of startups and learned that the hardest part about documentation is maintaining it. Everybody was developing so quickly that it was difficult for documentation to stay up-to-date. Common problems we heard were that documentation was inconveniently decentralized over multiple platforms, people weren’t aware when important documents changed, and when code changes the related documentation wouldn’t be updated.
The goal of Mintlify is to increase visibility over documentation across your entire team so that you can easily maintain it. Mintlify allows you to centralize your documentation into one searchable place; set up integrations to receive alerts when documents change; implement a CI Check for documentation - connect documentation to code and receive alerts to update your documentation when the code changes.
Other solutions to the problem of documentation maintenance tend to create an editor (e.g. Notion, ReadMe, Archbee, Gitbook). We decided instead to work with teams’ existing documentation stack, because of our belief that maintenance of documentation is the real hard problem in this space. Our software is designed to help documentation owners ensure that content stays in good condition.
Here is our 2 min demo: https://www.loom.com/share/892d08e178144cd89b109f9396e4db98, and you can also try it for yourself: https://www.mintlify.com/create
Ultimately, we want to create a suite of automations that makes maintaining high quality documentation easy. We plan on adding documentation owners and integrations with task management platforms so that tickets can be instantly generated prompting people to update documentation. We believe there is a market for this because of our experience with our earlier Doc Writer product, and because companies like Glean, Gitbook, and Whatfix are all tackling this problem in their own way.
We thank the open source community, our community, and our users for having helped us shape the product to what it is now. We look forward to your feedback, questions, ideas, and suggestions!