Ask HN: Working with large code base for the first time
I am working on on a very large codebase for the first time,mainly on the front end code.
Needs some tips on how to trace down the code base effectively?
89 comments
1. Get confortable using “grep” [1], or better, “ripgrep” [2], which is quite faster than the former. They are both available in Linux, macOS, and Windows via WSL.
2. If the project uses a version control system (Git, Mercurial, Subversion, etc.) then take a look at the most recent additions, modifications, and/or deletions in the version control log (git-log, or whatever you want to call it). Sometimes, the most relevant files in a project are the ones people modify the most… obviously, ignore files associated to third-party dependencies (vendor, node_modules, that kind of stuff).
3. Install a Language Server Protocol (LSP) server [3] with support for the programming language(s) that you are going to use. Configure your favourite code editor to take advantage of as many LSP features as possible, with enphasis on “Jump To Definition” and “Find References” [4].
Tell us what programming language(s) is the project written in to give you more suggestions.
[1] https://en.wikipedia.org/wiki/Grep
[2] https://github.com/BurntSushi/ripgrep
[3] https://microsoft.github.io/language-server-protocol/impleme...
[4] https://langserver.org/
2. If the project uses a version control system (Git, Mercurial, Subversion, etc.) then take a look at the most recent additions, modifications, and/or deletions in the version control log (git-log, or whatever you want to call it). Sometimes, the most relevant files in a project are the ones people modify the most… obviously, ignore files associated to third-party dependencies (vendor, node_modules, that kind of stuff).
3. Install a Language Server Protocol (LSP) server [3] with support for the programming language(s) that you are going to use. Configure your favourite code editor to take advantage of as many LSP features as possible, with enphasis on “Jump To Definition” and “Find References” [4].
Tell us what programming language(s) is the project written in to give you more suggestions.
[1] https://en.wikipedia.org/wiki/Grep
[2] https://github.com/BurntSushi/ripgrep
[3] https://microsoft.github.io/language-server-protocol/impleme...
[4] https://langserver.org/
2 and 3 are great points, but honestly - and especially if you're working in multiple repos, or multiple subdirectories in a monorepo - VS code's folder search is better than grep. It'll find you the exact place in the file, and a click will take you there, nice and easy. I've more or less forgotten the quirks/flags of grep, because I haven't needed to use it in a while.
VSCode uses ripgrep under the hood [1]
[1] https://github.com/microsoft/vscode-ripgrep
[1] https://github.com/microsoft/vscode-ripgrep
Almost the same can be achieved in vim even without plugins
And just like that, the file is open in yet another tab. The best part? Since the grep output is just another vim buffer, I can search it as well, just like any other buffer.
And yes, ofc the above can be put into a custom command, just by putting this in the .vimrc:
:tabnew | r ! shopt -o globstar && grep -sn STRING **
C-w gf
And just like that, the file is open in yet another tab. The best part? Since the grep output is just another vim buffer, I can search it as well, just like any other buffer.
And yes, ofc the above can be put into a custom command, just by putting this in the .vimrc:
command -nargs=1 Gr :tabnew | r ! shopt -s globstar && grep -sn <args> **
:Gr STRINGWhat if I told you VS code search is ripgrep? Pretty much every coding focused editor has built-in ability to view grep results, then click on the result to jump to the file location. It's 1980's UI technology.
You can also use `git grep` if you're using git, and I'm sure analogues exist for the other VCSs.
+1 for git grep.
To get a sorted list of the most touched java files for example you can run
git log --name-status --pretty=format: | sed 's!.*/!!' | grep java | cut --fields=2- | sort | uniq --count | sort --numeric-sort --reverse
git log --name-status --pretty=format: | sed 's!.*/!!' | grep java | cut --fields=2- | sort | uniq --count | sort --numeric-sort --reverse
I should add `tig` for navigating git commit history. Very useful sometimes when working with many people on a git repo
Tangential wish - I'd love it if someone figured out a good/generally working way of jumping to definition/references through a URL.
E.g. from `axios.get(`/users/${currentUserId}`) in frontend code jump to `class UserSingleView: def get(self, user_id):` in the backend, and vice versa.
E.g. from `axios.get(`/users/${currentUserId}`) in frontend code jump to `class UserSingleView: def get(self, user_id):` in the backend, and vice versa.
As someone who has worked for years on a UI consuming such information: it's bloody difficult. Each language has its own weirdnesses, you almost always have to build the code to figure out the cross references, and what's correct to the compiler is often non-intuitive to the user. Prime example: C++ code mixing macros and templates. There isn't even a good definition of what the definition is!
I probably do underestimate its difficulty, but certainly appreciate it's bloody difficult. Hence I'd love someone else to do it! I don't have the appetite for it, but it would be fantastic to use.
That is such an annoying thing in codebases. It's the same issue the other way around, I want to see all places in frontend that calls /v1/resource but then some people go and abstract the url into multiple variables.
If it was up to me the constant part of the relative url would never be abstracted away in the codebase. Just a simple grep would instantly get you everything you are looking for.
If it was up to me the constant part of the relative url would never be abstracted away in the codebase. Just a simple grep would instantly get you everything you are looking for.
For IDEs, CTRL+SHIFT+F can be a good replacement for grep. There are similar shortcuts for Jump to Definition and stuff without installation needed.
myself am partial to silver searcher (brew install ag)
ripgrep is a faster and newer alternative with many ergonomic improvements.
The lineage is grep (C, 1973) -> ack (Perl, ~2006) -> ag (C, ~2011) -> rg (Rust, ~2016)
Ack inspired a wave of tool rewrites to accommodate better interfaces and usability.
The lineage is grep (C, 1973) -> ack (Perl, ~2006) -> ag (C, ~2011) -> rg (Rust, ~2016)
Ack inspired a wave of tool rewrites to accommodate better interfaces and usability.
> Get confortable using “grep” [1], or better, “ripgrep” [2]
And as part of getting comfortable, the -A and -C options are particularly useful.
And as part of getting comfortable, the -A and -C options are particularly useful.
- Search for unique-looking strings from whatever view you're trying to modify to find the associated html/js file.
- Pull up the commit history for the file to see what other files were modified along with it the last few times. This will give you dependencies and linkages.
- Make your change and then ask your ide/command line to find all typing/lint errors in your project which will help you find other dependencies you may have missed.
- If you get stuck, reach out to the authors or reviewers of previous PRs. (Hint: you may want to include them as reviewers. They'll give good feedback and you'll engender good will by keeping them in the loop when you touch their corner of the codebase)
- Write a few solid unit tests. Maybe even clean up the testing code a little bit while you're there.
- Write a concise but informative description of your changes in your PR. If you made two or more logical changes, split your PR and stack them. Your teammates will appreciate the shorter PRs and you will get feedback more quickly.
- Land the PR in a timely manner and keep an eye on it until it hits prod.
- Once in prod, test it yourself and keep an eye on the logs for a day or three.
- Bonus: put all changes behind feature flags to do slow rollouts and so you can quickly revert without waiting for a deploy. Make a task to remind yourself to remove the dead code behind the flag in month or so when you're pretty sure it's stable.
- Pull up the commit history for the file to see what other files were modified along with it the last few times. This will give you dependencies and linkages.
- Make your change and then ask your ide/command line to find all typing/lint errors in your project which will help you find other dependencies you may have missed.
- If you get stuck, reach out to the authors or reviewers of previous PRs. (Hint: you may want to include them as reviewers. They'll give good feedback and you'll engender good will by keeping them in the loop when you touch their corner of the codebase)
- Write a few solid unit tests. Maybe even clean up the testing code a little bit while you're there.
- Write a concise but informative description of your changes in your PR. If you made two or more logical changes, split your PR and stack them. Your teammates will appreciate the shorter PRs and you will get feedback more quickly.
- Land the PR in a timely manner and keep an eye on it until it hits prod.
- Once in prod, test it yourself and keep an eye on the logs for a day or three.
- Bonus: put all changes behind feature flags to do slow rollouts and so you can quickly revert without waiting for a deploy. Make a task to remind yourself to remove the dead code behind the flag in month or so when you're pretty sure it's stable.
Thanks, didn't realize that you could pull commit history for a file.
Git blame is also super helpful when reading individual files, as it gives you a line by line commit history.
I recently used it in a major re factor of a file (I didn't write) as it gave me background info (commit messages) on why the code ended up looking like that.
I recently used it in a major re factor of a file (I didn't write) as it gave me background info (commit messages) on why the code ended up looking like that.
I'm a firm believer that the onus should be on the current development team to explain this to you. That is unlikely IME to happen without you asking.
What happens next will be an iterative process.
Do some initial investigation - don't waste time figuring things out, we just want a set of questions and areas to explain at this stage.
Ask for some help from the team - perhaps 30-60 mins, perhaps with different people to cover off the topics. It's essential that the team contribute - firstly they owe it to you, secondly only they know the background. Which bits of the codebase were experimental, which bits are ancient relics being removed, what's the direction of travel, which parts have subtleties. "Unknown unknowns" (to you).
Now "pay it forward". Ensure that the next person to join the team has a better time. Maybe this is writing some documentation - be it architectural, a glossary of domain-specific terms, updating/culling outdated docs. Maybe parts of the codebase could do with some re-organising or renaming to make it more self-evident. Perhaps the build/test process could do with some care...
I've done this several times, and after 3-4 hour sessions with "the one person who understands" (and a few hours writeup), we now have good documentation, better standards and multiple people including new joiners who understand. The guru also appreciates having more people to bounce ideas around with and no longer being overloaded with this work.
What happens next will be an iterative process.
Do some initial investigation - don't waste time figuring things out, we just want a set of questions and areas to explain at this stage.
Ask for some help from the team - perhaps 30-60 mins, perhaps with different people to cover off the topics. It's essential that the team contribute - firstly they owe it to you, secondly only they know the background. Which bits of the codebase were experimental, which bits are ancient relics being removed, what's the direction of travel, which parts have subtleties. "Unknown unknowns" (to you).
Now "pay it forward". Ensure that the next person to join the team has a better time. Maybe this is writing some documentation - be it architectural, a glossary of domain-specific terms, updating/culling outdated docs. Maybe parts of the codebase could do with some re-organising or renaming to make it more self-evident. Perhaps the build/test process could do with some care...
I've done this several times, and after 3-4 hour sessions with "the one person who understands" (and a few hours writeup), we now have good documentation, better standards and multiple people including new joiners who understand. The guru also appreciates having more people to bounce ideas around with and no longer being overloaded with this work.
And when someone teaches you a sequence of commands that are ~always run together, consider writing a script. Check the script into the repo. Now you have executable documentation.
(Obviously within reason: maybe `npm install && npm run start` doesn’t need to be in a script, but maybe two or three commands that print out some token which needs to be copied into the next steps would benefit).
Depending on the nature of the script, maybe even write a test. Now you have machine checked documentation.
(Obviously within reason: maybe `npm install && npm run start` doesn’t need to be in a script, but maybe two or three commands that print out some token which needs to be copied into the next steps would benefit).
Depending on the nature of the script, maybe even write a test. Now you have machine checked documentation.
Ask, ask, ask. Yes, you can spend days or weeks poring over the code in various ways, but asking those who made it (ideally) or those who maintain it will give you the "why". Why was it done this way? What are the implicit assumptions and invariants? Why were seemingly obvious ideas not implemented? Or were they and found problematic? Document these findings for the next generation.
But as you do this, keep an eye out for assumptions that may have changed. A feature now obsolete requiring weird code. Out-of-date assumptions about the behaviour of computers or other systems. New language features that can simplify or improve code. Talk them over with the people who know the code, and maybe you'll be the one to delete that awful code everyone hated.
Also, take notes, not just about the code, but about its environment, release process, surrounding systems, use cases, and people. Knowing who to ask about a given issue is gold.
If there are post-mortems available, they can give a great insight into how the system works and fails. Design docs to a certain extent, too, but they can be misleading especially if they are not kept up to date.
Pair programming can be a very effective way of learning, too.
But as you do this, keep an eye out for assumptions that may have changed. A feature now obsolete requiring weird code. Out-of-date assumptions about the behaviour of computers or other systems. New language features that can simplify or improve code. Talk them over with the people who know the code, and maybe you'll be the one to delete that awful code everyone hated.
Also, take notes, not just about the code, but about its environment, release process, surrounding systems, use cases, and people. Knowing who to ask about a given issue is gold.
If there are post-mortems available, they can give a great insight into how the system works and fails. Design docs to a certain extent, too, but they can be misleading especially if they are not kept up to date.
Pair programming can be a very effective way of learning, too.
Realistically, i agree with your point and it's probably a good course of action!
> Ask, ask, ask. Yes, you can spend days or weeks poring over the code in various ways, but asking those who made it (ideally) or those who maintain it will give you the "why".
My question, however, is why aren't these things documented in the first place? In DevTernity (a recent software development conference), the concept of ADRs (https://adr.github.io/) was discussed. To me, it made a lot of sense: the code is the implementation, whereas a bunch of Markdown files can explain the "why" behind it, as well as all of the stuff that wouldn't appear in Jira or regular code comments or wherever.
After all, not every senior dev has that many hours to spend onboarding new team members, nor are their memories also that good to remember everything.
> Ask, ask, ask. Yes, you can spend days or weeks poring over the code in various ways, but asking those who made it (ideally) or those who maintain it will give you the "why".
My question, however, is why aren't these things documented in the first place? In DevTernity (a recent software development conference), the concept of ADRs (https://adr.github.io/) was discussed. To me, it made a lot of sense: the code is the implementation, whereas a bunch of Markdown files can explain the "why" behind it, as well as all of the stuff that wouldn't appear in Jira or regular code comments or wherever.
After all, not every senior dev has that many hours to spend onboarding new team members, nor are their memories also that good to remember everything.
In my experience, many things are documented, but the documentation is unhelpful, redundant or outdated. Sometimes there are several documents on the same thing, some are outdated in different ways, or the author has given up on it.
In code itself, often you see comments that say one thing, the function name saying another, and the relationship doing something different.
It's probably good company culture for seniors to just accept that the new guy will be a productivity sink for a while. Get them up to par quickly so they'll be productive earlier. Knowledge is like compound interest, it's easier to start them out with a lot and then have them figure it out rather than have them figure it out from the start and ask if stuck.
In code itself, often you see comments that say one thing, the function name saying another, and the relationship doing something different.
It's probably good company culture for seniors to just accept that the new guy will be a productivity sink for a while. Get them up to par quickly so they'll be productive earlier. Knowledge is like compound interest, it's easier to start them out with a lot and then have them figure it out rather than have them figure it out from the start and ask if stuck.
A neat trick is to have the new person update the docs whenever they needed to ask a question.
Writing docs falls in the realm of error handling and testing. Lots of developers only want to deal with the happy path not those extra drags. Standard CS education isn't helping here.
Writing docs falls in the realm of error handling and testing. Lots of developers only want to deal with the happy path not those extra drags. Standard CS education isn't helping here.
> In my experience, many things are documented, but the documentation is unhelpful, redundant or outdated. Sometimes there are several documents on the same thing, some are outdated in different ways, or the author has given up on it.
> In code itself, often you see comments that say one thing, the function name saying another, and the relationship doing something different.
Is that not a red flag in of itself? Because it seems like at some point someone cared enough to write the docs, but eventually no one cared much to maintain them.
To me, that just expresses that later either the other developers or even that same dev cared only about keeping things running, which was the standard of quality that they chose for themselves, as opposed to actually caring about the long term sustainability and therefore the docs.
I've seen situations like that as well, but generally when everything surrounding the projects is also bad - improper risk management, old dependencies possibly with insecure code, unaddressed technical debt and either god objects or overly long functions with outdated names/docs all go hand in hand.
Personally, i don't take it as "just the way things are" but rather something to be addressed head on - in the past year, i've written everything from onboarding guides to initialization scripts and tooling for around 5-10 projects and have kept them up to date.
Furthermore, i've definitely sent back merge/pull requests on the basis that they need further refinement due to the aforementioned concerns. Not nitpicky stuff that's irrelevant, but rather things that would make the codebase lie to someone reading it, in one way or another.
Of course, one can argue that docs are useless in of themselves because they're not what's actually running on the servers, but personally i disagree with that point of view as well, though perhaps that's a conversation for another day.
> In code itself, often you see comments that say one thing, the function name saying another, and the relationship doing something different.
Is that not a red flag in of itself? Because it seems like at some point someone cared enough to write the docs, but eventually no one cared much to maintain them.
To me, that just expresses that later either the other developers or even that same dev cared only about keeping things running, which was the standard of quality that they chose for themselves, as opposed to actually caring about the long term sustainability and therefore the docs.
I've seen situations like that as well, but generally when everything surrounding the projects is also bad - improper risk management, old dependencies possibly with insecure code, unaddressed technical debt and either god objects or overly long functions with outdated names/docs all go hand in hand.
Personally, i don't take it as "just the way things are" but rather something to be addressed head on - in the past year, i've written everything from onboarding guides to initialization scripts and tooling for around 5-10 projects and have kept them up to date.
Furthermore, i've definitely sent back merge/pull requests on the basis that they need further refinement due to the aforementioned concerns. Not nitpicky stuff that's irrelevant, but rather things that would make the codebase lie to someone reading it, in one way or another.
Of course, one can argue that docs are useless in of themselves because they're not what's actually running on the servers, but personally i disagree with that point of view as well, though perhaps that's a conversation for another day.
- Before starting to dive into the code, take a look at it from the users point of view. Make sure you know what the code does by reading manuals and playing around with it.
- Don't try to learn domain knowledge from the code. I you need to get familiar with code of a JPEG decoder, learn how JPEG works first.
- Before reading code, make sure you can jump to definition and find references. The easiest and language agnostic way is to use ctags and grep.
- Start reading code from main() (or an API call in the case of a library) and then start to dig deeper. This way will get a feeling where the "important" code is"
- Don't try to learn domain knowledge from the code. I you need to get familiar with code of a JPEG decoder, learn how JPEG works first.
- Before reading code, make sure you can jump to definition and find references. The easiest and language agnostic way is to use ctags and grep.
- Start reading code from main() (or an API call in the case of a library) and then start to dig deeper. This way will get a feeling where the "important" code is"
Shadow a more experienced engineer and ask them lots of questions.
One of my favorites is “how did you know to do that?”
One of my favorites is “how did you know to do that?”
That's the most relevant comment. Analyzing code by reading is probably the worse way to understand it. Using preprocessed information in the form of a human being* is much faster and less error prone...
I like to trace down things in order.
For example, let say I'm working on an ecommerce system and I try to understand what happens when a buyer adds an item to its cart. I'm going to put logs / debugger's prompts on the important steps of that operation. Next to those, add a comment that describes why this step is important. The important thing is to label the comment with incrementing numbers. Those numbers allow me to keep track of the order of execution.
Finally, commit this to a dummy branch and share it with your co-workers if both of you are new to this code base.
That's one of the first thing I do when I jump in a new codebase. Pick something that interests you, and log the whole operation.
For example, let say I'm working on an ecommerce system and I try to understand what happens when a buyer adds an item to its cart. I'm going to put logs / debugger's prompts on the important steps of that operation. Next to those, add a comment that describes why this step is important. The important thing is to label the comment with incrementing numbers. Those numbers allow me to keep track of the order of execution.
Finally, commit this to a dummy branch and share it with your co-workers if both of you are new to this code base.
That's one of the first thing I do when I jump in a new codebase. Pick something that interests you, and log the whole operation.
If the project you are working on has enough time to burn, and the codebase has no (or near to) unit tests, add them.
Find large utilities that are not coupled with a particular part of the application, and put them under heavy testing. Do not fix any failing test before having completed the test suite.
If you have access to other developers that know the codebase, review the tests with them, and fix failing tests together.
Do the same for the frontend / interface code, but do not go for unit testing imho, go for visual regression testing.
And the end of the process you will have a very large knowledge of the codebase, and you will have improved it at the same time.
Find large utilities that are not coupled with a particular part of the application, and put them under heavy testing. Do not fix any failing test before having completed the test suite.
If you have access to other developers that know the codebase, review the tests with them, and fix failing tests together.
Do the same for the frontend / interface code, but do not go for unit testing imho, go for visual regression testing.
And the end of the process you will have a very large knowledge of the codebase, and you will have improved it at the same time.
How are you supposed to write tests for code if you don't understand what it does/is supposed to do?
You can create regression tests that prove that the code does what it does today - which helps you understand it, provides a readable description of what it does and protects against regression bugs in the future.
Here's a relevant trick I frequently use for this kind of work with Python: https://simonwillison.net/2020/Feb/11/cheating-at-unit-tests...
Here's a relevant trick I frequently use for this kind of work with Python: https://simonwillison.net/2020/Feb/11/cheating-at-unit-tests...
First of all you’re not going to be all by yourself. Ask previous developers or peers if there are any. Ask them to help you navigate the codebase and find good candidates.
On the opposite scenario, where you have to take ownership of a large codebase and there’s no one around that can guide you through it, if every time you learn something about it you turn it into a test case you will do your future self a favor and at the same time also tracing your progress.
On the opposite scenario, where you have to take ownership of a large codebase and there’s no one around that can guide you through it, if every time you learn something about it you turn it into a test case you will do your future self a favor and at the same time also tracing your progress.
Maybe I missed it but I don't see anyone mentioning Git blame. It is great tool to see how recently some part of the code you are working on were changed and by who, so you know who to ask. Also good to know what kinds of commits happens in your part of the code (if it is a good commit it probably describes what problem they were trying to solve). Hook it up to your IDE/editor (guessing it works with most of them, I have it hooked up with my vim) so you can see which lines changed in which commits.
You should be able to break down a front end project by documenting of all the pages and then components they depend on. Then document their flow (which pages link to which and what dialogs / components they open). Then you need to look at how data is managed among all the pages and components. Do they use something like redux for data management? Then how is the business logic is composed and how backend API's are used. Also for a front end project how are styles managed through out the application.
This sounds like some very good suggestions.
It's going to take time, for large code bases you won't actually look at all the code.
Make sure you're able to efficiently navigate through the code. If your IDE supports jumping to declarations and usages, make sure you get working.
Look at old commits to see how changes were added before, that will hint at how future changes should look.
Ask others for help if something doesn't make sense.
Make sure you're able to efficiently navigate through the code. If your IDE supports jumping to declarations and usages, make sure you get working.
Look at old commits to see how changes were added before, that will hint at how future changes should look.
Ask others for help if something doesn't make sense.
What you need, more than anything, is the mental model the authors used when writing it. In a company you want to sit down with the people who wrote it. If it's open source you'll have to do the same if the API is undocumented. Otherwise it's something like unbaking a cake. I've done it myself by first figuring out the dependency tree of the modules, then trying to get a high-level idea of the purpose of each component. From there, it should become easier. If the code is written with any sense at all it will get lower-level and less abstract as you work your way down to the function calls.
These are not rote advice, it’s literally the parts that have been successful as I’ve inherited maintainership of several large projects.
If it’s not already TypeScript, add a tsconfig.json, with allowJs: true, checkJs: true. You’ll probably need some tweaks from there. But merely adding the config is enough to kick VSCode or other language service providers into finding references and a lot more.
Apart from that:
- Keep notes as you uncover how things flow and interact.
- The best way to keep those notes is as part of your project’s documentation. Use JSDoc and TypeScript declaration files. Learn how these docs interact and aid navigation.
- @see is a particularly useful JSDOC tag!
- Speed up the build if it’s slow. No really, it’s going to help with discoverability in the codebase! Breaking flow to wait for a build is a sure way to lose track of your journey.
- Backfill tests before you touch implementation. Even if you suspect the tests might be redundant. Odds are they’re not, but even if they are, (1) that’s more documentation you can reference and (2) any other maintainers/contributors seeing redundancy can use that as a signal to shortcut your familiarization.
- Git blame is your friend. Some days it’s your best friend.
- Look in weird places! Sometimes that old version of a dependency is pinned for a reason. Sometimes it’s pinned to a fork, again for a reason.
If it’s not already TypeScript, add a tsconfig.json, with allowJs: true, checkJs: true. You’ll probably need some tweaks from there. But merely adding the config is enough to kick VSCode or other language service providers into finding references and a lot more.
Apart from that:
- Keep notes as you uncover how things flow and interact.
- The best way to keep those notes is as part of your project’s documentation. Use JSDoc and TypeScript declaration files. Learn how these docs interact and aid navigation.
- @see is a particularly useful JSDOC tag!
- Speed up the build if it’s slow. No really, it’s going to help with discoverability in the codebase! Breaking flow to wait for a build is a sure way to lose track of your journey.
- Backfill tests before you touch implementation. Even if you suspect the tests might be redundant. Odds are they’re not, but even if they are, (1) that’s more documentation you can reference and (2) any other maintainers/contributors seeing redundancy can use that as a signal to shortcut your familiarization.
- Git blame is your friend. Some days it’s your best friend.
- Look in weird places! Sometimes that old version of a dependency is pinned for a reason. Sometimes it’s pinned to a fork, again for a reason.
Asking the team is something to be done with care. Yes, you should ask them for help, but ask them for holistic help. Don't go to a team member for every piece of code you need to find - ask them about the overall structure so you understand how things are organized, then find the details on your own. Ask "why?", not "how?"
It is a fine line between asking for help coming up to speed on a codebase, and asking for people to hold your hands while you do your job. Your team will help show you where they draw that line (each team is different.) But be aware of it because I have seen people fired for landing on the wrong side of the line.
I'm not saying to be afraid to ask for help - I'm saying to be sure to focus that help on learning and understanding, not trivia.
FWIW, I'm fully on board with all the answers that say, in short, "Search"
It is a fine line between asking for help coming up to speed on a codebase, and asking for people to hold your hands while you do your job. Your team will help show you where they draw that line (each team is different.) But be aware of it because I have seen people fired for landing on the wrong side of the line.
I'm not saying to be afraid to ask for help - I'm saying to be sure to focus that help on learning and understanding, not trivia.
FWIW, I'm fully on board with all the answers that say, in short, "Search"
Debugger is your friend :)
Start by adding breakpoints for some key actions of the app. Then step through the flows with the debugger. First pass you can just step over functions to get the high-level idea. In subsequent passes, you can step into functions that seem important. Rinse and repeat until you understand those actions well. Then move on to other areas in the app.
This works because you can see the actual end-to-end execution flow (not always clear from reading code), inspect runtime data (impossible by reading code) and even change the runtime data (variables, DOM) to validate assumptions about how the code works.
Start by adding breakpoints for some key actions of the app. Then step through the flows with the debugger. First pass you can just step over functions to get the high-level idea. In subsequent passes, you can step into functions that seem important. Rinse and repeat until you understand those actions well. Then move on to other areas in the app.
This works because you can see the actual end-to-end execution flow (not always clear from reading code), inspect runtime data (impossible by reading code) and even change the runtime data (variables, DOM) to validate assumptions about how the code works.
Find out how to run unit tests in your project and then find interesting unit tests to run. Set break points preferably at some launch points of your code and debug through unit tests. The process will take you through relevant parts of the code, the decisions being made etc.
This method has also worked for me when building features in a large codebase. Write a unit test first and then keep checking where the code breaks and fix those until your test succeeds. This is effectively TDD. Note that you might have to refactor the code for better design but it gets you started towards understanding the flow.
This method has also worked for me when building features in a large codebase. Write a unit test first and then keep checking where the code breaks and fix those until your test succeeds. This is effectively TDD. Note that you might have to refactor the code for better design but it gets you started towards understanding the flow.
This is an incredibly broad question, and the answers will probably be all over the place. If you are working on the front end, try to find the entry points and work from there through various hypothetical scenarios. Hopefully there is documentation on various parts of the system, or architects/devs who built it, but that is unfortunately not always the case…
Find out the deployment process. Find out what to do if anything goes really, really wrong. Try to find patterns in the code. Just explore for a while and even try to create you own mental model of how things are structured.
Find out the deployment process. Find out what to do if anything goes really, really wrong. Try to find patterns in the code. Just explore for a while and even try to create you own mental model of how things are structured.
For classic backend server software, for years I found the best way in was to look at the database schema, to understand those relationships, and then to see how the software itself was segmented (or not) in its relationship with the different parts of the database. If it was easy to understand the segments, and their relationship to the database, then I could assume that the software had good module interfaces with clean separation, but if every module was a mix using data from the same database tables, then I assumed I was dealing with a problematic code base that lacked good clean interfaces.
When dealing with frontend code, my attitude remained the same, but instead of looking at the database, I'm typically looking at the API. The goal is to see how data flows through the app. That is certainly one way, for understanding the code. This used to be easier, with classic RESTful APIs, I find this approach is a bit more difficult when using GraphQL.
Also, just to state the obvious, run the unit tests. If the code base lacks good test coverage, ask if you can start there, writing more tests -- that will give you a purpose and a structured way of diving into the code. I also suggest that you deliberately break things, and then see which tests fail -- if a test fails and you weren't expecting it to fail, then you just discovered a linkage in the code that you didn't think would be there.
When dealing with frontend code, my attitude remained the same, but instead of looking at the database, I'm typically looking at the API. The goal is to see how data flows through the app. That is certainly one way, for understanding the code. This used to be easier, with classic RESTful APIs, I find this approach is a bit more difficult when using GraphQL.
Also, just to state the obvious, run the unit tests. If the code base lacks good test coverage, ask if you can start there, writing more tests -- that will give you a purpose and a structured way of diving into the code. I also suggest that you deliberately break things, and then see which tests fail -- if a test fails and you weren't expecting it to fail, then you just discovered a linkage in the code that you didn't think would be there.
A codebase has a time dimension, and utilizing this effectively is critical.
Not sure when a bug was introduced? Write a test then bisect (if you can't automatically bisect frontend code you have other problems, but manual bisects work in a pinch).
Not sure why something was written the way it was? Hey sometimes this stuff just evolves naturally, try looking back through blame history to see how different commits effected the code.
Not sure what needs touching to add a new feature? Try seeing if you can find a similar feature that someone else wrote in the past.
Not sure when a bug was introduced? Write a test then bisect (if you can't automatically bisect frontend code you have other problems, but manual bisects work in a pinch).
Not sure why something was written the way it was? Hey sometimes this stuff just evolves naturally, try looking back through blame history to see how different commits effected the code.
Not sure what needs touching to add a new feature? Try seeing if you can find a similar feature that someone else wrote in the past.
The architectural overview, either from the docs or from someone who can tell you, which gives you the 'lay of the land' and a rough idea of what does what. Like having a map.
Then, the custom idioms and weirdness that develops in every project, often it's a set of patterns that the team just 'acquires' - often it's in every file, every function. It's like learning 'words' specific to the language they have created for themselves.
Then get someone to explain the build systems, tooling etc..
And then, just of the sake of it - make sure to build it. Make a change and build it again. I think there's a weird leap in subconscious confidence that comes along with 'building it'. Like riding a wild horse for the first time, if you can do it for one second, you can do it for longer.
Once you have you head wrapped around the system, and the 'systematic things' (like idioms) and can navigate the tools ... then it a matter of breaking thing down into details. Which is where the work is of course.
But you need a 'map' and a 'cart' and your 'hammer' before you can start waltzing around Campus thinking about fixing things.
Then, the custom idioms and weirdness that develops in every project, often it's a set of patterns that the team just 'acquires' - often it's in every file, every function. It's like learning 'words' specific to the language they have created for themselves.
Then get someone to explain the build systems, tooling etc..
And then, just of the sake of it - make sure to build it. Make a change and build it again. I think there's a weird leap in subconscious confidence that comes along with 'building it'. Like riding a wild horse for the first time, if you can do it for one second, you can do it for longer.
Once you have you head wrapped around the system, and the 'systematic things' (like idioms) and can navigate the tools ... then it a matter of breaking thing down into details. Which is where the work is of course.
But you need a 'map' and a 'cart' and your 'hammer' before you can start waltzing around Campus thinking about fixing things.
Logging to console, to build a mental model of the codepaths data and events take.
Git grep (or ripgrep) to find usage - useful when refactoring and to see how data is used/accessed and where it's passed around. Also useful to note where certain data doesn't show up: you can infer some structure from this.
Looking at when something was last changed with git blame can be useful. Is something suddenly broken, but hasn't been changed in 5 years? Could give an indication of where not to look on a first pass.
Break some things (locally) on purpose. Get a feel for how errors bubble up through the application and how dependant code behaves when something is wrong.
Look over the last handful of PRs/merged patches. It can be helpful to see these smaller pieces of code, the changeset, and their associated context - whether it was a feature, a bugfix, and what the code was supposed to achieve.
Use existing code in the codebase as a styleguide. Most work on large codebases isn't groundbreaking or innovative, so you're likely to find existing code similar to what you're currently trying to achieve that you can use to guide you.
If possible, make use of code reviews with colleagues.
Git grep (or ripgrep) to find usage - useful when refactoring and to see how data is used/accessed and where it's passed around. Also useful to note where certain data doesn't show up: you can infer some structure from this.
Looking at when something was last changed with git blame can be useful. Is something suddenly broken, but hasn't been changed in 5 years? Could give an indication of where not to look on a first pass.
Break some things (locally) on purpose. Get a feel for how errors bubble up through the application and how dependant code behaves when something is wrong.
Look over the last handful of PRs/merged patches. It can be helpful to see these smaller pieces of code, the changeset, and their associated context - whether it was a feature, a bugfix, and what the code was supposed to achieve.
Use existing code in the codebase as a styleguide. Most work on large codebases isn't groundbreaking or innovative, so you're likely to find existing code similar to what you're currently trying to achieve that you can use to guide you.
If possible, make use of code reviews with colleagues.
- Definitely use an IDE like IntelliJ
- Familiarizing yourself with vim is great and one of the best timesaving skills I decided to pick up randomly. You can install vim plugins for most IDEs
- Pick a small feature that you are curious how it works, and focus entirely on how it was implemented. The simpler the better really. Use git history to see the commit that the feature was introduced, and look how that engineer implemented it. It's quite easier for me to learn how something works by just focusing on seeing the Pull Request for the implementation. Just focus on looking how various things were implemented, and you'll (hopefully) see a pattern in code practices & design patterns.
- If you have questions, you should feel free to ask other engineers that work on the project. I was in a pretty senior position at my last job and we had quite a large codebase, and I personally never minded helping out junior and new engineers ever when they were coming up to speed on a new codebase. I always took the viewpoint that it's always in my best interests to help out the newer team members get up to speed quicker and have a better understanding of the product so they are able to effectively contribute. Hopefully engineers on the project you are working on think similarly.
- Absolutely learn how to perform proper debugging if you are not familiar already. Learn how to hook the browser to your IDE, how to troubleshoot things on the backend if you get to the point where you're doing fullstack work. Effective debugging is an incredibly important skill to pick up.
- Learn how to use IDE / other editor tooling such as jump to definition, refactor, inspect, find all occurrences. If this is a react project learn how to use browser extensions like devtools to help with debugging and understanding document structure.
- Familiarizing yourself with vim is great and one of the best timesaving skills I decided to pick up randomly. You can install vim plugins for most IDEs
- Pick a small feature that you are curious how it works, and focus entirely on how it was implemented. The simpler the better really. Use git history to see the commit that the feature was introduced, and look how that engineer implemented it. It's quite easier for me to learn how something works by just focusing on seeing the Pull Request for the implementation. Just focus on looking how various things were implemented, and you'll (hopefully) see a pattern in code practices & design patterns.
- If you have questions, you should feel free to ask other engineers that work on the project. I was in a pretty senior position at my last job and we had quite a large codebase, and I personally never minded helping out junior and new engineers ever when they were coming up to speed on a new codebase. I always took the viewpoint that it's always in my best interests to help out the newer team members get up to speed quicker and have a better understanding of the product so they are able to effectively contribute. Hopefully engineers on the project you are working on think similarly.
- Absolutely learn how to perform proper debugging if you are not familiar already. Learn how to hook the browser to your IDE, how to troubleshoot things on the backend if you get to the point where you're doing fullstack work. Effective debugging is an incredibly important skill to pick up.
- Learn how to use IDE / other editor tooling such as jump to definition, refactor, inspect, find all occurrences. If this is a react project learn how to use browser extensions like devtools to help with debugging and understanding document structure.
vi-style editing is fast only if one can touch-type. If not, it is not faster. Its plus is the consistency of keybindings across platforms, but even then on international keyboard layouts it is not ideal (or just bad) and requires a lot of customization which may be highly non-trivial in vim modes for various IDEs.
So if one already uses vim, then sure vim-mode in IDE is nice to have, but if not, I suggest not to bother.
So if one already uses vim, then sure vim-mode in IDE is nice to have, but if not, I suggest not to bother.
Not gonna lie, it really blew my mind when I was onboarding junior engineers that not every engineer knows how to touch-type. Frankly, I had never even heard of the terminology "touch-type" until that moment, and basically thought that if you can't do this (in most cases) you never learned to type. Not trying to be rude in any way, any idea how common this is? Maybe more common internationally than in the US?
WebStorm is a good IDE for static analysis and understanding connections between modules and procedures. You can click your way through the call stack with decent accuracy.
A lot of people in this thread are recommending grep and ripgrep, but you do not have context with those tools, so it isn't as helpful, imo. With an IDE, you can find usages of symbols, trace connections, build graphs, etc.
A lot of people in this thread are recommending grep and ripgrep, but you do not have context with those tools, so it isn't as helpful, imo. With an IDE, you can find usages of symbols, trace connections, build graphs, etc.
If it's C, cflow (example usage: cflow .c .h) and doxygen (needs some config, but there's a GUI) could come in handy. Doxygen is capable of generating call graphs if you install some external dependencies.
If it's a different language, you could try looking around in the docs if anyone's generated callgraphs, or you could look up ways to do so.
To trace the code you could use a debugger -- e.g. if it's gdb just issue the command 'start' and then step through from the main function to see how things go. Or (assuming it's something like a C program) you could get an strace (example usage: strace -vvvttf -o strace.log ./program) and maybe get a feeling for the config/etc. files read or written to, network services accessed, etc.
It would help if you could tell us what kind of program it is, or what kind of programming language it's written in.
If it's a different language, you could try looking around in the docs if anyone's generated callgraphs, or you could look up ways to do so.
To trace the code you could use a debugger -- e.g. if it's gdb just issue the command 'start' and then step through from the main function to see how things go. Or (assuming it's something like a C program) you could get an strace (example usage: strace -vvvttf -o strace.log ./program) and maybe get a feeling for the config/etc. files read or written to, network services accessed, etc.
It would help if you could tell us what kind of program it is, or what kind of programming language it's written in.
Play with the application. In each of the main views try to find prominent/unique label strings. Search for those labels in the code base to get an entry point to the view. Then explore the code around that entry point and try to change something here and there and see if it has the effect you expected.
Other then that there is not much to that you would not do in other code bases as well. Set up your IDE probably do you have all the linters ready and can easily navigate the code base.
Ask your lead or mentor to show you the low prio backlog Tickets that have 1 or 2 story points and start the to get your feet wet. Once you have solve one, ask how your commit process works and follow it. Rinse and repeat three times then pick a ticket from the Sprint filter and start contributing.
Other then that there is not much to that you would not do in other code bases as well. Set up your IDE probably do you have all the linters ready and can easily navigate the code base.
Ask your lead or mentor to show you the low prio backlog Tickets that have 1 or 2 story points and start the to get your feet wet. Once you have solve one, ask how your commit process works and follow it. Rinse and repeat three times then pick a ticket from the Sprint filter and start contributing.
What I find more useful than anything else is to find the entrypoint to the code you’re interested in, set a breakpoint on the first line, and step through with a debugger.
IMO it’s by far the fastest way to understand how an application bootstraps itself and produces its output.
IMO it’s by far the fastest way to understand how an application bootstraps itself and produces its output.
For a UI app, I think it's critical to first know how the app works from a user point of view. Then pick a (small) feature, and try figure out how it's rendered to screen by:
- Read high level documentation (hopefully it exists) on the overall architecture, not deep-dives in the beginning
- Find text that is rendered and tracing to where it comes from
- Use the debugger; set breakpoints on client + server code to see the flow of data, particularly into lower layers that get data from DBs/APIs
- Hopefully it's a TypeScript project and you can use an IDE to find references and otherwise navigate from symbols to see where functions/class/fields are used
Assuming sourcemap pipelines work, open your browser and put debuggers on anything you don't understand tracing requests / invocations. Start with something conceptual like a particular page / component and work your way out beginning at the request entrypoint. Along the way you will likely figure out things like authentication management, routing, view hierarchy and design patterns leveraged, etc. Allow a particular issue you need to resolve guide you if needed, but don't sweat the learning curve, as it is required and should be expected.
I like sourcetrail (https://www.sourcetrail.com/). Sadly, in my latest position I have a Mac and I cannot get it to work on a Java project, as it complains that it cannot find a JRE at the $JAVA_HOME path that I specify (which works for all the other projects, including IntelliJ).
Maybe between Christmas and New Year I might have some time to figure out what is wrong.
Maybe between Christmas and New Year I might have some time to figure out what is wrong.
Sourcetrail is being discontinued. Are there any alternatives, even paid ones?
A coworker used this to produce interactive object/flow graphs, years ago:
https://en.wikipedia.org/wiki/Understand_(software)
By https://www.scitools.com/
It was $800 or $1000 then, but considered worth it. Now looks like $100/mo.
https://en.wikipedia.org/wiki/Understand_(software)
By https://www.scitools.com/
It was $800 or $1000 then, but considered worth it. Now looks like $100/mo.
* Use an IDE (Like IntelliJ Idea)
* If there's no architecture-document that gives a short (3-5) page overview on the layout of the code then start writing one.
* Skim through the folders / packages / classes and write down a few obvious questions that pop into your mind while "speedreading" the code.
* Try to make an improvement to see how the whole CI/CD process works.
* Look through the list of direct dependencies / libraries and see where they are used.
* If there's no architecture-document that gives a short (3-5) page overview on the layout of the code then start writing one.
* Skim through the folders / packages / classes and write down a few obvious questions that pop into your mind while "speedreading" the code.
* Try to make an improvement to see how the whole CI/CD process works.
* Look through the list of direct dependencies / libraries and see where they are used.
If you use Git, Mercurial, Bazaar or SVN and would like to visualize where most of the recent effort of development has gone into and what people have worked on the most, have a look at something like Gource: https://gource.io/ In my eyes, those visualizations are really nice for an overview of how the project has evolved.
Personally, i've also had good experiences with Sourcetrail, for seeing how bits of code fit together within a codebase, although the development on it has ceased recently: https://www.sourcetrail.com/
Also, some are recommending text editors with plugins or specialized tools, but i'd also like to suggest just getting a really good IDE that's integrated with the tech stack that you use. Personally, JetBrains fills that niche for me: https://www.jetbrains.com/products/
Depending on the language and framework support, it can lead to an amazing development, refactoring and testing experience, albeit has some drawbacks in comparison to text editors like Visual Studio Code - it uses more memory and CPU resources (especially when indexing the project, a tradeoff that most IDEs out there make in one way or another) and also is a paid product, i just got the Ultimate package of all tools for my personal and work needs.
Apart from that, i'm not even sure - jumping around definitions in source code and seeing how different things are connected, what the dependency graphs are like and so on is nice, but understanding why things were built that way might require ADRs (https://adr.github.io/) which many companies still don't use, or trudging through issue management systems (like seeing what issue a piece of code was developed under, then reading the user story in Jira etc.). Or course, having READMEs and automated scripts for project setup or common actions in it, ideally versioned alongside the code, can also be really nice!
Personally, i think that we as an industry would benefit a lot from more focus on DX (developer experience), both in regards to tooling to explore codebases, as well as practices in regards to documentation for the actual devs to use and dogfood.
Personally, i've also had good experiences with Sourcetrail, for seeing how bits of code fit together within a codebase, although the development on it has ceased recently: https://www.sourcetrail.com/
Also, some are recommending text editors with plugins or specialized tools, but i'd also like to suggest just getting a really good IDE that's integrated with the tech stack that you use. Personally, JetBrains fills that niche for me: https://www.jetbrains.com/products/
Depending on the language and framework support, it can lead to an amazing development, refactoring and testing experience, albeit has some drawbacks in comparison to text editors like Visual Studio Code - it uses more memory and CPU resources (especially when indexing the project, a tradeoff that most IDEs out there make in one way or another) and also is a paid product, i just got the Ultimate package of all tools for my personal and work needs.
Apart from that, i'm not even sure - jumping around definitions in source code and seeing how different things are connected, what the dependency graphs are like and so on is nice, but understanding why things were built that way might require ADRs (https://adr.github.io/) which many companies still don't use, or trudging through issue management systems (like seeing what issue a piece of code was developed under, then reading the user story in Jira etc.). Or course, having READMEs and automated scripts for project setup or common actions in it, ideally versioned alongside the code, can also be really nice!
Personally, i think that we as an industry would benefit a lot from more focus on DX (developer experience), both in regards to tooling to explore codebases, as well as practices in regards to documentation for the actual devs to use and dogfood.
Contrary to what the terminal wizards might tell you about configuring vim, tmux and using grep, just install a real IDE
You can eat your cake and have it too?
I use an ide for core development but knowledge of vim, tmux and linux tools has served me really well.
Also, it helps that a lot of the similarities in newer IDEs are inspired by the predecessors.
There are times where you can't use the ide (middle of ssh session to a remote server, quick edit in a file w/o loading entire folder &c.), so its worth investing some time into these basic tools.
I use an ide for core development but knowledge of vim, tmux and linux tools has served me really well.
Also, it helps that a lot of the similarities in newer IDEs are inspired by the predecessors.
There are times where you can't use the ide (middle of ssh session to a remote server, quick edit in a file w/o loading entire folder &c.), so its worth investing some time into these basic tools.
I switched from using real IDEs to good old fashioned terminal wizardry precisely because I was working on big projects and my IDE was just getting too slow at that scale
To find the frontend code associated with a feature, search the view code for text that you see in the app.
To see how it uses the backend, open the network tab in Chrome developer tools (or equiv in other browsers), look at what requests it makes, and then find corresponding controllers within the code.
To see how it uses the backend, open the network tab in Chrome developer tools (or equiv in other browsers), look at what requests it makes, and then find corresponding controllers within the code.
Look for distinct patterns and code boundaries/modules.
Look at git commit (messages) and issues/PRs. Which parts of the codebase/process is involved in making which type of changes.
Any codebase, especially large ones, may not be consistent during to legacy and many reasons.
Use an IDE + grep/ag.
Look at git commit (messages) and issues/PRs. Which parts of the codebase/process is involved in making which type of changes.
Any codebase, especially large ones, may not be consistent during to legacy and many reasons.
Use an IDE + grep/ag.
I'd actually recommend reading the whole thing quickly and frequently. No note taking needed. It's like playing scales on a guitar. Do it first thing in the morning for a few weeks, start out from different files.
From the root of the repo:
grep -HIron 'your search pattern' *
It is convenient to add an alias for this. Mine includes
If there is no unicode setting "LC_ALL=C" can speed things up considerably too.
--exclude-dir=.git --exclude-dir=.svn
If there is no unicode setting "LC_ALL=C" can speed things up considerably too.
Jetbrains WebStorm has find usages and goto definition features that actually work correctly most of the time, even for Javascript. It's a big step up from just grep'ing strings.
Step 1: Ask the previous programmer if available.
Step 2: Document their knowledge
Step 2: Document their knowledge
Ask your coworkers when you have questions about how/why/where/what something is in the code base, they're the best tools you have
Usually large codebases have to follow the best practice of the framework they use. If you already fluent in that framework, you can jump right in.
What about other people working on the same code base? Can you make them give you run-down of how things are structured?
When you're doing you're tasks, look at other ways similar work has been done throughout your codebase.
I suggest you try JetBrains IDEs depending on your tech stack. They have got an incredible search capabilities. You can search for a word/function where it has been used across project, you can find and replace across the project, etc. It's amazing!
Wouldn't Vscode alone can do the trick?
I have been using Vscode previously and could easily find implementions/definitions inside the project by searching.
Never used JB webstorm before, there will definitely be a learning curve with this new IDE. I am not sure the benefits weigh more than the time i have to get used to new IDE.
Never used JB webstorm before, there will definitely be a learning curve with this new IDE. I am not sure the benefits weigh more than the time i have to get used to new IDE.
VSCode is pretty barebones. You'll need to setup a language server and figure out how to search everywhere and also learn grep in the process if you don't know it already.
In Webstorm, you just press on Shift twice and a popup appears that allows you to search for everything you need. Since the IDE indexed all symbols in your project already, you'll be able to quickly find whatever you need.
In Webstorm, you just press on Shift twice and a popup appears that allows you to search for everything you need. Since the IDE indexed all symbols in your project already, you'll be able to quickly find whatever you need.
Yea came to comments to say this, absolutely +1 for a JetBrains IDE, it takes care of almost everything.
Invest in tooling, Jetbrains IDEs have served me well.
Attend incident postmortem meetings.
ripgrep :)