Show HN: Sourcetrail – Get productive on unfamiliar source code(sourcetrail.com)
sourcetrail.com
Show HN: Sourcetrail – Get productive on unfamiliar source code
https://www.sourcetrail.com/
118 comments
If you're looking for a similar tool which does have a code unravel-er look at SciTools Understand[1] for C/C++/Java
My previous company used Understand on a large unfamiliar codebase, it helped greatly, but was pricey.
[1] https://scitools.com/
My previous company used Understand on a large unfamiliar codebase, it helped greatly, but was pricey.
[1] https://scitools.com/
There's also Source Insight [1] which supports C/C++/Java/Obj-C - quite a fantastic tool, but unfortunately it's Windows only (Wine works with a few hiccups)
[1] https://www.sourceinsight.com/
[1] https://www.sourceinsight.com/
Wait a second, a tool that supports going through Obj-C is Windows only? That’s .. a shame.
I am playing around with an idea like this for editing Clojure in Emacs with cider-mode. You have to manually build the 'thread' that such a code unravel-er would return, but then you can read and edit functions from many files in one buffer. You start with the highest-level function you are interested in, and then pull the definitions of functions it calls into view as needed.
https://github.com/fazzone/multifiles.el/tree/cider-hacks
https://github.com/fazzone/multifiles.el/tree/cider-hacks
Hm does anyone know of a web-based tool like this that uses Clang as a back end? I want to put open source code like the bash and Python interpreters in it and be able to navigate symbolically. And I want nice permalinks.
EDIT: It looks like https://woboq.com/codebrowser.html is what I want. The code available but not under and open source license. https://github.com/woboq/woboq_codebrowser/
I will probably try this, but I'm interested in alternatives!
EDIT: It looks like https://woboq.com/codebrowser.html is what I want. The code available but not under and open source license. https://github.com/woboq/woboq_codebrowser/
I will probably try this, but I'm interested in alternatives!
We built Sourcegraph, which is a web-based code search and navigation tool. Our Python support is in beta, and you can try it out at https://sourcegraph.com/github.com/plotly/dash/-/blob/dash/d.... (And hit Alt+D/Opt+D to search by symbol.) Sourcegraph also supports a bunch of other languages: https://sourcegraph.com/help/languages.
It gives you nice URLs (hit Alt+Y/Opt+Y to get the full commit ID in the URL to make it permanent).
Would love to hear your feedback.
It gives you nice URLs (hit Alt+Y/Opt+Y to get the full commit ID in the URL to make it permanent).
Would love to hear your feedback.
I just spent a few minutes clicking around but had a bit of trouble.
EDIT: nevermind, most of this is due to the fact that you have to keep "repo" in the search box, which I found confusing.
I quickly found the repo I wanted, but discovered that C/C++ isn't supported, so Woboq is more useful to me at the moment.
I tried to build something like this once myself, and made the mistake of going "wide" rather than "deep". That is, trying to support all languages. I think it does make sense to try to get the absolute best experience for one language. But yeah for now since C/C++ aren't supported, it's not useful for me.
------
I typed "cpython" in the search box on https://sourcegraph.com/ and got no results. (I also typed "python" and got no results, which seems pretty odd.)
I would have liked to browse this repo:
https://github.com/python/cpython
I guess you have to import repositories yourself? There seem to be sample repositories, but they are in Go, and I'm interested in C/C++ (the Python interpreter, not Python source code).
I'm looking for something that uses Clang as a back end for intelligent navigation of C/C++ projects.
Also, I would like to browse the bash source code, but as far as I know it isn't available on Github. Can SourceGraph import from an arbitrary git repo?
https://savannah.gnu.org/git/?group=bash
That said I'm glad somebody is working on this problem! I like your mission statement.
EDIT: nevermind, most of this is due to the fact that you have to keep "repo" in the search box, which I found confusing.
I quickly found the repo I wanted, but discovered that C/C++ isn't supported, so Woboq is more useful to me at the moment.
I tried to build something like this once myself, and made the mistake of going "wide" rather than "deep". That is, trying to support all languages. I think it does make sense to try to get the absolute best experience for one language. But yeah for now since C/C++ aren't supported, it's not useful for me.
------
I typed "cpython" in the search box on https://sourcegraph.com/ and got no results. (I also typed "python" and got no results, which seems pretty odd.)
I would have liked to browse this repo:
https://github.com/python/cpython
I guess you have to import repositories yourself? There seem to be sample repositories, but they are in Go, and I'm interested in C/C++ (the Python interpreter, not Python source code).
I'm looking for something that uses Clang as a back end for intelligent navigation of C/C++ projects.
Also, I would like to browse the bash source code, but as far as I know it isn't available on Github. Can SourceGraph import from an arbitrary git repo?
https://savannah.gnu.org/git/?group=bash
That said I'm glad somebody is working on this problem! I like your mission statement.
Thanks for posting this. We'll think about how we can make that search UX clearer. And when we have C/C++ support, I hope you'll find it super useful.
> Sourcegraph also supports a bunch of other languages: https://sourcegraph.com/help/languages.
That's an interesting link. It only works when you go to "Help -> Language features".
That's an interesting link. It only works when you go to "Help -> Language features".
Website feedback: Supported languages is the #1 most important piece of info about this software. It should be front and center and probably repeated a few times already on the landing page.
This may be a result of your feedback, but for me looking just now it it front and center: "Sourcetrail - A cross platform source explorer for C/C++ and Java"
Haha there it is. Who said whining on HN doesn't change the world.
That honor goes to melling:
>Some guy whining on HN about a first generation product adds zero value.
https://news.ycombinator.com/item?id=9500455
https://news.ycombinator.com/item?id=9500455
it should be in a bigger font, and it seems like it's the only place where language compatibility is clearly visible without going into the documentation.
If I could upvote you more than once; I would.
I left the site not really knowing what language/s are supported.
After digging through the entire page it looks like maybe C/C++/Java?
There should be a huge button, or something, front, and center. <<SUPPORTED LANGUAGES>>
Looks like a cool idea, but I don't know if I can even utilize this tool.
I left the site not really knowing what language/s are supported.
After digging through the entire page it looks like maybe C/C++/Java?
There should be a huge button, or something, front, and center. <<SUPPORTED LANGUAGES>>
Looks like a cool idea, but I don't know if I can even utilize this tool.
I'd suggest second most important piece of info is probably supported editors.
Can you please share the roadmap for adding new language support?
To be honest: We have not decided what language we will pick up next. Our first priority is to improve on our Java coverage. In a few months we should be ready to choose our next language. We are thinking about either tackling Python or implement support for a more recent language like Kotlin, Swift or Go.
I doubt there's very much legacy-ish Kotlin or Swift code out there to be a significant selling point for your product.
I spend a lot of time in clients' ios apps which were developed by other contractors long ago and then I'm asked to fix bugs and add new features. I think this would help me save a lot of time. Since it already supports C/C++ would objective-c support be relatively straightforward to add? Not sure if there are many others who would like to see it, but I know I would definitely appreciate it.
Seems to me there is probably a huge amount of legacy Python code around that people need to explore. In fact, I've spent the entire day so far creating some so that we have enough legacy Python code around in a couple more years. Wouldn't want to run short.
I might buy this for Go if it was available, this looks interesting!
Note that I'm mainly interested because my current editor of choice does not have the full suite of tools I'm used to for Go. I used to use Vim, and vim-go has such an excellent suite of tools I don't think I'd buy this. However, I've since switched to Kakoune, and the tool support is quite young and I've not had time to improve it myself. So this tool (at the $100 self employment pricing) is a nice fit for a tool I'm on the fence about. At $200 I don't think I would, but at $100 I think I would.
Anyway, just providing a bit of feedback. Looks really nice!
Note that I'm mainly interested because my current editor of choice does not have the full suite of tools I'm used to for Go. I used to use Vim, and vim-go has such an excellent suite of tools I don't think I'd buy this. However, I've since switched to Kakoune, and the tool support is quite young and I've not had time to improve it myself. So this tool (at the $100 self employment pricing) is a nice fit for a tool I'm on the fence about. At $200 I don't think I would, but at $100 I think I would.
Anyway, just providing a bit of feedback. Looks really nice!
Thank you for sharing your opinion! This will be exactly what we will be interested in when deciding what language to tackle next :)
I know it's easier said than done, but a pluggable interface for community created language support would go a long way towards gaining traction.
A Lisp might be an easy language to target, because of its consistent syntax and "code is data" philosophy. With Clojure, you could both reuse some of your work on Java, and support Clojure's Java interop. Attempting to support Clojure would drive improvements in the Java support.
Seems to me like the obvious choices would be languages that have large, long standing legacy codebases. Swift code is going to be relatively tight just because it's so recent, whereas something like C# isn't.
Swift would be great. Perhaps you could leverage SourceKit.
Python, Scala and Go would be great additions (in that order).
C# please!
Um..you already have this if you have visual studio 2015. https://youtu.be/01MzfE8s6G8?t=54
Code Maps requires Enterprise, I think. You can view generated Code Maps from other editions, but you need Enterprise in order to generate them. Kind of unfortunate.
Just a feature request if it isn't already in the works:
Runtime analysis of code that actually matters ie live code.
Often its hard to figure out what in the code base actually matters.
Debugging is still sort of difficult and dead code is really hard to find and get rid of. Particularly if reflection is used heavily which it almost always is these days.
Otherwise (IMO) as a Java developer IntelliJ and Eclipse are actually not that far off in their static analysis and already pretty good to discover the code. Especially IntelliJ as I can easily see them enhancing their code view/discovery as well as they already have static dead code analysis.
I guess I don't think discovering code bases is that much of a problem compared to understand what is actually running.
That is I think debugging particularly multithreaded applications is vastly more difficult problem. Thus runtime analysis would be very useful.
Runtime analysis of code that actually matters ie live code.
Often its hard to figure out what in the code base actually matters.
Debugging is still sort of difficult and dead code is really hard to find and get rid of. Particularly if reflection is used heavily which it almost always is these days.
Otherwise (IMO) as a Java developer IntelliJ and Eclipse are actually not that far off in their static analysis and already pretty good to discover the code. Especially IntelliJ as I can easily see them enhancing their code view/discovery as well as they already have static dead code analysis.
I guess I don't think discovering code bases is that much of a problem compared to understand what is actually running.
That is I think debugging particularly multithreaded applications is vastly more difficult problem. Thus runtime analysis would be very useful.
Thank you for your input! I wouldn't call it a feature request though, I'd say it's another tool. But, you are right, exploration of dynamicly analyzed code would also greatly benefit understanding.
I really want something like this for heterogenous codebases. Maybe along the lines of ollydbg and ida but for "reverse" engineering Ruby, python, and go.
Not saying this is bad, but...
Instead of giving money to a tool like this, you could just use ctags/etags, both of which are Free Software. It doesn't show detailed graphs like Sourcetrail does, but it allows source code traversal by function definitions, and supports a crapton of languages.
Instead of giving money to a tool like this, you could just use ctags/etags, both of which are Free Software. It doesn't show detailed graphs like Sourcetrail does, but it allows source code traversal by function definitions, and supports a crapton of languages.
There doesn't seem to be a vscode plugin for Sourcetrail. Could be a worthwhile addition?
Thank you again for your request. We now offer a plugin for VS code: https://github.com/CoatiSoftware/vsce-sourcetrail
It hasn't been requested so far. Is it actively used by C/C++ or Java developers?
Nevertheless I opened a feature request to track demand: https://github.com/CoatiSoftware/SourcetrailBugTracker/issue...
Has anyone run a large codebase like chromium with this tool? Is the indexing looking at what's compiled like libclang or is it static like gtags?
Pretty sure it's compiled,I tried it out and it took a while to process on code that ggtags parses in seconds.
I'd imagine it's going to violently shit itself. Chromium has pretty good online code search btw.
Yes, but not the kind of visualization and calling stack shown by SourceTrial. Some of the code is 10 layers deep where the stuff actually happens with many complicated usage of inheritance, which I figure is where this kind of tool can shine.
On a first look, it looks amazing. This is actually what I've always imagined "graphical" programming to look like.
Interesting can you edit code in the text view too? Wonder if this could be the one place to write your code too.
We decided to leave out editing early in the process. Most developers already have their favorite editor and it's a lot of work to implement all features that users would expect from it.
We decided to focus on source exploration instead. Sourcetrail is made for the second screen. We implemented plugins for the most used source editors to enable easy switching. Please have a look at our documentation: https://www.sourcetrail.com/documentation/#CODEEDITORPLUGINS
We decided to focus on source exploration instead. Sourcetrail is made for the second screen. We implemented plugins for the most used source editors to enable easy switching. Please have a look at our documentation: https://www.sourcetrail.com/documentation/#CODEEDITORPLUGINS
Cool! Wonder what the graph exploration view would be like if you used a hyperbolic plane to embed it in haha: https://en.m.wikipedia.org/wiki/Hyperbolic_tree
Nice work; seems like you hit a nice medium between trying out something new and keeping it connected to the real world and practical. :)
Nice work; seems like you hit a nice medium between trying out something new and keeping it connected to the real world and practical. :)
Slightly tangential.. a code editor was showcased on HN few years back, with context sensitive "panes".. for example currently selected function/variable declarations in another pane. Very impressive stuff, but can't seem to find it anywhere now.
The best similar tool looks like http://brackets.io/, but it only works for HTML/CSS.
Watched the demo video, can you add support to Javascript and Ruby? This is an amazing tool
Looks really good: one possible suggestion would be to allow sharing/reuse of indexes. Let me commit the built index to git and then have someone else checkout and use the index that was prebuilt.
One (potentially paying) alternative would be to provide a precompilation service. Not sure if your index generation can be parallelized, but potentially you could take an archive of the code (or pull directly from Github or Bitbucket or any other repo endpoint) then spin up a large EC2 instance (or the equivalent for your cloud of choice) and accelerate the generation of the index. For large codebases it may provide a good boost.
One (potentially paying) alternative would be to provide a precompilation service. Not sure if your index generation can be parallelized, but potentially you could take an archive of the code (or pull directly from Github or Bitbucket or any other repo endpoint) then spin up a large EC2 instance (or the equivalent for your cloud of choice) and accelerate the generation of the index. For large codebases it may provide a good boost.
I had a quick play with this today and it seems like something that could be really useful.
A couple of relatively minor comments for you:
* In C++ it might be useful to filter include guards out of the symbol lists.
* I had some problems with using the menus (couldn't click on them, when I did select them couldn't get the menu to drop down). It's probably just because I'm using a weird window manager (xmonad, no window decorations, Ubuntu 16.04).
A couple of relatively minor comments for you:
* In C++ it might be useful to filter include guards out of the symbol lists.
* I had some problems with using the menus (couldn't click on them, when I did select them couldn't get the menu to drop down). It's probably just because I'm using a weird window manager (xmonad, no window decorations, Ubuntu 16.04).
I could use something like this for PHP spaghetti code.
You may be interested in Exakat:
https://www.exakat.io/up-and-running-with-exakat/
https://www.exakat.io/up-and-running-with-exakat/
Looks cool! This hits a pain point of mine working in a legacy Python and C/C++ codebase. Any plans for Python support?
Happy to hear that you like it! Full support of Python won't be easy but maybe we could tackle this one in stages. Implement highly demanded features in a first Python release and improve on that later on.
That'd be great. How can we stay in the loop to hear about Python and other feature releases?
Oh, that's an easy question :) You can either follow us on Twitter (https://twitter.com/Sourcetrail) or just sign up for our mailing list (https://www.sourcetrail.com/#get-started). I promise that we won't spam you ;)
+1 for python.
How different is this from CodeLens ? Seems like most of these features are in CodeLens anyways, maybe the licensing for CodeLens??
CodeLens => https://msdn.microsoft.com/en-us/library/dn269218.aspx
CodeLens => https://msdn.microsoft.com/en-us/library/dn269218.aspx
I think you're thinking of CodeMAP, not CodeLENS.
CodeMap => https://docs.microsoft.com/en-us/visualstudio/modeling/use-c...
CodeMap => https://docs.microsoft.com/en-us/visualstudio/modeling/use-c...
CodeLens is great for finding references within the IDE. It also provides information about the history, which Sourcetrail doesn't do yet.
I would say we go a step further, by also providing a visual representation of the references to symbol, which quickly conveys how a symbol is embedded into the rest of the code base, without even reading a line of code. That makes exploring a lot faster.
I would say we go a step further, by also providing a visual representation of the references to symbol, which quickly conveys how a symbol is embedded into the rest of the code base, without even reading a line of code. That makes exploring a lot faster.
Do you also have arrows to methods in your own class? In addition, how does this relate to code maps in VS?
https://msdn.microsoft.com/en-us/library/jj739835.aspx
Seems they cover a similar thing.
https://msdn.microsoft.com/en-us/library/jj739835.aspx
Seems they cover a similar thing.
Yes, Sourcetrail also shows class internal calls and accessed fields. Code Maps in VS has a different approach because they try to keep the graph small by making the user add symbols. So if you are trying to see every access of a symbol it is easy to miss something. Sourcetrail on the other hand keeps the graph small and comprehensible by showing only one level of dependencies. You don't need to add anything yourself. So you cannot miss any relation ;)
Thanks. Sounds more promising than I took from you landing page. I will give it a go on my c/c++ code bases. See if it can help me.
I now notice the free trial cannot be run on your own project. I have to say I won't buy anything that I didn't try on a real sample before. What made you decide to go for this option instead of regular limited functionality or time bound (my preference).
I now notice the free trial cannot be run on your own project. I have to say I won't buy anything that I didn't try on a real sample before. What made you decide to go for this option instead of regular limited functionality or time bound (my preference).
Oh, we have a time bound evaluation testing period available: https://www.sourcetrail.com/trial#test-license
How did I miss that? Will try tomorrow!
I doubt you could ever get something like this working for Ruby, which is where it is actually badly needed.
Yes, dynamically typed languages are much harder to tackle :/
You don't need it to be fully automated. For dynamic languages you would have people in the loop to annotate the parts the tool couldn't figure out.
This seems like no better off. Documentation is historically always out of date the second its written. The same would be true for annotation.
As noted by others the annotations would be active checks. More like proofs in coq than static text. That way you'd be building up a logical understanding of the code base that was machine verifiable sidestepping the issue of stale comments.
If done properly this could even feed into code dynamics to capture information on the runtime information. So that at every point in the program you could start asking questions like how many times did this line execute, what was the type of this variable at this point, how much memory was allocated, and so on and so forth.
If done properly this could even feed into code dynamics to capture information on the runtime information. So that at every point in the program you could start asking questions like how many times did this line execute, what was the type of this variable at this point, how much memory was allocated, and so on and so forth.
Seems like youve just abstracted things like static typing into a extension of the language thats checked at a less optimal time. At this point arent you better off just using a language that incorporates this functionality and compiles it? Sounds like youve just implemented static typing but in a really akward and unecessary way
Guess you could have some assertions together with the annotations to make sure things are up-to-date. Then integrate those tests with your CI of choice and it should be a bit easier to keep up-to-date.
You're best off getting traces from a modified runtime to annotate what's going on.
What's the C++ support like?
Sourcetrail developer here: Pretty solid. We use clang libTooling.
How's the performance on a large codebase?
North of 500k lines, few thousand cpp/h files, 40+mb
North of 500k lines, few thousand cpp/h files, 40+mb
We have been working on that for the past few month. Right now the performance is pretty good for C++ projects up to around 2 million lines of code. For C projects the performance is even better! If you want to test it, just download the pre-indexed clang project (900k loc) from our website (https://www.sourcetrail.com/downloads#extra). Indexing the project took 12 minutes on a 5 year old laptop.
Really great news! I think I'll give this a demo when I have some capacity.
Thanks
Thanks
I tried it on a fairly big repo today (~500k loc) and once it was done with the initial processing performance was fine.
Compile commands ? Please Yes?
There is also a tool called Bear which will generate compile commands from Make-based builds, in case you were wondering. I've used it with some success.
https://github.com/rizsotto/Bear
https://github.com/rizsotto/Bear
Im well aware of bear. Works fine in some cases, not so well in others. Author is also doing scan-build replacement in python which imho is far superior to bear.
Yes, we support Compile Commands :)
What are compile commands?
A Compile Command is a JSON file entry that specifies what the C/C++ compiler should do when compiling a file. So it defines a file to compile and all the required flags (like include paths or preprocessor defines). These Compile Commands are used for all kind of Clang based tools. You can read more here: https://clang.llvm.org/docs/JSONCompilationDatabase.html
What languages are supported? At least on mobile, there is zero mention of language support outside of testimonials and screenshots. Is it just Java and C++?
The intro video mentions Java, C, and C++.
Right, it's "just" Java, C and C++ for now but we are planning to extend our available languages as soon as possible!
Looks fantastic. Do you have plans to support additional languages?
This would be particularly useful in open source.
This would be particularly useful in open source.
I love the implementation for C++, I'm just waiting for more language support.
Any plans to support Kotlin ?
Kotlin is definitely on the shortlist for the next language to be supported! Which language we'll pick up exactly depends on the landscape of available tools for that language. Kotlin is pretty new so I guess that there could be a need for Sourcetrail :)
If you ever go that route and you plan or using/creating some open-source components I would be very interested on hearing more (I am a JavaParser/JavaSymbolSolver contributor)
Any plans to add support for PHP to this? it looks really cool :)
Got lots of compiling error when trying it on a large code base
That looks amazing. Does it also work with Objective-C ?
No, not yet. There is a feature request: https://github.com/CoatiSoftware/SourcetrailBugTracker/issue...
It reminds me of a feature I've always wanted, which would be a code unravel-er. Basically I'd like to be able to select a line anywhere in a codebase, and have said codebase expand to show me the entire execution path as if it were in one single scope, with the ability to re-collapse sections back into loops and functions while reading.
I think it would help to better understand the codepaths that can start to get fuzzy when jumping between abstractions. It would be great - not only for understanding unfamiliar code - but also for double-checking logic across longer code paths.
At any rate - great idea, great implementation, and beautiful website.