Ask HN: What are the best open source tools you use in your organization?
50 comments
Come on folks, don't just list the command line tools that come with your Linux distro :)
Let's talk about the big packages that bring value to organisations.
Let's talk about the big packages that bring value to organisations.
Rails
ElasticSearch (And Kibana, holy shit)
MySQL
React
Ractive
Ubuntu
NGINX
Redis
We stand on the backs of giants. Yet, for some reason, corporations feel entitled to not give back. Modern computing lives by open source. Companies should be obliged to contribute similarly.
ElasticSearch (And Kibana, holy shit)
MySQL
React
Ractive
Ubuntu
NGINX
Redis
We stand on the backs of giants. Yet, for some reason, corporations feel entitled to not give back. Modern computing lives by open source. Companies should be obliged to contribute similarly.
Phabricator for task managment, project management, and code reviews. It's open source, great software, and has a fun personality great for internal use.
Same here, really great as it packs a lot. The new Gitlab 8 with the CI integrated is great as well but we preferred Phabricator for the extended feature.
Ultimately we decided not to embrace fully yet the Phabricator CI (harbormaster/drydock) as it's been changing a lot (it's a prototype after all) and breaking frequently so we have a mix of Harbormaster and Jenkins which works very well.
Ultimately we decided not to embrace fully yet the Phabricator CI (harbormaster/drydock) as it's been changing a lot (it's a prototype after all) and breaking frequently so we have a mix of Harbormaster and Jenkins which works very well.
The main thing that's confusing me about Phabricator is that it seems to be centered around a one-repository workflow (which I gather FB uses). Does it work well when you have different projects that are unrelated to each other, or is the idea to set up dedicated instances for each project?
A bit different from the rest, is `repo`, from AOSP: https://source.android.com/source/downloading.html
We run dozens of microservices and have nearly 100 git repositories. Using `repo`, every dev has every repo in consistent (relative) path from each other. It also greatly simplifies cross-cutting changes across repositories, and staying in sync (`repo sync` updates all repos all at once.)
I also use it to manage my dotfiles: github.com/grahamc/manifest
Second: Go.CD (https://www.go.cd/) -- applying build templates to our 100 repositories has greatly simplified our management of the repositories. Using pipelines has added deep visibility into what is deployed where. I love Go.CD.
We run dozens of microservices and have nearly 100 git repositories. Using `repo`, every dev has every repo in consistent (relative) path from each other. It also greatly simplifies cross-cutting changes across repositories, and staying in sync (`repo sync` updates all repos all at once.)
I also use it to manage my dotfiles: github.com/grahamc/manifest
Second: Go.CD (https://www.go.cd/) -- applying build templates to our 100 repositories has greatly simplified our management of the repositories. Using pipelines has added deep visibility into what is deployed where. I love Go.CD.
HAProxy: http://www.haproxy.org/
The configuration language leaves much to be desired ... but it's otherwise a fantastic piece of software. It's the most configurable, full-featured, stable, and fast proxy server I've ever worked with.
The configuration language leaves much to be desired ... but it's otherwise a fantastic piece of software. It's the most configurable, full-featured, stable, and fast proxy server I've ever worked with.
What does it have over nginx?
Depending on what you care about, either everything or nothing. Some interesting things I find useful:
* Multiple load-balancing schemes (round robin, least conn, first, balance-on-src-ip, balance-on-uri, balance-on-uri-param, balance-on-header-value, etc)
* Advanced health-checking (out of band, script-based, remote socket based, HTTP send/expect, etc)
* Really, really easy HTTP request and response manipulation
* Generic counter support - think something like "rate limit request based on the average number of requests over the last 5 minutes that all had this random URL param". That's a contrived example, but it's easy in HAProxy.
* Fantastic ACL support. You can even dynamically update them on the fly based on a response from an upstream server, if you so wanted. Not that you should, but you could.
Honestly, you can do most things you'd need in nginx these days. But HAProxy tends to do it just as well, and in a much, much more flexible manner. And in some ways, the ACL support and terseness of the configuration allows you to write out really complex behavior in ways that would be annoying or difficult in nginx.
* Multiple load-balancing schemes (round robin, least conn, first, balance-on-src-ip, balance-on-uri, balance-on-uri-param, balance-on-header-value, etc)
* Advanced health-checking (out of band, script-based, remote socket based, HTTP send/expect, etc)
* Really, really easy HTTP request and response manipulation
* Generic counter support - think something like "rate limit request based on the average number of requests over the last 5 minutes that all had this random URL param". That's a contrived example, but it's easy in HAProxy.
* Fantastic ACL support. You can even dynamically update them on the fly based on a response from an upstream server, if you so wanted. Not that you should, but you could.
Honestly, you can do most things you'd need in nginx these days. But HAProxy tends to do it just as well, and in a much, much more flexible manner. And in some ways, the ACL support and terseness of the configuration allows you to write out really complex behavior in ways that would be annoying or difficult in nginx.
For us the list is very long, beginning with debian and ubuntu. Elasticsearch, redis, haproxy, mongodb, postgressql ... really too many to list. Of these certainly elasticsearch, redis, and haproxy are best in class in my opinion, and all the tools we use are proven performers with solid community support.
The question needs expansion.
What is the OP's definition of best? Why does it matter to the OP that it's open source (would your org contribute back)? Would specific family of licenses matter? And in what problem domain do the tools be used for?
I'm guessing most of HN's audience would be using some form of OSS. The coverage of the comments here is large: devops tools, programming tools, libraries, productivity, etc.
Our own org uses a lot of open source components, from desktops, office suite, dev environments, server OSes, programming platforms, libraries, databases, CMSes, heck, does Android count?
More context is better.
I'm guessing most of HN's audience would be using some form of OSS. The coverage of the comments here is large: devops tools, programming tools, libraries, productivity, etc.
Our own org uses a lot of open source components, from desktops, office suite, dev environments, server OSes, programming platforms, libraries, databases, CMSes, heck, does Android count?
More context is better.
<sigh> Not a single tool we use is open source. Mainly because with an open source application there's no one source for security to go to and ask for a letter certifying the safety of the software.
I wish I were kidding.
I wish I were kidding.
no linux servers anywhere? No open source network infrastructure?
I'm not the original poster, but I find it slightly humorous at the incredulity of not running open source software anywhere. 15 years ago nobody would have batted an eyelid.
Back in the "dark days", I worked for a couple of Microsoft Certified Partners. One of the things they all had in common was a "no open source" policy. I wasn't even allowed to run emacs as my editor. GPL was explicitly "evil" and even more permissive licenses were "evil enough". Hell, I had to use Visual Source Safe for version control! I would have killed for RCS, let alone something like Subversion.
And, no, not a single Linux box, or even BSD system to be found. Every single server was running Windows. I remember noticing strange network traffic one time and, lacking the tools in Windows, booted up Knoppix and diagnosed the problem -- a former employee had set up a program on an idle machine to forward the source code for our software outside the company. I couldn't tell the network admins how I had found the problem because it was more than my job was worth to admit to having used GPL software on the premises.
Of course, I don't work for companies like that any more. In fact, I'm gratified to discover that they are so rare that nobody believes they could possibly exist. In my current job, apart from 3 SaaS products (Github, Trello and Slack), every single tool I use is open source. I sometimes have to pinch myself to make sure I'm not dreaming...
Back in the "dark days", I worked for a couple of Microsoft Certified Partners. One of the things they all had in common was a "no open source" policy. I wasn't even allowed to run emacs as my editor. GPL was explicitly "evil" and even more permissive licenses were "evil enough". Hell, I had to use Visual Source Safe for version control! I would have killed for RCS, let alone something like Subversion.
And, no, not a single Linux box, or even BSD system to be found. Every single server was running Windows. I remember noticing strange network traffic one time and, lacking the tools in Windows, booted up Knoppix and diagnosed the problem -- a former employee had set up a program on an idle machine to forward the source code for our software outside the company. I couldn't tell the network admins how I had found the problem because it was more than my job was worth to admit to having used GPL software on the premises.
Of course, I don't work for companies like that any more. In fact, I'm gratified to discover that they are so rare that nobody believes they could possibly exist. In my current job, apart from 3 SaaS products (Github, Trello and Slack), every single tool I use is open source. I sometimes have to pinch myself to make sure I'm not dreaming...
Nope, not that I'm aware of. Everything is Windows Server.
We had an application from one of our system vendors that incorporated FFmpeg for playing video streams within the app. We had to go and ask them to remove FFmpeg and replace it with some other video package that IT security approved of.
This is normal for my industry. It's kind of depressing. I hear about all these nice open source tools (pandas! numpy! for a couple of examples) and have to make do with closed source equivalents I like much less (MATLAB).
We had an application from one of our system vendors that incorporated FFmpeg for playing video streams within the app. We had to go and ask them to remove FFmpeg and replace it with some other video package that IT security approved of.
This is normal for my industry. It's kind of depressing. I hear about all these nice open source tools (pandas! numpy! for a couple of examples) and have to make do with closed source equivalents I like much less (MATLAB).
GCC
GDB
VTK
OpenCV
I work for big monolithic company (not a software company). Most of the libraries and dev tools the engineers in my dept use are open source unfortunately the environment they run on isn't.
I work for big monolithic company (not a software company). Most of the libraries and dev tools the engineers in my dept use are open source unfortunately the environment they run on isn't.
Spark has improved our ETL jobs by orders of magnitude, both with respect to performance and ability to engage our workforce (mostly Python programmers).
Previous tools that improved workflow: docker, nginx.
Previous tools that improved workflow: docker, nginx.
What is Spark? Google turns up a bunch of projects called that.
I assume they are talking about Apache Spark http://spark.apache.org/
ZeroMQ, React, Node.js, Nginx, Redis, PostgreSQL, and of course Linux. Together they make a great platform for building quality apps very quickly (which is the point of our organization).
Kubernetes has been amazing for us. Getting better every month.
More recently, Drone for CI has been a quantum leap in usability for us over Jenkins. Built on top of Docker, has service container support built in, and has the unique feature of plugins-as-Docker-containers (so you can write them in any language): https://github.com/drone/drone
Ansible, ramping up with some Terraform and really loving it.
More recently, Drone for CI has been a quantum leap in usability for us over Jenkins. Built on top of Docker, has service container support built in, and has the unique feature of plugins-as-Docker-containers (so you can write them in any language): https://github.com/drone/drone
Ansible, ramping up with some Terraform and really loving it.
If you haven't heard of Heka (http://hekad.rtfd.org), you should check it out.
Oh man sentry is pretty awesome.
http://getsentry.com
http://getsentry.com
Lots of Node & NPM. Also Grunt, Brew, Ant, Maven. Not to mention the rest of the stacks.
Blender. For fully programmable 3D rendering, including command line only.
Node, React/Native,
Angular,
Spring,
RabbitMQ,
PostgreSQL,
Docker,
Vagrant,
Consul,
Spark,
Kafka,
Too much to name...
git
Ansible, Redis, Postgres, Django, Linux, Nginx, Spark, Git, AngularJS, Elasticsearch. Thanks!
GCC
GitLab
Git,Debian,Ruby on Rails,PostegreSql,Redis,nginx primarily.
Ubuntu, HAProxy, Ejabberd, Redis are the solid standouts.
mrtg/smokeping, netdisco2, nfdump/nfsen and previously request tracker, Nagios and racktables
WordPress
ssh, openvpn, ansible, git... bash :)
Git
Eclipse
Vagrant
Bash
Pmwiki
Ubuntu
Nginx
Apache
MySQL
Maven
Puppet
Eclipse
Vagrant
Bash
Pmwiki
Ubuntu
Nginx
Apache
MySQL
Maven
Puppet
GNU Make.
docker
It has been my observation that when many folks ask for open source software, what they really want and mean is free(as in beer) software.
Prosody https://prosody.im/. Open source XMPP server with support for MUCs and all the usual XMPP stuff. Best way for teams to communicate. Extend it with plugins for your NMS and so on to get alerts in realtime.
Nagios https://www.nagios.org/. Solid (if a little fiddly) monitoring / alerting solution. Works best if you write your own plugins and use check_by_ssh instead of NRPE.
bcfg2 http://bcfg2.org/. Configuration management that makes sense. Tidy and well defined "model" for how configuration should be managed, sensible defaults and a bunch of useful powerful features.
I could go on but there are some of my all-time favourites.