I wasn't quite a software engineer, but a data analyst/scientist/engineer/term-du-jour at a brand-ish name software company for ~8 years, so pretty close in terms of the day-to-day work and culture.
1) I'm a professional cartographer, sort of. I make wooden topographic maps.
2) A bunch of reasons. I was never "supposed" to work in software -- I went to school in mechanical engineering, and wanted to get closer to something like that. My side biz was becoming viable, I wanted to do something entrepreneurial, and even though I had a pretty good gig, no company is perfect if you're there long enough.
I don't know if I'll go back to data or software some day. Things were great in the map business before the pandemic, they're ok now, and hopefully they'll be great again in the future. I still do a lot of data analysis and write a lot of software for my business, it's just interspersed with a lot more sweeping, sanding, etc.
I actually sort of did this (but I cut them w/ a CNC router rather than 3D printing them).
https://imgur.com/a/fMy19 is the finished product; http://www.thingiverse.com/thing:1524543 are the files I generated and used. They aren't truly accurate maps -- they do have the elevation exaggerated to be more visually interesting, but it's consistently scaled. Some people have gone on to print them w/ 3d printers (rumor is they're on the wall at Makerbot now).
Basecamp is hiring programming, design, marketing, operations, and data interns for summer 2016.
Interns at Basecamp work on real projects and are mentored one-on-one by a member of our team who will guide you throughout your time at Basecamp. The projects you'll work on as an intern at Basecamp are all derived from real problems we face as a business, and we expect you'll have a meaningful impact during your time here. You'll leave Basecamp with new technical, creative, and business skills and having accomplished something significant.
Internships at Basecamp are remote -- you can work from anywhere you want, provided there's some overlap in time zones with your assigned mentor. We'll fly you to Chicago once or twice during the summer to get together with your mentor and the rest of the intern class, and you'll talk regularly with your mentor via phone, Skype, or Google Hangouts. You'll also participate in some of our dozens of Campfire chat rooms every day.
All internships are paid and require a commitment of 8-12 weeks of full time work between May and August 2016 (we're flexible on start/end dates, planned vacations, etc.).
The next two parts will have more detail, but the short answer is "yes" to both a raw word list and a Bayesian filter in terms of techniques we've tried here. One of the simplifying things that makes the problem a little easier is that we don't try to classify beyond "does this need an immediate reply or not."
A couple of weeks, probably, depending on how much I feel like working on it. It was designed to be modular and easily extracted, but still needs some cleanup work and has a few nasty bugs I'd like to fix first.
In short: we as a company have a ton of Ruby experience and comparatively little Python/Node.js experience (both in terms of understanding the tools that we use, which we like to do, and simply just in being able to confidently manage dependencies, etc.), and we knew we were going to want to build our own UI eventually anyway, which limited the utility of Graphite itself.
Edited to add: I can't say it enough, Etsy and Graphite are both fantastic pieces of engineering, with fantastic communities and support behind them (there's a fascinating writeup about Graphite in particular at http://www.aosabook.org/en/graphite.html).
We don't use the same backend as Etsy's original implementation, but you should be able to take any script, etc. that emits statsd measurements for use with the original implementation and point it at Batsd. This means that the 50+ statsd clients that are out there, as well as lots of custom instrumentation, should "just work", even though it's stored and accessed in different ways after it's received.
This is really just one piece in a bigger set of things to track performance, usage, etc.
You can think of it as:
Emitters --> Statsd (or in this case, Batsd) --> dashboards, alerts, etc.
We have emitters coming from Nginx, HAproxy, bluepill, postfix, etc. log files, a gem within all of our Rails apps, and a variety of other scripts that gather data. Those all point to batsd, which aggregates and stores them. We then extract the data into graphs on our dashboard, and use it extensively for Nagios alerting as well. There's a basic sample client included in this repository that we use for those purposes, though you're right, it just gets you raw numbers out of the box.
We're planning on releasing more of both the "emitters" that gather data, as well as a major part of our graphing/dashboard interface "soon".
And point well taken about making it more obvious how to get started and what you can use it for. I'll work on improving the documentation.
(I work at 37signals, though not as a sysadmin or developer)
Just to clarify - we do have a 24/7 on-call system administrator who is the first line of defense for when things go wrong. They're the ones who get phone calls when things do go 'bump' in the night, and they're fantastic in every way.
Our "on call" developers fix customer problems; rarely do these arise suddenly in the middle of the night, but our software has bugs (like most pieces of software) that impact customers immediately, and we've found it helpful to have a couple of developers at a time who focus on fixing those during business hours rather than working on a longer term project. Most companies probably don't call this "on call", but rather something like (as a commenter on the original post pointed out) "second level support". This is what Nick was describing in his post.
Of course, fixing root causes is the best way to solve bugs, and we do a lot of this too. We've taken a significant dent (>= 30% reduction) out of our "on call" developer load over the last 6-12 months by going after these root cause issues.
This is the second long-form piece about Caro and his new book I've seen this week (the other is in tomorrow's New York Times magazine - http://www.nytimes.com/2012/04/15/magazine/robert-caros-big-...). Interesting how similar the facts they tell are, but the angle is very different in each.
Great PR by his agent no doubt, but both enjoyable reads -- well worth the time to read them both.
I'm not aware of anyone offering it as a service, and I don't know how well it would work. Since it's all UDP, there's risk of data loss. We experience essentially no loss within our network (less than one one-hundredth of a percent in testing), but not sure what things would look like going elsewhere.
I know what you mean about fearing the maintenance -- in particular, getting the node.js statsd daemon, Graphite, and Whisper (the default storage and front-ends, in Python) running can be pretty tricky and intimidating. I'd suggest taking a look at the alternate implementations out there -- etsy has a list at https://github.com/etsy/statsd/wiki, and there are a few more not on there as well -- and see if any of them seem more your speed. For us, we really didn't want to introduce a whole extra set of infrastructure requirements to run a Django app and accouterments, which is how we landed on using our own implementation (in Ruby).
Absolutely no offense taken, and I actually share your desire to avoid KoolAid drinking. Every post should be evaluated on it's merits, and while I can only speak for myself, I hope that things that I write only end up on the front page when they have sufficient merit.
(I'm the author of the original post, and obviously I work at 37signals, so feel free to take what I say with a grain of salt.)
Why do posts make it to the front page? Because enough people upvote them, so there must be something that appeals to at least some people. If you have comments on how I could have made this post more interesting or relevant to you, or improvements in my writing style, I'd certainly appreciate hearing them. You can reach me at [email protected].
Sure, small A/B testing will probably only get you to a local maxima. I've never understood why most people think A/B testing is just about changing the color of a button -- it's that, but it's so much more. It's about testing any idea, no matter how big a change it is.
Most people don't lack original ideas, just like there's no shortage of people who would like to be major league baseball players. What people do lack is the ability (or perhaps just the will) to evaluate their ideas objectively.
And while users may evaluate software with their own eyes and emotions, that behavior manifests itself in objective, quantifiable outcomes that have some impact on your business. If measurable user behavior doesn't impact your business, I'd guess you aren't really in a user-driven business.
Our test logs in, causes some data to be fetched from the database, and renders a page which we then check against what it should return.
We haven’t (to date) had either any false positives or false negatives (when it alerts, the site is really down, and if it doesn’t alert, the site is really up).
This obviously isn’t a replacement for functional or integration tests to ensure that a commit doesn’t cause a piece of the app to stop working, but it does test the full infrastructure stack to make sure that it’s performing the way we expect it to.
I have a similar answer - I'm a data analyst at a company full of rubyists. I can (and do) use R for most of my analyses, but there's a cost in transparency -- I can't realistically ask someone to review my R code when they don't know the language, and the bus factor is high if anything I write in R is at all important.
That's why I'm trying to use Ruby where possible, even at the cost of a small productivity hit. The benefits of others being able to read my code far outweigh the few extra minutes it takes for me to do something (and in many cases, the sheer brevity of Ruby as a language means it's faster, simply because it's less typing).
I'd love to see SciRuby become a more useful project, and I'd love to contribute. Unfortunately, they don't make it especially easy to get involved -- the mailing list points people to the roadmap, but it's not at the level of detail where someone could jump in (and the component gems don't seem much better), so it's a bit hard to know where help would actually be useful.
This looks fantastic. From the short demo, nice UI for setting up charts, and the visualizations produced are quite nice as well.
The only downside to something like this (and Graphite, etc.) is that you can only visualize data that you store in Cube (if I understand correctly). That's great for things you sample yourself, but what if you want to compare something you sampled yourself with something from New Relic or another data source? Cube looks to go a step further than Graphite in allowing for easier custom visualizations, but data is still a one-size-fits-all thing. You can write an emitter/parser that grabs the data and stores it in Cube, but there's a huge amount of duplication (and essentially polling then).
I think a better architecture is to separate data storage/acquisition from visualization. Establish a set of standards for data interfaces that specify the output they provide, and build your visualization on top of that. Then, your interfaces can be API wrappers, mysql connections, a mongo-db based storage system, a redis-backed storage system, etc.