I couldn't agree more with your final thought. I assumed there may only be one or two diverted flights in a day, and certainly a handful more during winter weather, etc.
I clearly remember when I finally let my initial prototype run on a laptop for a day and ignored it, an old HP laptop just chugging away parsing flight plan data, plugged in and running too hot in my office. By the next day I had > 20 diversions, with no notable weather, and assumed I had a bug in my logic.
Lots more to do here, including adding some weather details and someday predictive analytics. Thanks again for checking it out and the feedback.
Finally getting around to sharing my long overdue side project that I had started back in Jan 2021.
The data processing is handled by two EC2's running Java to parse the FAA's flight plan feed. Peak midday that's running about ~65msgs/sec, a reduced and manageable number because I'm filtering for major airlines only (no GA, no bizjet, etc). The cleaned-up and filtered flight plan data heads to both Postgres for storage as well as SQS.
Consuming the SQS messages is a Python app that's crunching flight plans and confirming each flight is still heading to it's original destination (thanks redis!). If the destination airport changes, the flight gets flagged and some add'l business logic runs.
Ultimately you get the frontend (Django) presented here with a sprinkling of terrible JS to get the map and some timeline data.
I've been hacking with the FAA's SWIM data feeds for a few years now and hope to start sharing more of the technical details. Happy to answer any questions or hear some feedback!
Good question. I haven't stumbled upon any, but it's possible they are out there.
The FAA offers a sample client. Depending on your level of skill, the hardest part might be the onboarding with the FAA which includes establishing a site-to-site VPN between your network and theirs. Once that is setup and some other networking is configured, you can use their Java demo client with the credentials they provide to get the hang of it.
The source of data is an event queue, powered by Solace, using Java's JMS standard. You can write a little bit of a Java to parse the incoming messages to get to the XML payload. Honestly I found that part to be the easiest and the Linux networking parts (as part of the VPN) to be the most difficult.
I wrote up a bit of a guide here, it's a little rough and high-level but I hope it helps clarify things a bit if you feel like working with these feeds/data.
I don't think I've seen FlightAware showing aircraft tracks on the ground.
They do show taxi times for arriving/departing aircraft and I wonder if they are stitching that data together using the STDDS feed.
To give you an example, for an arriving aircraft you would get an alert of "wheels down", which is as literal as it sounds: a timestamped event when the aircraft made contact with the ground. In addition to ground position events (allowing the tracking you see here), you would also receive an alert when the aircraft left the tracked movement surface, i.e. transitioned to the gate/ramp area and no longer on an active airport surface.
(Pretty sure I have that correct, it's been awhile since I got back to my STDDS-powered projects. If anybody familiar with STDDS wants to chime in, please do!)
For the curious, all these "alerts" are JMS messages with XML payloads. Pretty much non-stop stream of data, incredibly fun if you're a developer+av geek. For FlightAware-style tracking without using any ADS-B you'll want to subscribe to the SFDPS feed. You'll get position updates every 60 sec for all airborne flights in the US, which last week amounted to ~40GB in my Postgres DB. You can request higher resolution reporting which is every 12s.
Pretty cool. For those wondering the source of the data, it is likely ASDE-X/ASSC streaming in from the FAA's STDDS SWIM feed[0]. The terminal and ground data is especially cool and not as common as the usual flight tracking you see of en-route aircraft.
I'm using the STDDS feed to build a rough "go-around detector" in AWS for approaching aircraft by monitoring each approaching aircraft glide slope and dispatching a SNS notification "if currAlt > prevAlt".
If anyone has any questions about this stuff feel free to reach out, my email is in my profile. The easy part (IMHO) is slurping in the data which only requires a couple small EC2's and RDS, ~$125/mo. The hard part is often the presentation and making demos like these fun and shareable.
HN'ers might appreciate learning the FAA has a developer-friendly program called SWIM [0] to help facilitate getting realtime FAA data (flight plans, weather data, etc) in to the hands of coders. I recently attended the SWIM Developer Workshop at the Volpe Center (near MIT) and really enjoyed it.
There's quite a bit to the SWIM program. The onboarding can be a bit tricky though and you have to know your way around site-to-site VPNs. I recently completed the onboarding process and am receiving live flight plan data -- proposed flights, en route, completed etc.
Would be curious to know if there are others on HN that are working with SWIM data or if there are companies that are hiring in this space.
In 2012, YC invested in a company InstallMonetizer[0] which, from my understanding, helps align software products with bundling other installers for additional revenue.
I agree that this is a tactic that preys on the less tech savvy folks, and share your continued astonishment that reputable companies would engage in these drive-by toolbars/extensions that are bundled in installers.
YC has invested[0] in a company called InstallMonitizer[1] that appears to help developers and advertisers connect in the pay-per-install marketplace.
Sadly it does't seem like a practice that will go away any time soon. I'd like to do some digging around on developer forums and see if any folks have shared their experience and would be able to comment on the amount of extra revenue that they see from such programs.
Migrations[0] are being included in Django 1.7 which is currently under development. For those unaware, Andrew Godwin, the author of South, ran a successful Kickstarter campaign[1] the fund this effort to incorporate migrations in Django core.
Yes, this is surprising. Twilio has a reputation of being awfully transparent -- they had a billing systems issue back in July and handled it quite well, IIRC. They also do a great job with their developer evangelist program.
I wonder if someone has created a pricing matrix/table that updates and compares the SMS/voice API providers. For example, I know Plivo is slightly cheaper for SMS, domestically (USA). A price comparison table might make a good holiday project :)
Although I understand this wouldn't help your case. As you pointed out, the SMS provider would have already increased rates by the time you could be notified of increases.
I should hope not. I'm trying to think of a situation where a developer wants to document a process (ANY process. In this case: building a profitable product) and it would be considered "bad".
> That's why success-story books sell, but you don't see a lot of "will-try-to-do" books.
This is a blog, not a book. Will-try-to-do blog posts are incredibly healthy exercises for the author/do'er and usually provide helpful content for the readers (if the author follows through and documents his or her thought process).
I'm not sure what point you're trying to make here, and (to me) your comments and tone sound dismissive. This fellow has nothing to sell -- it's an exciting endeavor that the author wants to document.
There are quite a few sites/services out there that are catering to the more "seasoned" developer that isn't particularly interested in the race-to-the-bottom jobs found on elance and similar sites.
Sites like gun.io, matchist, ooomf are some good examples. Google those and you'll undoubtedly find a few more.
If you end up using a service that I didn't cover in my post, shoot me an email (if you don't mind). I'd love to keep that blog post updated and evergreen.
Can anyone speak to the demand, or lack thereof, for Django books and educational material? Do large(r) technical publishers not see a viable market for Django books?
I read Two Scoops Of Django (on vacation, no less) and would love to see more books on Django best practices.
Have you looked at Mailgun? (YC alum, now Rackspace)
(disclaimer: I did a guest blog post for them, but that was a result of me being a happy customer before they even contacted me)
They have lists, webhooks/reporting for tracking opens and clicks, HTML support, unsub notifications, etc. All accessible via the API. Poke around the docs and trial, sounds like it could be what you're looking for.
Your comment is unnecessary and I'm shocked that's sitting high on the comments page.
Trivializing the amount of effort and energy this fellow put in to producing/developing this cross-platform mobile app because you disagree with...well, I still don't know what your issue is with the quotes you pulled, care to share?
You might have felt better after posting your comment but you also gave up your chance to add something constructive to the discussion. To paraphrase pg's reaction, it makes me embarrassed for HN when people include projects in their blog profile and this is the sort of response they get.
This is great. Spinning out a consumer-friendly front end to the Zencoder API has been in my sparkfile for too long. I even shared the same idea with HN[0] because I know some bootstrapper is going to make a killing if they can pair it with Stripe and push it out the door in a weekend.
A co-founder of Zencoder even replied to my suggestion mentioning that there's a market for it.
(If you're a hardcore wantrepreneur, here is your idea. Now go forth and build the bootstrapped SaaS biz you always wanted. Send me a link when its done, I know some people that would probably use it. Seems I can't ever find a spare weekend to build it myself)
I'm no MBA or startup guru, but I don't often hear "Founders in very early stage/prototype stage should avoid interacting with their users". Early stage interaction is critical to the success of the product.
>> Having a dedicated person to do it is actually more productive than the founders themselves doing it
If the goal is to get to know their audience needs, and iterate the produce development quickly, then dev-to-dev feedback is absolutely critical. Now is probably not a good time to clog the feedback filter by adding another human to the mix.
You can spend 30 hours working on your product backlog of features that haven't been vetted, or you can spend 30 hours getting to know your audience and discover that NSCoreLibFoo gives people the most trouble with push notifications.
What would convert more: a homepage hero about the three new features you dreamed up OR the fact that PushLayer can fix everything that sucks about NSCoreLibFoo (which half of your audience hates with a passion as you discovered).
Lots of useful information here and kudos to the author for composing a well-written post: summary of the issue, "here's how we fixed it" (with code samples) and concludes with some caveats/things to watch for. (Reminder: quality content like this is how inbound marketing works)
IMHO the official Django docs are some of the best you'll find anywhere of any open source project. My suggestion, though, is to read and re-read and make sure you gather all documented information about $YOUR_TOPIC before formulating your own game plan. For example, here is the chapter on handling static assets in Django:
There are a lot of relevant links on that page that are worth following if you want to make sure you a) get the picture and b) know how you want it to all come together for your project.
Lastly, on the topic of storing assets: if you want to offer a Django-ready library for your new cloud storage startup/setup, consider writing your own file storage handler
I clearly remember when I finally let my initial prototype run on a laptop for a day and ignored it, an old HP laptop just chugging away parsing flight plan data, plugged in and running too hot in my office. By the next day I had > 20 diversions, with no notable weather, and assumed I had a bug in my logic.
Lots more to do here, including adding some weather details and someday predictive analytics. Thanks again for checking it out and the feedback.