Designing and Building Stockfighter, Our Programming Game(kalzumeus.com)
kalzumeus.com
Designing and Building Stockfighter, Our Programming Game
http://www.kalzumeus.com/2015/08/20/designing-and-building-stockfighter-our-programming-game/
159 comments
Out of curiosity, have you gotten any press inquires about your launch from major tech publications? I know that your company is outside of the VC funding publicity bubble but I am curious if anyone has reached out?
This sounds really fun but that is one long post/email. Almost 8000 words! Will read it later though.
Looks like the schedule has already slipped. The email said "2~3 weeks of today" but the blog post says "3 weeks of today"! :P
Looks like the schedule has already slipped. The email said "2~3 weeks of today" but the blog post says "3 weeks of today"! :P
Imagine that! An ambitious development project with a constantly slipping schedule! Maybe we're the Duke Nuke'Em Forever of CTFs! :)
I know how it is. At least you didn't compare it to Daikatana. :)
I swear to god, and this is terrible, but the Daikatana Effect is one of the reasons we've been so quiet about this.
Since we're going to keep rolling out new level sets continuously, we should probably have been a lot more chatty. But I keep telling Erin and Patrick that I want to control expectations, at least until we know exactly what we're shipping in chapter 1.
Of course, now we know what chapter 1 is, so I've lost my excuses to keep them from writing!
Erin & I have an emulator post ("Everything Thomas Got Wrong About Emulators This Time That He Didn't Get Wrong In Microcorruption") brewing, too.
Since we're going to keep rolling out new level sets continuously, we should probably have been a lot more chatty. But I keep telling Erin and Patrick that I want to control expectations, at least until we know exactly what we're shipping in chapter 1.
Of course, now we know what chapter 1 is, so I've lost my excuses to keep them from writing!
Erin & I have an emulator post ("Everything Thomas Got Wrong About Emulators This Time That He Didn't Get Wrong In Microcorruption") brewing, too.
I actually think this is the right way of approaching lots of projects. There's a strong culture which encourages people to release things as early as they have a skeleton, but I've always preferred "Be discreet until you have a fully functional 0.1." There are certainly some initiatives that benefit from immediate fanfare as soon as you have a web page and a 200-line shim (if even that), but not all that many.
I have ADD and was able to read it.
Well apart from "there goes October, dang I had real work planned", I am really welcoming this game. I would normally have jumped all over their compiler "tree", but I have been in my first Finance contract this year and I have not yet understood HFT.
Edit: this post got longer than planned - apologies - but this HFT / algo thing has been bugging at me for ages and this seems a likely thread for knowledgeable folks.
The references to Flash Boys worry me slightly - Lewis sold a false scenario there - it's pretty clear that no fool is going to see Bob buying a million Apple shares in London, then zip over their speed of light towers and front run Bob in Frankfurt for another million Apple shares. Who knows if Bob wants a million? Flash boys it seemed relied too much on a protected type of trader (market maker) getting roasted by the emergence of real competition - the "algo" traders.
It seems incontrovertible- equity market spreads have fallen by about 5/6ths in the past decade (no wonder Lewis' hero trader thought someone was stealing from him) but ... How and Why?
what algo traders actually do is a bit of a mystery to me.
The basics I kind of understand :
There are correlations between shares - spot those and you can make money. Let's say oil goes up, car manufacturers will go down by some amount. Short them and make money. The first time people did this they probably thought they had found a money printer. But then someone else starts playing the same algorithm - so you have to get your orders in taste than that guy - pretty soon you are worrying about the speed of light over glass.
But other correlations exist and get spotted and tried out and ... Well it seems a bit of a weak business model to me to live in P of .95.
So the business model of HFT is a bit weird, and the way their actions feed into reduced spread / liquidity is a bit unclear - but anyway, I look forward to totally screwing up my social life in a few weeks.
Anyone who can enlighten me, please shout.
Edit: this post got longer than planned - apologies - but this HFT / algo thing has been bugging at me for ages and this seems a likely thread for knowledgeable folks.
The references to Flash Boys worry me slightly - Lewis sold a false scenario there - it's pretty clear that no fool is going to see Bob buying a million Apple shares in London, then zip over their speed of light towers and front run Bob in Frankfurt for another million Apple shares. Who knows if Bob wants a million? Flash boys it seemed relied too much on a protected type of trader (market maker) getting roasted by the emergence of real competition - the "algo" traders.
It seems incontrovertible- equity market spreads have fallen by about 5/6ths in the past decade (no wonder Lewis' hero trader thought someone was stealing from him) but ... How and Why?
what algo traders actually do is a bit of a mystery to me.
The basics I kind of understand :
There are correlations between shares - spot those and you can make money. Let's say oil goes up, car manufacturers will go down by some amount. Short them and make money. The first time people did this they probably thought they had found a money printer. But then someone else starts playing the same algorithm - so you have to get your orders in taste than that guy - pretty soon you are worrying about the speed of light over glass.
But other correlations exist and get spotted and tried out and ... Well it seems a bit of a weak business model to me to live in P of .95.
So the business model of HFT is a bit weird, and the way their actions feed into reduced spread / liquidity is a bit unclear - but anyway, I look forward to totally screwing up my social life in a few weeks.
Anyone who can enlighten me, please shout.
Trust me, Patrick agrees with you about Flash Boys. If you haven't yet, read the followup/critique book he cited, even if you haven't read Flash Boys. It's fantastic and chock full of interesting technical details. It's way nerdier than Lewis' book.
That's the "Not so Fast" book? I actually read that before flash boys. Somehow the narrative from flash boys had penetrated the NYT/Economist meme-spreading network and I had accepted it at face value.
However the microstructure book looks ... Deep - but I guess I need to decide if I am in or out :-)
However the microstructure book looks ... Deep - but I guess I need to decide if I am in or out :-)
It is, as I'm fond of saying, The TCP/IP Illustrated of Money. As with Stevens, it's a bit outdated, but still the best reference.
Well I will treat it as I did "TCP/IP illustrated" - buy the whole thing, read most of the middle one and keep it on the shelf for a few years nagging at me guiltily.
I have read Flash Boys but not the books written in response.
How do the other viewpoints explain why the IEX exchange was created and designed specifically to stop HFT? That is the central narrative of the book, and in Lewis's telling it makes complete sense as a way to save the common man from the evil HFT traders. But if Lewis got it all wrong and HFT is perfectly fine, why would a completely new exchange be set up to stop it?
How do the other viewpoints explain why the IEX exchange was created and designed specifically to stop HFT? That is the central narrative of the book, and in Lewis's telling it makes complete sense as a way to save the common man from the evil HFT traders. But if Lewis got it all wrong and HFT is perfectly fine, why would a completely new exchange be set up to stop it?
There are hundreds, probably thousands, of little exchanges nobody has ever heard of. Why set one up to "combat HFT"? Because if you can convince companies to trade on it based on nebulous "anti-HFT" features, you can collect fees without having to compete with the major exchanges, who both bid prices down and also have a 100:1 advantage in engineering and support staff.
Believe it or not, that's not even the "cynical" answer.
Believe it or not, that's not even the "cynical" answer.
There are about 40 active registered ATSs for trading US equities. It's a high number but it's not thousands.
There are many multiples more private exchanges, trading all sorts of electronically tradable instruments. I wasn't referring simply to stock exchanges. I was making a point about the return on investment you can get from developing a trading platform.
You should read the other books, but the short answer is that IEX was created not to save the common man, but to save the big block hedge fund traders.
There is a natural and good competition between HFT market makers who provide liquidity and big block traders who consume it. IEX is purpose built to advantage the big block traders at the expense of HFT market makers.
Note: neither side in this is good or evil, they are just natural opponents.
What I would say is dodgy is presenting yourself as "for the common man" when what you are doing is serving the common man up to hedge funds.
There is a natural and good competition between HFT market makers who provide liquidity and big block traders who consume it. IEX is purpose built to advantage the big block traders at the expense of HFT market makers.
Note: neither side in this is good or evil, they are just natural opponents.
What I would say is dodgy is presenting yourself as "for the common man" when what you are doing is serving the common man up to hedge funds.
A few years back there was a HFT guy that posted a few blog entries describing (and defending through it's liquidity creating properties, IIRC) HFT setups. I don't have the article, but I imagine both tptacek and patio11 (and a slew of others) have a good idea of what I'm talking about, and may be able to dig up a link. You might find it interesting (if it's not already entirely covered by Flash Boys, which I haven't read).
Was that HN's own yummyfajitas?
https://www.chrisstucchio.com/blog/2012/hft_apology.html https://www.chrisstucchio.com/blog/2012/hft_apology2.html
https://www.chrisstucchio.com/blog/2012/hft_apology.html https://www.chrisstucchio.com/blog/2012/hft_apology2.html
Yes! I can confirm that from the name alone, now that you've reminded me. :)
I do wonder about the logic of collecting the email addresses of interested parties, then sending them 2000 and 8000 word (respectively) essays. Do people actually read and value such long emails? Do you not lose a bunch of on-the-fence users each time you send them an email which isn't immediately useful or actionable?
Then again, I know very little about running a startup; patio11 has a track record. Maybe this works as a strategy.
Then again, I know very little about running a startup; patio11 has a track record. Maybe this works as a strategy.
Do you not lose a bunch of on-the-fence users each time you send them an email which isn't immediately useful or actionable?
That's a number which is trivially calculable. I don't know what it will be yet, but if you asked me to pick the over/under, I'm in for 80 out of 16k people on our list at present. An unsubscribe is not "I hate you guys and wish you would die in a fire"; it's the email equivalent of a browser's back button. "Not for me, not right now." Many startups let each unsubscribe punch them in the gut. Don't do that. The only way to avoid unsubscribes is to avoid email. You profoundly don't want to avoid email.
We've sent ~2 emails since... March? If I were a consulting client of myself, I'd have written somewhere between one and two dozen.
Is this email going to produce business results for the company? Hard to say. Wild guess? I get a contact in the next hour from one of the companies who has a stalled LOI saying "Oh great you're shipping let's get that signed." Depending on which company that is, the economics are worth hundreds of thousands of dollars.
Many professional email marketers strongly disagree with writing long emails. I will chalk that up to a respectable professional disagreement. They work very well for me, by any trackable metric and by most of the intangibles. I've never felt like I created much value with the little 125-word-and-a-picture squibs that the industry blasts out by the hundred million; I know (because people tell me) that some people like my writing rather a lot and are disappointed that I don't hit them with 8k words every Friday. As it happens, a portion of Starfighter's list is folks who are in exactly that bucket.
Does that answer the question? Not trying to be very defensive about this, just trying to help other startups out. The modal startup sucks at email marketing. It is, far and away, the highest ROI channel you have and probably the easiest to use well. You should send more email. You should send better email.
That's a number which is trivially calculable. I don't know what it will be yet, but if you asked me to pick the over/under, I'm in for 80 out of 16k people on our list at present. An unsubscribe is not "I hate you guys and wish you would die in a fire"; it's the email equivalent of a browser's back button. "Not for me, not right now." Many startups let each unsubscribe punch them in the gut. Don't do that. The only way to avoid unsubscribes is to avoid email. You profoundly don't want to avoid email.
We've sent ~2 emails since... March? If I were a consulting client of myself, I'd have written somewhere between one and two dozen.
Is this email going to produce business results for the company? Hard to say. Wild guess? I get a contact in the next hour from one of the companies who has a stalled LOI saying "Oh great you're shipping let's get that signed." Depending on which company that is, the economics are worth hundreds of thousands of dollars.
Many professional email marketers strongly disagree with writing long emails. I will chalk that up to a respectable professional disagreement. They work very well for me, by any trackable metric and by most of the intangibles. I've never felt like I created much value with the little 125-word-and-a-picture squibs that the industry blasts out by the hundred million; I know (because people tell me) that some people like my writing rather a lot and are disappointed that I don't hit them with 8k words every Friday. As it happens, a portion of Starfighter's list is folks who are in exactly that bucket.
Does that answer the question? Not trying to be very defensive about this, just trying to help other startups out. The modal startup sucks at email marketing. It is, far and away, the highest ROI channel you have and probably the easiest to use well. You should send more email. You should send better email.
As a data point - I'm one of the people Patrick mentions. I like his writing a lot and would be very happy to see an 8k word essay from him every Friday.
He's on my very short list of 3 or so people who do email marketing particularly well.
One of the other people on that list, incidentally, writes 200-ish word emails every day. I read everything she writes too. There's not a single optimal solution here.
(I'm also very much looking forward to playing the game after reading that post. Given I'm not a professional programmer, I have very little desire to become one, I'm very happy in my current jobs, and I suspect Stockfighter is going to kick my ass until I can wear it like a hat, that's of dubious business value to Starfighter, but I guess I'll probably also recommend it to some folk who may be of more value...)
He's on my very short list of 3 or so people who do email marketing particularly well.
One of the other people on that list, incidentally, writes 200-ish word emails every day. I read everything she writes too. There's not a single optimal solution here.
(I'm also very much looking forward to playing the game after reading that post. Given I'm not a professional programmer, I have very little desire to become one, I'm very happy in my current jobs, and I suspect Stockfighter is going to kick my ass until I can wear it like a hat, that's of dubious business value to Starfighter, but I guess I'll probably also recommend it to some folk who may be of more value...)
Just anecdotal evidence: The two e-mails y'all have sent since I signed up have caused a pavlovian response, I literally left my office and sat down on a couch to read your last e-mail the second I saw it in my inbox.
I do that whenever I see anything from Patrick posted on HN, and getting it delivered in my email box has a similar effect. I would say that I have an irrational enjoyment of reading things that he writes, but I actually think it's pretty rational -- I've never read anything by him that didn't exceed my expectations of information, humor, and valuable content.
To add to the anecdotal evidence I had exactly the same response. Most of the emails are light and fluffy. It's refreshing to receive something with some real meat to it. I've gone from initially being on the fence, to being very excited after the first email, and more so after the second.
For what it's worth, I just signed up for the mailing list. I did this because I want to read articles like this that are mostly high-level technical stuff with a dash of business details. That sort of article is the reason I pay attention to HN.
I'd personally be fine with first finding out about Starfighter itself well after launch. I'll probably check it out, but it's not like I anticipate spending hours on it every week. On the other hand, your blog takes only a few minutes to read, and has great signal:noise.
I'd personally be fine with first finding out about Starfighter itself well after launch. I'll probably check it out, but it's not like I anticipate spending hours on it every week. On the other hand, your blog takes only a few minutes to read, and has great signal:noise.
Counter-anecdote: I got the email, was excited to see that Starfighter was launching soon... but was annoyed that I couldn't quickly skim the rest of the email to get the "tl;dr" version (the basics of what the challenge will involve, or if there's anything else I need to know about the launch, etc). I've been cutting down on my RSS reader lately because there are a lot of things I want to read and there's not time to read them all; I wish the emails were more respectful of my time. Just including a clearly-marked summary, followed by the long version, would make me far happier.
I concur it is about respect for time most of all. Though the technical portions are interesting, they could have been broken out into separate articles for people who want those additional details.
I may not be the target audience (novice coder) but I also feel the writing style would greatly benefit from greater focus.
A sentence that stands out in that regard is "Although my co-founders Thomas and Erin have done substantial work with securing real stock exchanges over their career, I have no particular background in finance." This sentence could be refined to "My co-founders Thomas and Erin have substantial experience securing real stock exchanges". This kind of editing would help the author avoid 8k word essays without a significant loss of content value.
I may not be the target audience (novice coder) but I also feel the writing style would greatly benefit from greater focus.
A sentence that stands out in that regard is "Although my co-founders Thomas and Erin have done substantial work with securing real stock exchanges over their career, I have no particular background in finance." This sentence could be refined to "My co-founders Thomas and Erin have substantial experience securing real stock exchanges". This kind of editing would help the author avoid 8k word essays without a significant loss of content value.
You've reduced the word count with that edit, but you've also lost some texture. The voice and texture of Patrick's writing contribute greatly to its being so absorbing... there's no point in trying to turn a Melville into a Hemingway.
My advice to Patrick would not be to change his style, which is clearly very effective, but to also throw some shorter update emails into the mix so people can keep tabs with less of a time investment if they so choose.
My advice to Patrick would not be to change his style, which is clearly very effective, but to also throw some shorter update emails into the mix so people can keep tabs with less of a time investment if they so choose.
Your comment was too long and was not respectful of my time. Couldn't you have found a way to convey your sense of entitlement to the content that other people freely provide you in a more succinct way?
I’m not quite sure how sharing an anecdote about how I, personally, found it difficult to find information in a particular email conveys a “sense of entitlement”—perhaps my phrasing didn’t come across well? The “respectful of my time” thing is just a phrase I’ve generally heard as shorthand for “make the information easier to access,” and I was merely expressing a wish, not a demand.
Maybe you didn't mean it this way, but "not respectful of my time" is semantically equivalent to "you are disrespecting my time". Disrespect (verb): to show a lack of respect for; insult. It makes it sound like something that the author has done to you. Your other points were fine about your personal experience. But saying the author is not respectful of you is the step too far.
To improve the wording, I'd suggest something that doesn't imply intent on behalf of the author and solely reflects your own personal views. For example: "I wish the emails were shorter as I don't have time for long-form content". It gets your point across that you have limited time and would prefer concise emails, but doesn't infer anything regarding respect (or the lack thereof) towards you.
To improve the wording, I'd suggest something that doesn't imply intent on behalf of the author and solely reflects your own personal views. For example: "I wish the emails were shorter as I don't have time for long-form content". It gets your point across that you have limited time and would prefer concise emails, but doesn't infer anything regarding respect (or the lack thereof) towards you.
Okay. The phrase is in pretty wide use with more casual implications (googling “be respectful of your reader’s time” brings up a lot of marketing email advice pages, for instance), but obviously it's not in as wide of use as I’d thought, so I apologize for overstepping!
I do wish your original comment had been a little less snarky; I’d like to contribute more to HN but it’s not so easy when replies seem to assume entitlement/bad faith/etc.
I do wish your original comment had been a little less snarky; I’d like to contribute more to HN but it’s not so easy when replies seem to assume entitlement/bad faith/etc.
Agreed... sorry for being snarky; I should have explained what I meant originally instead.
I think the difference is that people give authors advice prior to writing an article which amounts to "be mindful that your readers will have to read this". And given that authors have a broad range of readers, it's perfectly fine for an author to consider this and still go ahead with something that's long form and detailed because there's a good chunk of their readership who actively appreciates it (and possibly expects it given patio11's typical article length).
Essentially, claiming the author shows you a lack of respect is disrespectful of the author for the work they put into it. Who has the greater onus to offer respect: someone who actively works to put out free in-depth articles that nobody is under any obligation to read, or those who passively consume those articles? Those who wish to read short announcements can simply say "Great post with lots of details, but is there a channel I can subscribe to which gives me short announcements?" That demonstrates respect.
I think the difference is that people give authors advice prior to writing an article which amounts to "be mindful that your readers will have to read this". And given that authors have a broad range of readers, it's perfectly fine for an author to consider this and still go ahead with something that's long form and detailed because there's a good chunk of their readership who actively appreciates it (and possibly expects it given patio11's typical article length).
Essentially, claiming the author shows you a lack of respect is disrespectful of the author for the work they put into it. Who has the greater onus to offer respect: someone who actively works to put out free in-depth articles that nobody is under any obligation to read, or those who passively consume those articles? Those who wish to read short announcements can simply say "Great post with lots of details, but is there a channel I can subscribe to which gives me short announcements?" That demonstrates respect.
juletide's comment was excellent constructive feedback. It had the right tone[1], explained context[2], and included a specific actionable suggestion[3]. This is the exact sort of feedback good authors want; it was in no way an expression of entitlement.
While patio11 may not follow this particular suggestion for potentially valid reasons that he may or may not have, it was good feedback either way.
[1]-It mostly referred to his or her experience negatively, not the text itself. This seems superficial, but it does matter anyway.
[2]-The RSS point validates the experience, and makes it easier for the author to understand.
[3]-The tl;dr bit was a specific suggestion to improve his or her experience.
While patio11 may not follow this particular suggestion for potentially valid reasons that he may or may not have, it was good feedback either way.
[1]-It mostly referred to his or her experience negatively, not the text itself. This seems superficial, but it does matter anyway.
[2]-The RSS point validates the experience, and makes it easier for the author to understand.
[3]-The tl;dr bit was a specific suggestion to improve his or her experience.
Many professional email marketers strongly disagree with writing long emails.
I might venture an opinion that this is likely because they don't write as well or as interestingly as you do. By a long shot.
I might venture an opinion that this is likely because they don't write as well or as interestingly as you do. By a long shot.
great response, Patrick. And count me in with the folks who love the longer emails. Your writing is invariably thoughtful and thorough.
I think the long email is a great example of how it's better to have 1,000 people who love you rather than 100,000 people who like you.
I think the long email is a great example of how it's better to have 1,000 people who love you rather than 100,000 people who like you.
80 out of 16K is bang on the half-of-one-percent benchmark I've always used for Unsubscribes. If it gets too close to 1%, you'd have cause for concern, and if it gets too close to 0% then you have a spam filter issue that stops a lot o delivery.
One more anecdote: I read the email immediately, shared it with anyone I thought would be interested, and started to count down how long it would be before I could play the game with my 6yo daughter. So, um, it's working for me so far.
Agreed - it just feels nicer to read fewer longer emails - sort of New Yorker vs CNN tickertron.
Of course writing them is a pain so it's an investment. Considering I just gave a friend the "you must write long articles and build an audience" advice, it's amazing how badly I take my own advice...
Of course writing them is a pain so it's an investment. Considering I just gave a friend the "you must write long articles and build an audience" advice, it's amazing how badly I take my own advice...
> An unsubscribe is not "I hate you guys and wish you would die in a fire"
That's not universally true ;)
That's not universally true ;)
I always enjoy the times that the service polls me to find out why I've unsubscribed. But I also have to admit that I also hate it when I check the "I never signed up for this list" and I don't get an apology or something.
Keep sending your long emails. I have butterflies in my stomach. Very excited.
It may work to radicalize a certain segment. I've noticed that some segment of those that are eagerly anticipating something find that the more information they get about it, the more they anticipate it, and the more eagerly they evangelize it to others.
I hadn't heard of this, or microcorruption, before the prior blog post regarding writing the emulator. Then I started microcorruption, and it clicked, and then I spent the next few days talking about both microcorruption and starfighter to anyone I thought might understand my eager rambling.
Let's just say I'm looking forward to reading this...
I hadn't heard of this, or microcorruption, before the prior blog post regarding writing the emulator. Then I started microcorruption, and it clicked, and then I spent the next few days talking about both microcorruption and starfighter to anyone I thought might understand my eager rambling.
Let's just say I'm looking forward to reading this...
I read it, and look forward to more!
I signed up for a tech-heavy game-ish thing, so I expect (and welcome!) tech-heavy content.
I signed up for a tech-heavy game-ish thing, so I expect (and welcome!) tech-heavy content.
I find the emails verbose, and don't really care for the for the prose. That being said, at least in my head, when its public i'll say 'here goes'.
To be one data point.
I read everything Patrick (and Amy Hoy and Brennan Dunn and Ramit Sethi and a few others) emails me. All tend to be long form (typically x,000 words) essays and/or sales letters.
My ROI on time spent reading has been totally positive and I have no plans to stop.
I read everything Patrick (and Amy Hoy and Brennan Dunn and Ramit Sethi and a few others) emails me. All tend to be long form (typically x,000 words) essays and/or sales letters.
My ROI on time spent reading has been totally positive and I have no plans to stop.
Interesting sentence in the article - "Trading systems touch low-level coding, networking (all seven layers of hello OSI Model), APIs, end-user UI, databases, embedded systems, enterprise web frameworks, cutting-edge programming language research, Big Data, state management, etc etc.".
However, whenever the topics of programming in finance come up in HN, I see a general tone of disdain somewhere along the lines of "why do you want to waste your life in finance when you could be starting a start-up in Silicon Valley?". Of course I am generalizing a bit here, but I have always felt that finance provides incredible challenge to technical people - especially those with a background in mathematics or physics and interested in programming.
However, whenever the topics of programming in finance come up in HN, I see a general tone of disdain somewhere along the lines of "why do you want to waste your life in finance when you could be starting a start-up in Silicon Valley?". Of course I am generalizing a bit here, but I have always felt that finance provides incredible challenge to technical people - especially those with a background in mathematics or physics and interested in programming.
I have only a years perspective on the sector now, but I think finance has started down a road all sectors must tread over the next decade or two. We will not escape software eating our world, and while most banks have had too much money and so not confronted many of these upcoming realities, they have had twenty years of faster and faster software-isation- and I can reasonably state that finance are mostly software houses that think they are banks.
If you can't code, your future there is bleak. If you can its not that much better. And this is where we are all heading.
Yes there are challenges - but it's not often technical but organisational, project, people and process challenges that are the main issues. Those interesting challenges for good STEM people will be in transport, farming and pharmacy too by 2020.
Wages might be lower :-)
If you can't code, your future there is bleak. If you can its not that much better. And this is where we are all heading.
Yes there are challenges - but it's not often technical but organisational, project, people and process challenges that are the main issues. Those interesting challenges for good STEM people will be in transport, farming and pharmacy too by 2020.
Wages might be lower :-)
Oops commented too fast without reading the entire article. It says later -
"I hope people will, in the course of robbing them blind, actually learn what major Wall Street institutions do for a living and come to a more nuanced understanding of their position in society."
When I read the title "stockfighter," I thought it would be a game about joining a tech company 10 years past the IPO and you have to work your ass off for .000001% in options that'll take a year to afford once you factor in rent.
I'll admit to not finishing that - it was cool to see Patrick geeking out about the impressive tech he's been working on, rather than marketing, though.
Good luck, guys!
Good luck, guys!
We solved this in a roughly similar way to a real stock exchange: the order book is persisted entirely in memory, in Go.
What happens if your process crashes? how do you rebuild the order book?
Great work btw guys, can't wait to get started!
What happens if your process crashes? how do you rebuild the order book?
Great work btw guys, can't wait to get started!
At that moment, we can't recover from that and players connected to one of the crashed exchanges (1 server process: N exchanges) are told to restart levels, which is not an ideal outcome but it's essentially free. I hope to eventually be able to do something like use our persisted-out-of-memory (e.g. to NSQ) tapes of the orders, bring up the crashed exchanges, and fire every order back into it in order, which (since it's a stock exchange) should result in the same orderbook it crashed with minus whatever we lost during the crash.
After we can do that reliably, then I will strongly consider making an advanced mode level where a crash / market reset / rollback of trades is a planned feature of the level (they are absolutely a thing which happens in real life) and dealing with it from the player's perspective is the main task. (e.g. What would you do if you got a message from the stock exchange saying, basically, "We lost data. How much? Umm, not sure, the last orderId we have is X. Orders after X are no longer on our books but, um, I guess there exists the possibility that they will nonetheless settle successfully if they matched before we went down. ... Good luck to you!")
After we can do that reliably, then I will strongly consider making an advanced mode level where a crash / market reset / rollback of trades is a planned feature of the level (they are absolutely a thing which happens in real life) and dealing with it from the player's perspective is the main task. (e.g. What would you do if you got a message from the stock exchange saying, basically, "We lost data. How much? Umm, not sure, the last orderId we have is X. Orders after X are no longer on our books but, um, I guess there exists the possibility that they will nonetheless settle successfully if they matched before we went down. ... Good luck to you!")
I imagine you've read it, but if you haven't, you may want to read about the LMAX architecture (from which they created the well-known Disruptor) because they dealt with this exact same problem: http://martinfowler.com/articles/lmax.html
For other readers who aren't aware of that pattern, it's called Event Sourcing. (currently implementing it using Firebase as a combined message bus + storage)
Ref: http://ookami86.github.io/event-sourcing-in-practice/#title....
Ref: http://ookami86.github.io/event-sourcing-in-practice/#title....
Maybe that's one of the attack vectors we are supposed to discover ourselves; trigger a kernel panic?
Please don't trigger a kernel panic in one of the stock exchanges; you're on a server with many other actual humans even if they're not hooked up to your stock exchange. More guidance on "bugs you should not attempt to exploit" versus "Things which are totally in scope" at a later date -- Thomas is writing it.
I suspect breaking out the simulated world into the real world will become an interesting meta-challenge, whether you like it or not :).
Maybe provide a public QA server for people who are interested in that as another "game". "Can you break our stock exchange?"
Dumb idea. If you can trigger a kernel panic, you can do things much more interesting than triggering a kernel panic.
Looking forward to figuring out the REST API for the GM server, then writing a command-line client for that and the exchange itself. Then, presumably, we can play the trading tree of Stockfighter mostly or entirely from the terminal.
Maybe it's from the length of the message or the general impressiveness of the project, but the first thought that came to mind was, "Wow, I'd have to be unemployed to have enough time to work on this - there's no way I could tackle these challenges in the real world otherwise."
Worse, if this is something that's designed to showcase one's technical skills to potential employers, I don't imagine there's a way for the system to distinguish between "took a long time solving challenges because just not smart enough" vs. "took a long time solving challenges because have a day job and a life outside of work/tech".
Since this is designed, essentially, to showcase how smart someone is to potential employers, assuming solving time is a factor, it's almost better not to compete until you know you'll definitely have lots of time to devote to it, because otherwise there is the risk of taking too long and looking not-good.
I'll add it to the long list of "things I really, really wish I had time for" - or I'll assign it to clone #1023 once that becomes viable :-/
Worse, if this is something that's designed to showcase one's technical skills to potential employers, I don't imagine there's a way for the system to distinguish between "took a long time solving challenges because just not smart enough" vs. "took a long time solving challenges because have a day job and a life outside of work/tech".
Since this is designed, essentially, to showcase how smart someone is to potential employers, assuming solving time is a factor, it's almost better not to compete until you know you'll definitely have lots of time to devote to it, because otherwise there is the risk of taking too long and looking not-good.
I'll add it to the long list of "things I really, really wish I had time for" - or I'll assign it to clone #1023 once that becomes viable :-/
We want it to be playable by people who have demanding commitments like families, jobs, etc. This affects our level design (the first six will actually time out after 30 minutes; they're beatable with the right script in seconds or minutes after you get it working or by hand in ~10 with play at least as good as mine) and how we go about executing on the business.
Ultimately the mechanics of our matchmaking come down to "Starfighter spits a heck of a lot of signal about you at Thomas/Erin/Patrick; we try to decide if it makes sense to introduce to to clients and, if so, to which." All of us are working parents. We will certainly not penalize people for prioritizing piano lessons over being one of the first 5 people to crack a new level.
I won't commit to "I'll never look at time between a level is first opened and when it is solved" but, as somebody who has fed his family with code for as long as I've had a family, that strikes me as being among the least interesting things I could possibly learn about an engineer's ability.
Ultimately the mechanics of our matchmaking come down to "Starfighter spits a heck of a lot of signal about you at Thomas/Erin/Patrick; we try to decide if it makes sense to introduce to to clients and, if so, to which." All of us are working parents. We will certainly not penalize people for prioritizing piano lessons over being one of the first 5 people to crack a new level.
I won't commit to "I'll never look at time between a level is first opened and when it is solved" but, as somebody who has fed his family with code for as long as I've had a family, that strikes me as being among the least interesting things I could possibly learn about an engineer's ability.
After para 1, I was going to ask "what's to stop someone from tossing a solution in a pastebin and sharing it around?" Related, "what's to stop someone from sharing the questions ahead of time?"
From the second, it sounds like you guys will seek out the top performers and weed out the bozos. I like it! Are you looking at a traditional recruiter fee model or something else? Apologies if this is posted somewhere else.
From the second, it sounds like you guys will seek out the top performers and weed out the bozos. I like it! Are you looking at a traditional recruiter fee model or something else? Apologies if this is posted somewhere else.
The big problem isn't some bozo posting it to pastebin, it's shady "recruiters" that are set up to make their candidates look awesome. This is extremely difficult to detect without on-site validation.
I'm dumb and slow- can I still submit a solution to know if I'm correct, even if I get a dingus label?
Hiring is an incredibly difficult problem to solve at any scale, though it's different on different scales. As times goes on my impression of how hard the task is keeps shifting upwards.
It's a huge deal for both parties. I think that if there is a way to spend money and/or time to solve it, both sides can come out on top with room to spare.
There's plenty of evidence that both sides are willing to invest a lot in it. For employers that's easily measurable in fees they pay for recruiting services (which usually suck), for referrals and such. For Applicants, it's harder to say. But, people do whole degrees and certifications in order to appeal to employees. Often that stuff is at a total tangent to acquiring useful skills or pursuing interest. It's basically time an money invested in being (more easily) hirable.
Basically, anything that can improve hiring meaningfully may be worth this kind of big investment for an employee as well as an employer. One way it could pay off is higher salary. A higher quality, de-risked hire is worth more.
I think one of the things that rely needs to happen though is that applicants need to be able to evaluate their chances. Spending 3 days interviewing may be no big deal in the context of a job that will last 3 years, but it is in the context of some unknowable probability of getting said job. A third party would probably be helpful here.
It's a huge deal for both parties. I think that if there is a way to spend money and/or time to solve it, both sides can come out on top with room to spare.
There's plenty of evidence that both sides are willing to invest a lot in it. For employers that's easily measurable in fees they pay for recruiting services (which usually suck), for referrals and such. For Applicants, it's harder to say. But, people do whole degrees and certifications in order to appeal to employees. Often that stuff is at a total tangent to acquiring useful skills or pursuing interest. It's basically time an money invested in being (more easily) hirable.
Basically, anything that can improve hiring meaningfully may be worth this kind of big investment for an employee as well as an employer. One way it could pay off is higher salary. A higher quality, de-risked hire is worth more.
I think one of the things that rely needs to happen though is that applicants need to be able to evaluate their chances. Spending 3 days interviewing may be no big deal in the context of a job that will last 3 years, but it is in the context of some unknowable probability of getting said job. A third party would probably be helpful here.
Not exactly what you are looking for, but maybe close; check http://InterviewKickstart.com.
I believe that most engineers fail in interviews not because they are incapable, but because they underestimate the prep required. And prep is difficult for experienced engineers because the employers expect a lot of academic stuff which they have forgotten and need to refresh in a short time.
We hence designed a solid program, which takes you through interview important CS in less than two months. We are having fantastic results with that. Good Engineers who would normally not crack into stellar companies, are now cracking in.
This way, candidates don't need additional expensive degrees and employers waste less time hiring . At scale, this is one way to fix the hiring problem.
I believe that most engineers fail in interviews not because they are incapable, but because they underestimate the prep required. And prep is difficult for experienced engineers because the employers expect a lot of academic stuff which they have forgotten and need to refresh in a short time.
We hence designed a solid program, which takes you through interview important CS in less than two months. We are having fantastic results with that. Good Engineers who would normally not crack into stellar companies, are now cracking in.
This way, candidates don't need additional expensive degrees and employers waste less time hiring . At scale, this is one way to fix the hiring problem.
If we get this right, it will hopefully be the case that we eventually have so much content that most people won't ever get through all of it.
I don't think you have much to worry about in chapter 1, though.
I don't think you have much to worry about in chapter 1, though.
I think you're looking at Starfighter as a much more adversarial thing than it's designed to be. From what they've said, it's my impression that it's supposed to be a fun challenge that demonstrates marketable skills, not a pop quiz. I don't think they intend to grade you on whether you spend all your time on it or do it casually or whatever.
> "took a long time solving challenges because have a day job and a life outside of work/tech"
What's the problem with just saying that? "I worked on this during my commute to work" or "I did this cool thing on my weekends" or "I tinkered with this in the evenings"
There is a lot more you can do with 1h per day than you think. If you're consistent.
Hell, I am currently making decent money on the side with book sales by working on writing and marketing for about an hour a day. My biggest problem has gone from "Fuck I have no time" to "Fuck, I've run out of ideas to try".
What's the problem with just saying that? "I worked on this during my commute to work" or "I did this cool thing on my weekends" or "I tinkered with this in the evenings"
There is a lot more you can do with 1h per day than you think. If you're consistent.
Hell, I am currently making decent money on the side with book sales by working on writing and marketing for about an hour a day. My biggest problem has gone from "Fuck I have no time" to "Fuck, I've run out of ideas to try".
> There is a lot more you can do with 1h per day than you think. If you're consistent.
Or, for some of us, there's a lot more you can do with 1h per day than you think, if you're willing to let that one hour consume your thoughts for the rest of the time you're awake. That's much more my concern than anything else --- when I'm at work, I need to think about work.
Or, for some of us, there's a lot more you can do with 1h per day than you think, if you're willing to let that one hour consume your thoughts for the rest of the time you're awake. That's much more my concern than anything else --- when I'm at work, I need to think about work.
”There is a lot more you can do with 1h per day than you think. If you're consistent.”
I’ll second that. This fact is a great opportunity e.g. for people who get bored on their train commute or similar.
I’ll second that. This fact is a great opportunity e.g. for people who get bored on their train commute or similar.
I felt the exact same way when reading the email. To me it sounded like each level is a sizeable/realistic project for which most companies would probably pay people to develop.
You found us out! This is actually an evil scheme we've cooked up to provide YC companies with free trading platform implementations crowdsourced from unwitting Starfighter participants!
So it turns out that Ender's Game is actually kind of 'the sharing economy'!
I know you're kidding, but wasn't that sort of the plot of "The Last Starfighter"? Now I know why you call it that. Heh.
By the time Alex leaves for space he is not unwitting, he has accepted that the bad guys that followed the good guy recruiter will try to kill him and his family regardless of the choice he makes.
[deleted]
My order came back saying that I had bought 0 shares. And the quote from the exchange now read “500 shares available at $120.”
Maybe I don't get the full picture, but it sounds like you have to poll via REST API to query the book? Modern exchanges just give you a feed of updates (adds, deletes, trades), so you don't run into situations like this. You can just check the feed to see what happened.
Maybe I don't get the full picture, but it sounds like you have to poll via REST API to query the book? Modern exchanges just give you a feed of updates (adds, deletes, trades), so you don't run into situations like this. You can just check the feed to see what happened.
The feed is hooked up via a websocket to the UI, but not persisted anywhere on the client side. From my eye's perspective, the cancel never registered. If I had had wireshark open I would have seen it. (I encourage folks who understand why this is not desirable to build their own trading UIs using our provided e.g. websockets.)
[deleted]
Patrick, you should just admit it that the reason you are starting off with a financial theme is so you can include a level about breaking into a Bitcoin exchange. :-)
More seriously, to somewhat address the "8000 words?" folks, the runup to Starfighter reminds me a lot of StackOverflow. Both seem like projects that would have flopped if not for the creators' existing audience and great care in cultivating excitement by explaining in depth the need, the goals, and the philosophy of the project. It'd be interesting to revisit the early StackOverflow marketing and compare the two more closely.
More seriously, to somewhat address the "8000 words?" folks, the runup to Starfighter reminds me a lot of StackOverflow. Both seem like projects that would have flopped if not for the creators' existing audience and great care in cultivating excitement by explaining in depth the need, the goals, and the philosophy of the project. It'd be interesting to revisit the early StackOverflow marketing and compare the two more closely.
There's a lot of things going on here. But what is the point of Arduino?
AVR. Not Arduino.
Starfighter is the successor to Microcorruption, a microcontroller-based memory corruption CTF.
We wrote Microcorruption to provide hands-on learning for memory corruption exploits.
We used microcontrollers instead of full-featured CPUs so we could (a) lose most of the irrelevant details of operating systems and (b) fit the whole memories of thousands of emulated CPUs into a single server process. We also liked the idea of leveling the playing field, so that pros with large libraries of X86 tools didn't have too much of an advantage.
Microcorruption used MSP430. We picked MSP430 because it's the simplest, most elegant MCU instruction set for which GCC support was trivial.
Starfighter uses AVR because we didn't want to use MSP430 twice, and AVR is the most popular MCU instruction set among hobbyists. It turns out AVR does other interesting things for a security CTF, because unlike MSP430, it's a Harvard architecture.
The AVR levels are intertwined with the trading levels, so that the objectives for the AVR levels are more interesting than those of Microcorruption. You won Microcorruption mostly by figuring out how to exploit simple memory corruption bugs. My hope is that Starfighter requires a bit more in-sim programming.
Starfighter is the successor to Microcorruption, a microcontroller-based memory corruption CTF.
We wrote Microcorruption to provide hands-on learning for memory corruption exploits.
We used microcontrollers instead of full-featured CPUs so we could (a) lose most of the irrelevant details of operating systems and (b) fit the whole memories of thousands of emulated CPUs into a single server process. We also liked the idea of leveling the playing field, so that pros with large libraries of X86 tools didn't have too much of an advantage.
Microcorruption used MSP430. We picked MSP430 because it's the simplest, most elegant MCU instruction set for which GCC support was trivial.
Starfighter uses AVR because we didn't want to use MSP430 twice, and AVR is the most popular MCU instruction set among hobbyists. It turns out AVR does other interesting things for a security CTF, because unlike MSP430, it's a Harvard architecture.
The AVR levels are intertwined with the trading levels, so that the objectives for the AVR levels are more interesting than those of Microcorruption. You won Microcorruption mostly by figuring out how to exploit simple memory corruption bugs. My hope is that Starfighter requires a bit more in-sim programming.
I think that is just to refer to the instruction set that is being emulated (8-bit AVR - see Erin Ptacek's blog post about it on sockpuppet.org), used by the microcontroller on many Arduino boards
So, do I prioritise Stockfighter or TIS-100?
Decisions, decisions :)
Decisions, decisions :)
This is awesome. I still don't see how this isn't easily cheatable. What's stopping me from writing up a little tool that can win these levels and providing minimal instructions?
At one point, it even says they will award people that write OSS for it. Doesn't that just further reduce the effort needed to pass levels?
Not to be negative. I'm very, very much looking forward to this.
At one point, it even says they will award people that write OSS for it. Doesn't that just further reduce the effort needed to pass levels?
Not to be negative. I'm very, very much looking forward to this.
I have some ideas jumbling in my head about this, but it's getting late and I'm not sure they're maximally coherent. Briefly:
a) What's the worst thing that possibly happens? Someone scams a row out of our database? It has more rows. 10,000 someone scams a row out of our database? I will write a for loop which scams a million rows out of our database. What about the actual world or Starfighter's capability to positively impact our players, our customers, the industry, or our own situations just got worse as a result of that for loop?
2) I think you will find that a solution which beats level six also sails over the Turing Test, so if you have a tiny script which does so, I recommend not burning it on Starfighter, but would be honored if you decided to try.
3) At the end of the day, the model is not "Someone pays us for a PDF automatically spat out of our database", the trivialization of which PDF would be dangerous to the business model. It is "We become sufficiently enthusiastic about someone's performance that we successfully introduce them to a company which is not us and then that company hires them." Is the threat model "You use a script which trivializes our levels to win a free conversation with Patrick about your career?" Anyone can get that! At any time! Save yourself a git clone; just send me an email!
The more interesting question is then, given that you've learned absolutely nothing from Starfighter because you've git cloned your way to the finish line, whether you can (in thirty minutes with me, or Erin, or Thomas) convince us "Yes, this person is totally someone I should burn my stack of accumulated karma with the CTO of $CLIENT with to suggest they interview them."
a) What's the worst thing that possibly happens? Someone scams a row out of our database? It has more rows. 10,000 someone scams a row out of our database? I will write a for loop which scams a million rows out of our database. What about the actual world or Starfighter's capability to positively impact our players, our customers, the industry, or our own situations just got worse as a result of that for loop?
2) I think you will find that a solution which beats level six also sails over the Turing Test, so if you have a tiny script which does so, I recommend not burning it on Starfighter, but would be honored if you decided to try.
3) At the end of the day, the model is not "Someone pays us for a PDF automatically spat out of our database", the trivialization of which PDF would be dangerous to the business model. It is "We become sufficiently enthusiastic about someone's performance that we successfully introduce them to a company which is not us and then that company hires them." Is the threat model "You use a script which trivializes our levels to win a free conversation with Patrick about your career?" Anyone can get that! At any time! Save yourself a git clone; just send me an email!
The more interesting question is then, given that you've learned absolutely nothing from Starfighter because you've git cloned your way to the finish line, whether you can (in thirty minutes with me, or Erin, or Thomas) convince us "Yes, this person is totally someone I should burn my stack of accumulated karma with the CTO of $CLIENT with to suggest they interview them."
Ohhh. Ok I get it. I was under the impression that the goal was to have a Starfighter profile publicly, like a new type of credential/cert. Which, I guess now that I'm thinking about it, wouldn't be much better than those MS certs. If it's just a way to generate great leads for you then it makes total sense. And I imagine even if there are public scoreboards, the end point of your company isn't to provide an embeddable seal like those SSL resellers.
Thanks for taking the time to respond.
Thanks for taking the time to respond.
> The more interesting question is then, given that you've learned absolutely nothing from Starfighter because you've git cloned your way to the finish line, whether you can (in thirty minutes with me, or Erin, or Thomas) convince us "Yes, this person is totally someone I should burn my stack of accumulated karma with the CTO of $CLIENT with to suggest they interview them."
So, how is this any different from $CODER burning their accumulated karma directly with CTO of respective $CLIENT ? Why should we expect them to behave any differently because it's you, Erin or Thomas?
So, how is this any different from $CODER burning their accumulated karma directly with CTO of respective $CLIENT ? Why should we expect them to behave any differently because it's you, Erin or Thomas?
Not the coder's karma. Patrick's karma.
Patrick is saying that if someone is incompetent gets through with scripts, they'll still have to pass a chat with Patrick before Patrick puts his reputation on the line with a CTO.
Patrick is saying that if someone is incompetent gets through with scripts, they'll still have to pass a chat with Patrick before Patrick puts his reputation on the line with a CTO.
That's my point. Why can't all of the IT recruiters that exist today simply just do the same thing? I'm failing to see how this is any different than the current funneling/filtering system?
They totally could do the same thing. They can come up with their own outreach programs to start meeting lots of developers at scale on the Internet. They can invent systems that reveal programming aptitude that isn't on the resumes of people who have been stuck doing line-of-business J2EE apps at insurance companies, or QA for enterprise products. They can hire people that can competently talk to developers on the phone or in person and actually learn things about them.
Maybe, with companies like ours and TripleByte in the mix, that's what they'll start doing. Won't that be interesting?
We're a bootstrapped startup founded by 3 veteran consultants. Consultancies always compete for gigs; that's why we have to write proposals. This isn't a consultancy, but we've learned our lessons. We sincerely do not give a shit about what other incumbent recruiting companies do. Our approach will work or it won't, and if it works, we're going to grind on it and continuously improve on it, and if other people do the same thing, eventually we'll start having meetups and sharing beers with them.
Maybe, with companies like ours and TripleByte in the mix, that's what they'll start doing. Won't that be interesting?
We're a bootstrapped startup founded by 3 veteran consultants. Consultancies always compete for gigs; that's why we have to write proposals. This isn't a consultancy, but we've learned our lessons. We sincerely do not give a shit about what other incumbent recruiting companies do. Our approach will work or it won't, and if it works, we're going to grind on it and continuously improve on it, and if other people do the same thing, eventually we'll start having meetups and sharing beers with them.
My impression is that Stockfighter is a reaction to the fact that recruiters are exceptionally lazy and prefer a basic keyword match on a resume to actually checking anything. If Stockfighter works well then I would guess IT recruiters will do the same thing, else they'll be competing against a company with a tremendous advantage over them and they'll lose lots of their competitive advantage.
To that end, it might be worthwhile starting a company to build something similar to Stockfighter that can easily be whitelabelled for recruitment companies.
To that end, it might be worthwhile starting a company to build something similar to Stockfighter that can easily be whitelabelled for recruitment companies.
Maybe cause IT recruiters don't have the karma? I'm sure there's some good ones, but most stories I've heard are pretty bad. They aren't actually developers and can't vet people. Patrick et al can. And recruiters that can do this? Well I imagine they're supply constrained.
Random devs don't have karma either.
Now that Starfighter's portrayed as a recruiting system, it makes total sense.
Random devs don't have karma either.
Now that Starfighter's portrayed as a recruiting system, it makes total sense.
Don't want to speak for them, but why would they want to stop you from writing that tool?
Remember their business model, it isn't "get people to the end of the game following the rules", it's "identify people with talent to recruit". Your tool would just be evidence of the latter.
Remember their business model, it isn't "get people to the end of the game following the rules", it's "identify people with talent to recruit". Your tool would just be evidence of the latter.
The fear is that it would introduce noise into the signal of those who are using the tool, rather than the creator of the tool.
The only thing we're worried about is people spoiling the challenges for other people (it's not a problem for us if people get help clearing levels, but it's a problem if people find ways to suck all the fun out of it for everyone else).
We like the idea of people publishing and sharing tools; they did for Microcorruption, too. We'll have more to say about this.
We like the idea of people publishing and sharing tools; they did for Microcorruption, too. We'll have more to say about this.
I have to say, this is both impressive and infuriating; they beat me to it!
My effort was started in Nov. 2014, and is similarly written in Go. Clocking in at ~1600 LoC, it is certainly nothing special. I have no background in finance and only a cursory understanding of the technical aspects of the exchanges. It's mostly working, but I've really stalled because I can find neither a real purpose to continue, nor any resources on building interesting models. All it does is make financial noise (random trades between similarly irrational parties)
All in all, I'm happy for the guys working on this for doing what I apparently couldn't.
If anyone's interested, I'd be happy to post it to Github.
My effort was started in Nov. 2014, and is similarly written in Go. Clocking in at ~1600 LoC, it is certainly nothing special. I have no background in finance and only a cursory understanding of the technical aspects of the exchanges. It's mostly working, but I've really stalled because I can find neither a real purpose to continue, nor any resources on building interesting models. All it does is make financial noise (random trades between similarly irrational parties)
All in all, I'm happy for the guys working on this for doing what I apparently couldn't.
If anyone's interested, I'd be happy to post it to Github.
After a bit of stage fright (and Git/Markdown reeducation), here it is.
https://github.com/Efruit/marqit
https://github.com/Efruit/marqit
You'll be a little bit late to the party, but why not repurpose into a bitcoin exchange :) Everyone else seems to have done it.
I'm interested.
I'm very interested in Starfighter and am looking forward to the release. However, there's something from your original announcement that still worries me. In this post you said:
The designed operation of our trading levels is “Play them, lose horribly, write a program, reset the level, run the program, have it crash with a bug, fix and re-run, lose modestly, adjust your approach, re-start the level, run the program, win.” (Folks have asked whether we’ll hold this against you. No, no, of course not. We log everything under the sun, but losing/restarting levels is planned. That capability is an advantage of doing this in a rich simulation as opposed to real life — you can go Knight Capital as many times as you want and no one loses their job!)
This feels good. But earlier, this was said [1]:
We will track player behaviors and skill at incredible levels of granular detail, instrumenting them like they were built by the Orwellian MiniPeace... We can tell you exactly what happened when your candidates tried to implement a REST API. We can compare their performance against hundreds of other talented engineers (including your current employees) on the same task.
Can you please elaborate on what exactly is going to be tracked? Is any of this configurable by the player? Personally, I'm going to have a hard time enjoying and learning if I feel like every mistake I make, or every extra piece of time something takes is going to be judged and count against me later on.
[1] http://www.kalzumeus.com/2015/03/09/announcing-starfighter/
The designed operation of our trading levels is “Play them, lose horribly, write a program, reset the level, run the program, have it crash with a bug, fix and re-run, lose modestly, adjust your approach, re-start the level, run the program, win.” (Folks have asked whether we’ll hold this against you. No, no, of course not. We log everything under the sun, but losing/restarting levels is planned. That capability is an advantage of doing this in a rich simulation as opposed to real life — you can go Knight Capital as many times as you want and no one loses their job!)
This feels good. But earlier, this was said [1]:
We will track player behaviors and skill at incredible levels of granular detail, instrumenting them like they were built by the Orwellian MiniPeace... We can tell you exactly what happened when your candidates tried to implement a REST API. We can compare their performance against hundreds of other talented engineers (including your current employees) on the same task.
Can you please elaborate on what exactly is going to be tracked? Is any of this configurable by the player? Personally, I'm going to have a hard time enjoying and learning if I feel like every mistake I make, or every extra piece of time something takes is going to be judged and count against me later on.
[1] http://www.kalzumeus.com/2015/03/09/announcing-starfighter/
Remember their incentives. They want you to perform well (or more precisely they want performing well in their game to correlate with you being a good employee).
It is not in their best interests to count against you, quite the contrary they want to sell you to their clients.
It is not in their best interests to count against you, quite the contrary they want to sell you to their clients.
What is a better way to sell a candidate to clients than to say she is in the top 1%? They have every reason to count out as many people out as possible. Nobody looking at this kind of service wants to hire candidates who are not the best.
Because the population of devs who don't play at all is gigantic. So they can make the argument that "this person played and got through 65% which has x correlation with being a good employee" and that still puts that person in the elite status.
Just knowing what stockfighter is puts them into some extremely small percentage of devs, which in the current market is a drop in the bucket for demand.
Just knowing what stockfighter is puts them into some extremely small percentage of devs, which in the current market is a drop in the bucket for demand.
That's not how our business model works. We're not going to make extra money based on where people rank.
I understand that, but if you want to sell any kind of screen you have to represent that the screen discriminates "better" candidates from "worse" ones. The people who are willing to pay for this screening want the top 1% or 10% or whatever (even if the way it goes down is that they request data dumps and then make up the scoring function themselves). That is how the whole industry works, it's not your fault and you almost certainly can't do anything about it.
Edit: even if you present yourself as a recruiter, it doesn't make any real difference. The value you are adding and charging for is that you are handling part of the recruitment process so that your clients don't have to. Nobody wants a recruiter who doesn't hand them the best candidates.
Edit: even if you present yourself as a recruiter, it doesn't make any real difference. The value you are adding and charging for is that you are handling part of the recruitment process so that your clients don't have to. Nobody wants a recruiter who doesn't hand them the best candidates.
We're not selling a screen. We're not a job interview. We're just smarter recruiters.
Pardon me. You're selling recruitment services, which happen to involve preliminary screening through Stockfighter. I am venturing the very risky speculation that these recruitment services would have no value to your clients if they were not somehow selective. If all you want is to sift through a million resumes, that has been cheaply available for years.
Sorry, it's just not that simple.
I see that you aren't finding this productive to continue. That's okay. Thanks for your responses anyway.
Answered up here: https://news.ycombinator.com/item?id=10095199
I think the key point is that they don't want the top 10% or 1% or .1%, they want as many people as possible above some particular skill level. The client is just as happy if that happens to be 50% of players or whatever.
They don't want "the best" candidates, they just want competent ones.
They don't want "the best" candidates, they just want competent ones.
Yes, that's closer to the idea.
In particular: clients cannot pay us for a list of people who have exceeded some threshold in SF.
In particular: clients cannot pay us for a list of people who have exceeded some threshold in SF.
Clients would be irrational to pay for a recruiting service which removed 0 bits of uncertainty as to the quality of presented candidates (by the client's definition of quality). It's safe to generalize that recruiting services which survive reduce uncertainty or at least give a strong impression that they do, i.e., they discriminate among candidates in some way (I don't mean discrimination in the legal or pejorative sense).
This isn't an accusation of wrongdoing, just a simple observation about reality. The incentive for a recruiter using a game like this really isn't for every candidate (err, I mean, player) to be seen as equally recommendable. Even if you were so suicidally idealistic that you saw it that way, clients wouldn't. But you aren't stupid, you know that.
quote66555 is right to think that there is an incentive for extensively recorded details to be used as a signal of his individual "quality" as a candidate. And if he is not in the top n% (where n < 100) then at best he will not be recommended. So it is possible that his little delays or minor fuckups will count against him, assuming he cares about being recommended. It's a reasonable thing for him to think. He isn't scare-mongering.
This isn't an accusation of wrongdoing, just a simple observation about reality. The incentive for a recruiter using a game like this really isn't for every candidate (err, I mean, player) to be seen as equally recommendable. Even if you were so suicidally idealistic that you saw it that way, clients wouldn't. But you aren't stupid, you know that.
quote66555 is right to think that there is an incentive for extensively recorded details to be used as a signal of his individual "quality" as a candidate. And if he is not in the top n% (where n < 100) then at best he will not be recommended. So it is possible that his little delays or minor fuckups will count against him, assuming he cares about being recommended. It's a reasonable thing for him to think. He isn't scare-mongering.
Look: this is easy.
We promise: your data is private. Nobody sees anything you do unless you want them to. It doesn't matter what employer incentives are; revealing a log of how you've engaged with this CTF would involve us breaking a promise to you.
Meanwhile: what we're doing as a business is opt-in. If you don't think we're trustworthy, there is absolutely no reason to work with us to help find jobs. Just goof around with the game and ignore the fact that we do other stuff. We are no company's job interview.
>>>Most of the best performers on this CTF aren't going to get placed by us.<<<
It's frustrating talking about this, because we've had our hands absolutely full just getting a CTF that's better than Microcorruption done. We haven't had a spare moment to think about how to make money from it.
Catch me sometime when I'm not debugging a weird interaction between my emulated SPI bus and my emulated UART. :)
We promise: your data is private. Nobody sees anything you do unless you want them to. It doesn't matter what employer incentives are; revealing a log of how you've engaged with this CTF would involve us breaking a promise to you.
Meanwhile: what we're doing as a business is opt-in. If you don't think we're trustworthy, there is absolutely no reason to work with us to help find jobs. Just goof around with the game and ignore the fact that we do other stuff. We are no company's job interview.
>>>Most of the best performers on this CTF aren't going to get placed by us.<<<
It's frustrating talking about this, because we've had our hands absolutely full just getting a CTF that's better than Microcorruption done. We haven't had a spare moment to think about how to make money from it.
Catch me sometime when I'm not debugging a weird interaction between my emulated SPI bus and my emulated UART. :)
I feel like this is largely an issue about expectations and disclosure more than it is about logging per se, right? We have a privacy settings page, which I'll show you in a minute, but here's the public view of myself on the staging server. I have it configured to Maximally Public (and will have that on in prod).
https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442...
The redactions are just to avoid spoiling levels. The "replay" link is a UI templating bug. (It isn't for showing a replay, it is a convenience method for replaying the level, and it's now supposed to show unless you're logged in and able to play the level yourself. Fixing this in the next 5 minutes.)
Now, if I (as a player) log into and view my own profile, I get a better sense of some stuff the system knows about me:
https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FD...
That is obviously not all the information the system has about me. If a sparrow yawned in a world simulation that I took part in, that fact is dumped onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in. Every execution I got back. Snapshots of the GM's memory state as I passed through the levels. etc. We grab all that. That isn't an exhaustive list. Assume if it is amenable to being discovered by a computer that it will be discovered and persisted until the end of time.
This is not exposed publicly.
Here's the preferences UI. Verbose but it gets the point across, right. Defaults are "Anonymous" and "No Public Profile", respectively.
https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX...
Now to a question which may be implied: "OK, I now understand that my info is locked up pretty tightly within Starfighter. But you want to tell people that eventually, right? That's how you justify your service to clients." At the point I'm on the phone with the CTO of $CLIENT I am a contigency recruiter. That means I only get paid if that CTO decides to interview you and ultimately hire you. You can bet that I will be doing my best Enterprise Sales work and showcasing your performance in its most positive truthful light. If I did not feel like that would be possible, that phone call never happens. If the CTO had previously mentioned to be "Be on the lookout, specifically, for people who are good at analyzing mountains of data. We need them for our fraud team.", and I had relevant signal on that question from level X, you can be reasonably assured that if you did really well on it I'd open the call with "The reason I'm recommending Foobar to you is because you told me you wanted folks who are really good at analyzing data. Let me walk you through the solution Foobar came up with -- it's a hoot."
(Worth re-iterating explicitly -- that that point, we've talked on the phone and you've explicitly given me the go-ahead on placing you.)
To the question: "OK, so I get that people outside the system don't get arbitrary read on my history, and I understand that you're essentially a firewall which is incentivized to only allow the egress of the best possible true packets: what about you? Aren't you going to be influenced by, you know, taking a look directly at my stream of HTTP requests and seeing that I fumble fingered a URL the first time I played the level?" To which I say: I don't care and don't have time for that nonsense -- I'd much rather concentrate on things which actually matter to an engineer.
https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442...
The redactions are just to avoid spoiling levels. The "replay" link is a UI templating bug. (It isn't for showing a replay, it is a convenience method for replaying the level, and it's now supposed to show unless you're logged in and able to play the level yourself. Fixing this in the next 5 minutes.)
Now, if I (as a player) log into and view my own profile, I get a better sense of some stuff the system knows about me:
https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FD...
That is obviously not all the information the system has about me. If a sparrow yawned in a world simulation that I took part in, that fact is dumped onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in. Every execution I got back. Snapshots of the GM's memory state as I passed through the levels. etc. We grab all that. That isn't an exhaustive list. Assume if it is amenable to being discovered by a computer that it will be discovered and persisted until the end of time.
This is not exposed publicly.
Here's the preferences UI. Verbose but it gets the point across, right. Defaults are "Anonymous" and "No Public Profile", respectively.
https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX...
Now to a question which may be implied: "OK, I now understand that my info is locked up pretty tightly within Starfighter. But you want to tell people that eventually, right? That's how you justify your service to clients." At the point I'm on the phone with the CTO of $CLIENT I am a contigency recruiter. That means I only get paid if that CTO decides to interview you and ultimately hire you. You can bet that I will be doing my best Enterprise Sales work and showcasing your performance in its most positive truthful light. If I did not feel like that would be possible, that phone call never happens. If the CTO had previously mentioned to be "Be on the lookout, specifically, for people who are good at analyzing mountains of data. We need them for our fraud team.", and I had relevant signal on that question from level X, you can be reasonably assured that if you did really well on it I'd open the call with "The reason I'm recommending Foobar to you is because you told me you wanted folks who are really good at analyzing data. Let me walk you through the solution Foobar came up with -- it's a hoot."
(Worth re-iterating explicitly -- that that point, we've talked on the phone and you've explicitly given me the go-ahead on placing you.)
To the question: "OK, so I get that people outside the system don't get arbitrary read on my history, and I understand that you're essentially a firewall which is incentivized to only allow the egress of the best possible true packets: what about you? Aren't you going to be influenced by, you know, taking a look directly at my stream of HTTP requests and seeing that I fumble fingered a URL the first time I played the level?" To which I say: I don't care and don't have time for that nonsense -- I'd much rather concentrate on things which actually matter to an engineer.
As a piece of unsolicited UX feedback, if you chose a different word for one of "cleared"/"cleaned" it would make those two radically different actions stand out from one another in the list. I do see that "cleared" is accompanied by a score, so it's not the end of the world, but I did warn you it was unsolicited feedback. :-)
Also, do you know why evernote chooses to send "Content-Disposition: attachment; filename=skitch.png" forcing my browser to download instead of display the png inline?
Also, do you know why evernote chooses to send "Content-Disposition: attachment; filename=skitch.png" forcing my browser to download instead of display the png inline?
Thanks for the thorough response. Yeah, pretty much. It's good that you all have thought about this stuff. There's always a PVP aspect if you're ever going to put yourself public or on the leaderboards, but that's just part of the game. I can always start out anonymous and go public if I turn out to be any good...
[deleted]
@patio11:
Why should I "play" this and not blog about programming? I still have to go through the usual technical interview when applying for a job. What's my incentive? I'm trying to find a reason to spend time learning this, because it doesn't translate into marketable skills. I can't put "Level 65 player in StockFighter" (because this might turn out to be a niche product that no one outside HN will know about), but I will definitely include "XSS research on Java Spring".
Note: I'm not trolling or trying to put the project down. I'm interested in it but unsure.
Why should I "play" this and not blog about programming? I still have to go through the usual technical interview when applying for a job. What's my incentive? I'm trying to find a reason to spend time learning this, because it doesn't translate into marketable skills. I can't put "Level 65 player in StockFighter" (because this might turn out to be a niche product that no one outside HN will know about), but I will definitely include "XSS research on Java Spring".
Note: I'm not trolling or trying to put the project down. I'm interested in it but unsure.
Well, I think at least one intended reason is: because it's fun. I get to play games/solve interesting puzzles, and I might get contacted about a job that some person in a position to know thinks I might like and do well at? Count me IN.
That's not how it works. Enter the CTF and you get supplied as a lead to companies as a potential hire for doing well (if you want), it's not that you use it as a competency canary on your CV.
That's my point. I'm being supplied as a lead for investing time learning something that I will not be able to use in any potential position. Why should I do it? I'm genuinely interested in understanding this because it might blow up and I'll have to learn it anyway.
In addition to the "it's fun" thing mentioned by other commenters: I always learn gobs of transferrable stuff when I do things like this. Just solving different problems with languages and tools I already know usually teaches me a ton of stuff, let alone using it as an opportunity to try out new things in a low-risk environment. Take the difficult, data heavy, level 6 described in the article: maybe you can solve it using formulas you already know how to write, but maybe instead, it's a good chance to deploy a real Hadoop instance for the first time and see if you can get that to work, or maybe you want to learn Julia, or make a really optimized version in C++14 or Rust. I find these sorts of "programming exercise" things to be great learning experiences, regardless of the level of artifice.
My understanding from reading past posts is that they think this could be a great way to discover people who would be interested in roles at companies they hadn't previously considered. Shotgunning everyone who plays as leads isn't a great model as it lowers perceived quality of the candidates they send on ("I got a lead for this amazing developer, but she didn't want to change jobs at all, wtf?!").
Think of it as a game with "new job" as a piece of available loot. Play it because it seems fun (and you still might learn something that you can use later).
Think of it as a game with "new job" as a piece of available loot. Play it because it seems fun (and you still might learn something that you can use later).
Because it'll be fun. Fun. That's a thing we, as humans, do. Some other animals too.
Because its fun. If its not fun then they've failed.
It's a recruiting firm. Their business plan is to help companies with hiring (for a price) by matching the people who demonstrate skills in the game and opted in with their partner companies that want to hire people with those skills. The thesis is that successful starfigher candidates are all-star people that companies want on their team, but for whatever reason don't make it through the traditional hiring processes.
I suspect the right answer is to do both.
I suspect the right answer is to do both.
> (because this might turn out to be a niche product that no one outside HN will know about)
If this is the outcome, doing well at starfighter may not translate into much additional career success. For it to be clearly worth your time, they probably have to do reasonably well. But HN-famous might be enough for a tech recruiter.
If this is the outcome, doing well at starfighter may not translate into much additional career success. For it to be clearly worth your time, they probably have to do reasonably well. But HN-famous might be enough for a tech recruiter.
If nothing else reading this has me interested in trying Go. Kind of wish the new book by Kernighan was out before November, as that book feels like a great excuse t do a deep dive on the language.
On the topic of automated trading:
I strongly encourage anyone interested in real-world algorithmic trading to take a look at Quantopian.
https://www.quantopian.com/home
It's not HFT by any measure, but it manages to make algorithmic trading extremely accessible by providing a platform where users can develop their algorithms, backtest them against 13 years of market data, live trade them with paper money, or live trade with real money by linking a brokerage account.
They have also open-sourced the algorithmic trading engine used to run their platform:
https://github.com/quantopian/zipline
I strongly encourage anyone interested in real-world algorithmic trading to take a look at Quantopian.
https://www.quantopian.com/home
It's not HFT by any measure, but it manages to make algorithmic trading extremely accessible by providing a platform where users can develop their algorithms, backtest them against 13 years of market data, live trade them with paper money, or live trade with real money by linking a brokerage account.
They have also open-sourced the algorithmic trading engine used to run their platform:
https://github.com/quantopian/zipline
Yep. There is like, zero overlap between Quantopian and what we're doing; they're both good things to play with.
(I'm the CTO and cofounder of Quantopian)
I am incredibly excited to try my hand at Stockfighter. Hopefully some of the things I've learned over the last few years at Quantopian will be useful, but I expect I'll learn quite a few new things as well.
I am incredibly excited to try my hand at Stockfighter. Hopefully some of the things I've learned over the last few years at Quantopian will be useful, but I expect I'll learn quite a few new things as well.
People who are at the leading edge of HFT do everything they can to trim microseconds off their trading time ... it can get pretty insane. One talk on HFT performance at JavaOne [1] included the recommendation that you buy more memory - so you can turn off garbage collection completely during trading hours.
[1] http://www.slideshare.net/PeterLawrey/writing-and-testing-hi...
[1] http://www.slideshare.net/PeterLawrey/writing-and-testing-hi...
Do you guys model regulation through world simulation (e.g. the twitter feeds) and is there a Federal Reserve/Mint with related data?
Really simple question: do the privacy settings have any impact on the recruiting stuff? I would love for you guys to poke around my data, I trust you guys, and would love for you to be able to get my name, but I would otherwise prefer to be anonymous. Ideally it would be a privacy-orthogonal checkbox, maybe?
YES. Reminder: most people who play with Starfighter CTFs will never be interested in a new job. We're designing this stuff for those players just as much as job seekers; we're competitive about CTFs, and trying to make a mark.
Look at the leaderboard on Microcorruption.com and notice how few real names are there. That's what CTFs are like. If you want to have a profile page, we'll give you one, if you want to link up your Twitter account, you can do that, but if you want to be secretsquirrel1235813, you can do that too. From experience with CTFs: I think a slight majority of people will be anonymous.
There is nothing we will do with your data that won't be opt-in.
Look at the leaderboard on Microcorruption.com and notice how few real names are there. That's what CTFs are like. If you want to have a profile page, we'll give you one, if you want to link up your Twitter account, you can do that, but if you want to be secretsquirrel1235813, you can do that too. From experience with CTFs: I think a slight majority of people will be anonymous.
There is nothing we will do with your data that won't be opt-in.
> We do not intentionally expose you to the speed of light as a limiting factor early in Stockfighter… but we can’t remove latency as being an actual limitation distributed systems have to overcome.
Couldn't you remove latency as a factor, at least in a game world? To play devil's advocate, it seems like you could build a game where time advances according to the player's control by API.
Imagine if each API call optionally took a parameter describing by how much time should have advanced since the last call, upon the arrival of the current call. If the parameter is omitted, the server advances the world by the corresponding amount of real clock time. The game simulates that time passing, and simulates receiving the call at the time specified. Perhaps an API to stop and start the flow of game time, or run the game for a certain time and stop.
Players can interact with the game naturally, and solve problems without controlling time, but if they run into latency issues they can orchestrate the advance of time on the server precisely. I understand this will not be simple to implement, especially if the game consists of multiple communicating agents, but it could allow you to remove the uncertainty of players interacting with the game over a network, and more easily set up complex race conditions or timing attacks. It will also more gracefully handle any slowness that occurs in the game or bot simulations, since while game calls might take longer real time to return, they will presumably not take more game time. Last but not least, it could allow you to design test cases that are deterministic in ways that involve time, i.e., this event occurs exactly 100ms after the game world begins, every time. Players can exactly reproduce situations that would otherwise occur nondeterministically, while you retain your ability to simulate interactions that you want to be nondeterministic.
(I'm just exploring the idea because it seemed interesting, not making the case that this is especially important in the scheme of things.)
Couldn't you remove latency as a factor, at least in a game world? To play devil's advocate, it seems like you could build a game where time advances according to the player's control by API.
Imagine if each API call optionally took a parameter describing by how much time should have advanced since the last call, upon the arrival of the current call. If the parameter is omitted, the server advances the world by the corresponding amount of real clock time. The game simulates that time passing, and simulates receiving the call at the time specified. Perhaps an API to stop and start the flow of game time, or run the game for a certain time and stop.
Players can interact with the game naturally, and solve problems without controlling time, but if they run into latency issues they can orchestrate the advance of time on the server precisely. I understand this will not be simple to implement, especially if the game consists of multiple communicating agents, but it could allow you to remove the uncertainty of players interacting with the game over a network, and more easily set up complex race conditions or timing attacks. It will also more gracefully handle any slowness that occurs in the game or bot simulations, since while game calls might take longer real time to return, they will presumably not take more game time. Last but not least, it could allow you to design test cases that are deterministic in ways that involve time, i.e., this event occurs exactly 100ms after the game world begins, every time. Players can exactly reproduce situations that would otherwise occur nondeterministically, while you retain your ability to simulate interactions that you want to be nondeterministic.
(I'm just exploring the idea because it seemed interesting, not making the case that this is especially important in the scheme of things.)
I've just started programming professionally. I've noticed I'm a quick learner and I'm pretty good at problem solving. However, because I just started I lack quite a lot of knowledge. Will I still be able to play this game, or is 5 years of full-time work on multiple languages a prerequisite?
The first few levels are specifically designed to be solvable by someone near your skill level, and after you get through them, you might surprise yourself on the "hard" ones.
Two anecdotal data points on somewhat similar sites from an employed programmer and father.
Codehunt (from Microsoft Research), very addictive yet nice concise format, polished interface, allowing Java or C# a plus.
Minuses - Requires some unnatural code golf for maximum stars. Needs new problems and difficulty is too variable. I got in Top 5 but realized it was a completely worthless achievement as it just was not hard enough, somehow I hoped for a short note from MS saying GJ here is the next challenge level.
Bonus: Did not feel bad about playing this at work on breaks.
Codecombat - program in javascript, using classical MMORPG terminology, this should have been more addictive but something was off, maybe it was the latency or browser glitches.
My bot using simple strategy got in top, but I realized that I would rather go back to writing my scripts for real MMORPGs where my bots make actual money from RMT.
I still get Codecombat monthly e-mail reminders of new features but somehow the appeal of fake MMORPG is not there.
Also, felt very guilty about firing Codecombat up at work...
So you guys not going for the standard DnD type of setting might be a good idea.
Codehunt (from Microsoft Research), very addictive yet nice concise format, polished interface, allowing Java or C# a plus.
Minuses - Requires some unnatural code golf for maximum stars. Needs new problems and difficulty is too variable. I got in Top 5 but realized it was a completely worthless achievement as it just was not hard enough, somehow I hoped for a short note from MS saying GJ here is the next challenge level.
Bonus: Did not feel bad about playing this at work on breaks.
Codecombat - program in javascript, using classical MMORPG terminology, this should have been more addictive but something was off, maybe it was the latency or browser glitches.
My bot using simple strategy got in top, but I realized that I would rather go back to writing my scripts for real MMORPGs where my bots make actual money from RMT.
I still get Codecombat monthly e-mail reminders of new features but somehow the appeal of fake MMORPG is not there.
Also, felt very guilty about firing Codecombat up at work...
So you guys not going for the standard DnD type of setting might be a good idea.