I've recently written a resolution engine for keywords when matching cv's to job descriptions. it also helps finding patterns in other skills that may be transferrable. i hope to show HN as soon as i finish some performance improvements (which includes writing a brand new db driver :'()
I am building a similar product for the same market and what everyone has told you on this thread is great advice. People thinking that there is a one size fits all solution to hiring methods usually go out of business (path.to work4pie etc). Big hitters entering the market validates what you are doing. We have a very specific proposition for a very specific group of people which we want to solve best. If you are really interested in this problem maybe we could talk more outside of HN.
I've been lucky to work in Australia, east & west coast USA and the UK. In that time I have witnessed the similarities and differences between hiring practices and to be honest I feel that a lot of what we experience is deserved. We as problem solvers have just not bothered with solving the one problem we as an industry ubiquitously hate the most: IT recruitment agents.
We continue to buy into an antiquated paradigm of job descriptions and CV's. In my opinion these artefacts hurt our industry more than help us. We are so much better than this (and I don't think that answer is portfolio work as not all programmers work with public facing products - which is a cohort i think HN can sometimes overrepresent).
I've been working on a startup to hopefully change the way all this works. Modelling how to address how different types of developers seek opportunities, whether we are in work or out and tying that into how different types of hiring manager (and their teams) meet us. If we do it right it won't be too long before the era of 15-30% 'placement fees' are back to being a much more manageable cost.
From the languages.. I've worked with lots of quants and they all rave about R!
Yeah..the aggregator is where you get done in both cost and latency. The prop shops and hedge funds pay through the nose for that stuff so unless you come packing a little capital, true HFT is an issue.
On the middleware, not really.. so you would have a market data component that will be pushing stuff to various components (real time risk, the pricer, and the trading engine). The disruptor sits on the 'in' queue to those components.. the middleware is what pushes messages between instances running on the same/different machines.
I think given the languages you have selected you are coming more from a quant background? These languages are great for heuristics and analysis but you would really want all 'static' components such as connectivity built in assembly/C/C++. For 'algo' components I like Java as you can still pull microsecond order latencies when crunching numbers but more importantly it gives you a huge time to market advantage than the C/C++ for almost the same speeds. I'm also not clear on if you are connecting directly to the market for market data or using aggregation (like reuters). The latter would be too slow. I'm also not clear on what middleware you are using which is probably the biggest decision you will have to make. Most either use inhouse tech or 29west LBM (everyone still calls it that even though they were bought out).
An overlooked part of HFT in my experience OS optimisation and even things like TCP bypass (for some components) which can lead to huge speed advantages and end to end latency reductions. I agree with those about FPGA.. in my experience they really don't come into the equation except for components that rarely change.
Anyway a few guys including Martin Thompson have felt similarly to you and initiated the lodestone project (http://lodestonefoundation.wordpress.com/). If you are keen to learn more their architectures for low latency, distributed and componentisation then I feel that you could join forces and contribute to that initiative. FYI: Martin built most of the technology behind the disruptor (http://lmax-exchange.github.io/disruptor/).
Great to see interest in making this knowledge more widely available :)
I should have mentioned that in my original post. I do tend to talk about my idea with everyone. It helps to weed out poor ideas and germinate new ones.
My question really was literally to do with a direct competitor getting an insight into my business plan while I was in the middle of building the MVP.
I think you are right in that i should stop worrying though :)
Also: I got a message from another mate in YC. He said that it's mostly the guys that are close with YC that do a lot of the screening so less worrying ahoy!