Nanex Gets $700k Whistleblower Award from SEC(nanex.net)
nanex.net
Nanex Gets $700k Whistleblower Award from SEC
http://www.nanex.net/aqck2/4712.html
236 comments
Can you comment on how hard it was to find another period exhibiting this behavior? The article says We chose this period because there was a noticeable lag in the public quote from the NYSE versus quotes from other exchanges, allowing us to rule out the consolidation process as a source of the delay. Did you have to look for a few periods to see the behavior, or did you look for a few to get a sample that showed the problem more clearly?
Do you have any info on how often lag was present, or how often over a specific threshold?
Do you know how representative you sample was with respect to when lag was present?
I understand you may not have gone into this much detail in your research, but if you have any info like this, I think it helps illustrate the scope of the problem.
Do you have any info on how often lag was present, or how often over a specific threshold?
Do you know how representative you sample was with respect to when lag was present?
I understand you may not have gone into this much detail in your research, but if you have any info like this, I think it helps illustrate the scope of the problem.
It happens all the time (ALL. THE. TIME.) and have documented dozens of examples. There are links to many of those examples on that page (fantaseconds).
I've also posted many examples on twitter @nanexllc using recent data (yes, this is still a problem!)
I've also posted many examples on twitter @nanexllc using recent data (yes, this is still a problem!)
I second that answers to these types of questions would be very helpful! The more context the better.
@TheNanex, Eric?
I have always been facinated by the underlying technology that drives Nanex's products. I follow http://www.nanex.net/NxResearch/ regularly and would like to see some more technical anotomy of some of the tools you guys have built.
Thanks and great work!
I have always been facinated by the underlying technology that drives Nanex's products. I follow http://www.nanex.net/NxResearch/ regularly and would like to see some more technical anotomy of some of the tools you guys have built.
Thanks and great work!
It's about the NYSE not delivering a product (SIP real-time data) while collecting $100M a year for that service
And, just to be clear, the NYSE generously agreed to pay a $5M fine as punishment?
The actual SEC order is linked to by Nanex, and can be found here: http://sec.gov/litigation/admin/2012/34-67857.pdf It explains NYSE's failings in more detail, e.g. "Inadequate Compliance Efforts" and "Failure to Retain Business Records".
And, just to be clear, the NYSE generously agreed to pay a $5M fine as punishment?
The actual SEC order is linked to by Nanex, and can be found here: http://sec.gov/litigation/admin/2012/34-67857.pdf It explains NYSE's failings in more detail, e.g. "Inadequate Compliance Efforts" and "Failure to Retain Business Records".
Excellent question: they were ECSTATIC to pay $5M
Covington and Burling - who represented the NYSE in this matter, used the paltry $5M fine as bragging rights in their year end letter to clients! 'We got the NYSE off with just $5M'
Nice.
Covington and Burling - who represented the NYSE in this matter, used the paltry $5M fine as bragging rights in their year end letter to clients! 'We got the NYSE off with just $5M'
Nice.
Thank you for the work you do Eric. The light you shine on the dark parts of markets is incredible.
Please excuse my ignorance if this question is dumb, but what is the name of the kind of Chart 3? I've never seen one like it and it's fascinating to me - it seems like it's a 2d line graph with some shaded regions, but not a heat map.
I don't have a name for that chart. It's a customized display I wrote that I found makes it easiest to view quote spreads and trades from multiple exchanges. I write in "c" combining simple graphics: lines, pixels and text.
Then I propose "TheNanex c chart". It's information rich, multi-multi-dimensional, and really tells the story in a nutshell.
I printed one out and put it on my "Tufte" shelf [1].
Edit: Add [1] https://en.wikipedia.org/wiki/Edward_Tufte
I printed one out and put it on my "Tufte" shelf [1].
Edit: Add [1] https://en.wikipedia.org/wiki/Edward_Tufte
I am a big fan of Edward Tufte.
[deleted](1)
Hey, a bit late to the game here, but I just want to say I'm a huge fan of your work.
At my last job I worked on one of the Reg CAT bids, and I spent a lot of time looking at Nanex Research--totally mind blowing. The markets would be a better place if the regulators were doing Nanex-style analysis. Keep up the good work!
At my last job I worked on one of the Reg CAT bids, and I spent a lot of time looking at Nanex Research--totally mind blowing. The markets would be a better place if the regulators were doing Nanex-style analysis. Keep up the good work!
Please send me info on variable rate annuities and what they can do for my family.
So essentially the NYSE is a provably fixed game, with large HFT houses paying for the right to trade on non-public information, and it hides behind the idea that it was just a few hundred milliseconds.
All the while they are selling non-public information as a product that is deliberately used to micro-manipulate the market?
I mean, deep down I always knew this, but to have it all spelled out is shocking because it exposes the NYSE as systemically fraudulent. The implication of that is so overwhelming it causes me to want to ignore it out of sheer inability to think of a way to solve the problem.
All the while they are selling non-public information as a product that is deliberately used to micro-manipulate the market?
I mean, deep down I always knew this, but to have it all spelled out is shocking because it exposes the NYSE as systemically fraudulent. The implication of that is so overwhelming it causes me to want to ignore it out of sheer inability to think of a way to solve the problem.
Market data is not non-public definition. NYSE was simply slow in aggregating market data during times of high volume.
This did not affect you as a retail investor, as you get the NBBO price. This only affected you if you were an HFT firm with bad infrastructure who depended on the aggregate feed and not their direct line.
Like tptacek said, the price for getting a direct line, while expensive, is not unreasonable for a business. The largest cost will be salaries for the people writing your code and maintaining your infrastructure.
This did not affect you as a retail investor, as you get the NBBO price. This only affected you if you were an HFT firm with bad infrastructure who depended on the aggregate feed and not their direct line.
Like tptacek said, the price for getting a direct line, while expensive, is not unreasonable for a business. The largest cost will be salaries for the people writing your code and maintaining your infrastructure.
> NYSE was simply slow in aggregating market data during times of high volume.
Maybe I misunderstood, but it seems like they're saying that's not the case:
FTA: "First, we had to rule out the possibility of the delay being caused by the consolidation process (which people often erroneously point out)."
Maybe I misunderstood, but it seems like they're saying that's not the case:
FTA: "First, we had to rule out the possibility of the delay being caused by the consolidation process (which people often erroneously point out)."
By consolidation, he means the Consolidated Tape Association which takes feeds from all exchanges (NYSE, Nasdaq, AMEX, etc) and puts out a single consolidated feed that most non-HFT entities use because its good enough.
They get their data from NYSE from apparently an aggregate feed, which was unintentionally delayed by NYSE. In my opinion, they should have been getting the direct feed and doing all consolidation themselves.
However, HFT firms will still have a vested interest in making more efficient aggregate feeds than the consolidators/exchanges.
They get their data from NYSE from apparently an aggregate feed, which was unintentionally delayed by NYSE. In my opinion, they should have been getting the direct feed and doing all consolidation themselves.
However, HFT firms will still have a vested interest in making more efficient aggregate feeds than the consolidators/exchanges.
> Like tptacek said, the price for getting a direct line, while expensive, is not unreasonable for a business. The largest cost will be salaries for the people writing your code and maintaining your infrastructure.
Be that as it may in reality, per below quote I assume that behavior is still illegal.
"A crucial sub-ruling in the regulations prohibits exchanges from giving stock quotes to special groups faster than to the public. This sub-rule is of such importance, that without it, the rest of the rules (Reg. NMS) essentially become meaningless."
Be that as it may in reality, per below quote I assume that behavior is still illegal.
"A crucial sub-ruling in the regulations prohibits exchanges from giving stock quotes to special groups faster than to the public. This sub-rule is of such importance, that without it, the rest of the rules (Reg. NMS) essentially become meaningless."
Outside of labor, the primary costs are hardware, colocation, and networking, none of which is a fee to the exchange.
Please tell, how the world is supposed to provide that for free to anyone who wants it?
Colocated servers are the public.
Please tell, how the world is supposed to provide that for free to anyone who wants it?
Colocated servers are the public.
The regulation is about equal access, not free access. Now, the SIP feed contains combined market data from all of the exchanges to which the National Best Bid / Offer (NBBO) applies, so that a broker-dealer can look at the SIP feed to determine nationally what the best bid across many exchanges is and what the best offer across many exchanges is. The Reg NMS mentioned is that the broker can't give you a price (before fees) that's outside the NBBO. (There's an exception if the investor explicitly opts the order out of Reg NMS requirements and directs that the order be filled on a specific venue. Last I checked there are 70-some equity venues in the US.)
The consolidated feeds are necessarily slower than the direct feeds, due to processing delays, unless the exchanges intentionally slow down their direct feeds. There are several companies that will sell you expensive FPGA or IBM Cell-based cards that will locally create a consolidated feed from all of the direct feeds. These cards aren't cheap, but are necessary for trading latency arbitrage strategies, and are also useful for low-latency market making.
Of course the consolidated feed requires a slight processing delay, so unless they intentionally slow down the direct feed, quotes from NYSE will appear first in the direct feed and very slightly later in the SIP feed. The argument here is about if the delay between SIP and the direct feed is small enough to be permissible.
The "public" being talked about is the paying public. Nothing in the regulation requires a free SIP feed, and I assure you NYSE doesn't provide a free real-time SIP feed. The regulation is to prevent selling an intentionally slow connection to everyone, and giving a faster feed to only their top tier customers.
Disclaimer: I'm not a lawyer.
The consolidated feeds are necessarily slower than the direct feeds, due to processing delays, unless the exchanges intentionally slow down their direct feeds. There are several companies that will sell you expensive FPGA or IBM Cell-based cards that will locally create a consolidated feed from all of the direct feeds. These cards aren't cheap, but are necessary for trading latency arbitrage strategies, and are also useful for low-latency market making.
Of course the consolidated feed requires a slight processing delay, so unless they intentionally slow down the direct feed, quotes from NYSE will appear first in the direct feed and very slightly later in the SIP feed. The argument here is about if the delay between SIP and the direct feed is small enough to be permissible.
The "public" being talked about is the paying public. Nothing in the regulation requires a free SIP feed, and I assure you NYSE doesn't provide a free real-time SIP feed. The regulation is to prevent selling an intentionally slow connection to everyone, and giving a faster feed to only their top tier customers.
Disclaimer: I'm not a lawyer.
The SIP (consolidated feed) is tasked with using the most efficient hardware and software available. If a private company can consolidate faster than the SIP, then the SIP isn't using the most efficient hardware and software available now is it?
Pretty simple.
The exchanges take in $500M a year in SIP fees, which are supposed to go to the most efficient hardware and software available. Pretty sure that's not happening.
Pretty simple.
The exchanges take in $500M a year in SIP fees, which are supposed to go to the most efficient hardware and software available. Pretty sure that's not happening.
> Pretty simple.
Oversimplification, forgetting the laws of geometry and physics.
The triangle inequality and relativity conspire against the SIP. If you're getting the SIP from B, any new prices at C go to B and then forwarded to you, so it's faster (best case equal in the degenerate triangle case) to get direct feeds from B and C and consolidate locally.
Oversimplification, forgetting the laws of geometry and physics.
The triangle inequality and relativity conspire against the SIP. If you're getting the SIP from B, any new prices at C go to B and then forwarded to you, so it's faster (best case equal in the degenerate triangle case) to get direct feeds from B and C and consolidate locally.
Are you saying the SIP runs on the best hardware/software available? I want to get that cleared up before moving on to a new topic.
I don't know about the hardware being used by the SIP. It might very well be very slow hardware. You clearly have a lot more knowledge than I do and have done some very good research.
I was just pointing out that
> If a private company can consolidate faster than the SIP, then the SIP isn't using the most efficient hardware and software available now is it?
is faulty reasoning.
I was just pointing out that
> If a private company can consolidate faster than the SIP, then the SIP isn't using the most efficient hardware and software available now is it?
is faulty reasoning.
Not to mention the CAP theorem. For the SIP to work you have to have a much stronger consistency model across many more nodes. To arbitrage latency in it you need neither.
If NYSE wants to lobby for laws to be changed, then they're welcome to do that. Just like Uber or Airbnb when they get hit with fines.
But the law's the law. "It doesn't make sense" isn't a valid defense.
But the law's the law. "It doesn't make sense" isn't a valid defense.
Imagine that they're providing free firewood to the public - what obligates them to provide the pickup trucks?
The law, in the same way we regulate many markets that can only efficiently support fewer participants than would be required for a fully competive marketplace.
Usually due to capex: energy infrastructure, network effect: financial, or limited availability of resources: wireless spectrum.
If it's in the public interest to have trucks and no firewood seller would otherwise provide them. That's absolutely something government can (and IMHO should) mandate in exchange for a firewood license.
Usually due to capex: energy infrastructure, network effect: financial, or limited availability of resources: wireless spectrum.
If it's in the public interest to have trucks and no firewood seller would otherwise provide them. That's absolutely something government can (and IMHO should) mandate in exchange for a firewood license.
Well I demand that my free server have 128GB of ram, and holograms. Fucking fatcats.
[deleted]
Where to begin.
The retail investor gets the slowed down price (the NBBO is really what the internalizer/wholesaler saw from the SIP when they executed your order). That same internalizer buys/sells from the faster feed. Whenever the two feeds are out of sync, it's like printing money.
"The price for getting a direct line is not unreasonable?" I just laughed out loud reading that. Is $60,000/month for just one exchange (there are a dozen), reasonable? Then you need network engineers, infrastructure, etc. It used to be nowhere near this expensive for "real-time" data. Orders of magnitude less.
The retail investor gets the slowed down price (the NBBO is really what the internalizer/wholesaler saw from the SIP when they executed your order). That same internalizer buys/sells from the faster feed. Whenever the two feeds are out of sync, it's like printing money.
"The price for getting a direct line is not unreasonable?" I just laughed out loud reading that. Is $60,000/month for just one exchange (there are a dozen), reasonable? Then you need network engineers, infrastructure, etc. It used to be nowhere near this expensive for "real-time" data. Orders of magnitude less.
What would be your proposal for a solution to a sync'd feeds obvious distributed systems problems?
How much latency would be acceptable for a fully consistent, high throughput distributed log of events, that when it becomes unavailable the entirety of US trading shuts down and if it ever gets out of order can cause gigantic lawsuits?
How would you change the obvious organizational disincentives as well? Given that maintenance of the SIP is full cost, no profit operation for the exchanges?
Finally, don't take this as a criticism of your whistle blowing, I think that is critical to a functioning market, I am curious to know how you would solve the obvious technical challenges the consolidated feed faces?
How much latency would be acceptable for a fully consistent, high throughput distributed log of events, that when it becomes unavailable the entirety of US trading shuts down and if it ever gets out of order can cause gigantic lawsuits?
How would you change the obvious organizational disincentives as well? Given that maintenance of the SIP is full cost, no profit operation for the exchanges?
Finally, don't take this as a criticism of your whistle blowing, I think that is critical to a functioning market, I am curious to know how you would solve the obvious technical challenges the consolidated feed faces?
You are making straw man arguments. While I'm happy to discuss what the SIP should be in an ideal world, the topic at hand is what is current law. For those that don't know, it's Reg NMS (google it).
Actually, I was asking how you would design an ideal SIP. I have no opinions on the current law, but find the technical challenges of the SIP terribly interesting.
I'd still love to hear you discuss it...
I'd still love to hear you discuss it...
If you are so happy to discuss what the SIP should be in an ideal world, why don't you? This reply seems to be a deliberate non-answer.
What do you mean by "fully consistent" in the context of the SIP feed? The SIP isn't required to be consistent in the CAP sense of "consistent".
The SIP isn't currently required to be consistent in the CAP sense of "consistent", but that is something people complain about. I think rightly, they point that the audit/compliance components of RegNMS are very troubling in the face of N inconsistent SIPs.
Similarly, people complain about the SIP feeds being latent and I'm sure it would be a very real problem if any single exchange being unavailable made the entirety of the SIP unavailable.
Its a fascinating problem because the engineering trade-offs are relatively static and well known, but each and every decision you could make provides an opportunity for arbitrage market activity that someone would consider predatory.
That we've layered a regulatory requirement that is pretty vague on some of the edge cases on top of this, just adds to it.
Similarly, people complain about the SIP feeds being latent and I'm sure it would be a very real problem if any single exchange being unavailable made the entirety of the SIP unavailable.
Its a fascinating problem because the engineering trade-offs are relatively static and well known, but each and every decision you could make provides an opportunity for arbitrage market activity that someone would consider predatory.
That we've layered a regulatory requirement that is pretty vague on some of the edge cases on top of this, just adds to it.
Please point to the exact paragraph in Reg NMS that backs up your story. To those reading, don't waste your time awaiting an answer because there is no such paragraph in Reg NMS.
The Regulations (Reg NMS) are very clear about this. That's why I was awarded $750K from the government after all.
The Regulations (Reg NMS) are very clear about this. That's why I was awarded $750K from the government after all.
I'm not sure what story you want me to back up is? If you are claiming that the current law requires the SIP to be consistent in the CAP sense, I'll defer to you and ask that you point out why you think so?
If you are asking about evidence that some people have claimed that the SIP being non-consistent is a problem for audit and regulatory compliance, is that not what you yourself are arguing in this post? http://www.nanex.net/Research/IsNBBOIgnored.html
I think maybe you are reading into my comment something that isn't there?
If you are asking about evidence that some people have claimed that the SIP being non-consistent is a problem for audit and regulatory compliance, is that not what you yourself are arguing in this post? http://www.nanex.net/Research/IsNBBOIgnored.html
I think maybe you are reading into my comment something that isn't there?
What if you want to write your own code and run your own infrastructure? The direct line may still be cost prohibitive and bestow a distinct advantage to your competitors.
Some people think this is not the case?
Some people think this is not the case?
No one is entitled to be able to enter a market. It would be practically infeasible for me to compete against Amazon by writing code at home. That doesn't mean the ecommerce market is a bad thing.
[deleted]
Are you saying that there is a special market operating in the NYSE that some traders cannot access? Why would my order not be eligible for being matched, but a HFT's would?
Edit: wow, rate limited after three posts this morning. A new HN low.
My response to tptacek below:
My dumb order?
How can HFT's intercept and redirect my trades to their, appently, captive pool of dumb trades?
Sure sounds like multiple markets are operating...and you are even telling me that my orders will be scraped before they can even reach some markets.
Edit: wow, rate limited after three posts this morning. A new HN low.
My response to tptacek below:
My dumb order?
How can HFT's intercept and redirect my trades to their, appently, captive pool of dumb trades?
Sure sounds like multiple markets are operating...and you are even telling me that my orders will be scraped before they can even reach some markets.
No, that's not what he's saying. But aside from that: the reason orders you place won't be matched on any market at NYSE, let alone a special one, is that HFT market makers will give you a price break to route your dumb order to them so they can collect their tiny spread from your trade without worrying that you're a hedge fund about to steamroll them with a giant block order.
Who is subsidizing the HFT market makers so that they can undercut the rest of the market on their private exchanges?
This is like a question from a markov generator. I won't try to answer it.
Some help:
>is that HFT market makers will give you a price break to route your dumb order to them
You claim a price break on a private exchange. How can they (HFT market makers) deliver a better price than the public exchange? Presumably they cannot have bought it on the public market at the better price, so some party must absorb the loss on the price break.
My question can now be translated to:
Who absorbs the loss from the price break?
>is that HFT market makers will give you a price break to route your dumb order to them
You claim a price break on a private exchange. How can they (HFT market makers) deliver a better price than the public exchange? Presumably they cannot have bought it on the public market at the better price, so some party must absorb the loss on the price break.
My question can now be translated to:
Who absorbs the loss from the price break?
When the going gets tough, the questions are dumb? This is the hallmark of lobbyist talk.
Oh boy, you got this 100% the wrong way around.
As a small trader, you get access to special lower prices that a hedge fund can't get. Your broker will be routing your order to a wholesaler who will fill it at lower prices (ie, narrower spreads) than you would get on the open market, because they have a legal obligation to not screw you over.
You can request that they route it to anywhere you want, and they are legally obligated to do it if you ask, but you really don't want to do that. Unless you like giving money away of course.
(Also, you don't seem to understand basic market mechanics. You use words like "scraped" or "picked up" which are nonsensical in context.)
IEX recently suggested that retail investors should request their broker to route their orders to IEX, for which they got criticised very harshly. And rightly so.
As a small trader, you get access to special lower prices that a hedge fund can't get. Your broker will be routing your order to a wholesaler who will fill it at lower prices (ie, narrower spreads) than you would get on the open market, because they have a legal obligation to not screw you over.
You can request that they route it to anywhere you want, and they are legally obligated to do it if you ask, but you really don't want to do that. Unless you like giving money away of course.
(Also, you don't seem to understand basic market mechanics. You use words like "scraped" or "picked up" which are nonsensical in context.)
IEX recently suggested that retail investors should request their broker to route their orders to IEX, for which they got criticised very harshly. And rightly so.
This is incorrect. We destroyed a Barron's article for making similarly false claims and we back it up with solid data: http://www.nanex.net/aqck2/4685.html We even caught Barron's cheating! The nerve of some people to lie with abandon.
As a small trader I don't get paid to provide liquidity. A hedge fund may.
I used the word scrape in response to tptacek's response below my above post. If a trader is not aware that what tptacek describes is happening to their orders, then I believe my characterization is accurate.
I used the word scrape in response to tptacek's response below my above post. If a trader is not aware that what tptacek describes is happening to their orders, then I believe my characterization is accurate.
You're still not addressing my main point.
As a small investor, your order is probably routed to a wholesaler like Citadel, where you will get a better price than you would otherwise get.
And you are upset because you incorrectly believed that your order would be routed somewhere else (where, exactly?), where you would get a worse price, which you think would be better because...why?
> I believe my characterization is accurate.
It's not even slightly accurate; it's literally the opposite of the truth.
As a small investor, your order is probably routed to a wholesaler like Citadel, where you will get a better price than you would otherwise get.
And you are upset because you incorrectly believed that your order would be routed somewhere else (where, exactly?), where you would get a worse price, which you think would be better because...why?
> I believe my characterization is accurate.
It's not even slightly accurate; it's literally the opposite of the truth.
Are you from Citadel?
Would you trust Wall Street to give you the best price on anything if there was no way to verify it?
There is no way to reasonably know if you are getting the best price (because they can change many times in the same second).
You are relying on the internalizer to give you the best of many possible prices. History says: "that's not going to happen"
Would you trust Wall Street to give you the best price on anything if there was no way to verify it?
There is no way to reasonably know if you are getting the best price (because they can change many times in the same second).
You are relying on the internalizer to give you the best of many possible prices. History says: "that's not going to happen"
Just as a heads up as you're new here, accusing someone of shilling is a pretty big no no.
Full disclosure, I have in fact worked with your data and at an HFT in the past but have moved out of the industry, if you were going to ask.
Full disclosure, I have in fact worked with your data and at an HFT in the past but have moved out of the industry, if you were going to ask.
Read Dark Pools (http://smile.amazon.com/gp/product/B006OFHLG6/) and it will make the setup at least comprehensible.
I think I get the setup and that my characterization is rather accurate.
Well just to clarify a little. There are 2 different concepts that sound sort of similar.
1) A dark pool. This is just another name for an exchange, but one that is not public. I don't know if NYSE runs one, but it wouldn't be surprising. The theory behind these is that they are provided as a service to large institutional investors to trade with each others outside the vagaries of the public markets for lots of reasons (mostly advertised as a way to not impact prices). The reality of these has been mixed at best. You have to opt into using a dark pool.
2) Payment for order flow. In this case your broker (not the exchange) has an agreement with a third party to sell your trades to them so that they can either trade with you directly or route the trade to an exchange. The reason they want to do this is that the aggregate of all the trades they are buying is not price opinionated (as opposed to hedge funds say) so the market maker can make the aggregate difference between all of them. This mechanism is largely how all discount brokers offer cheap or free trading, so it is likely a huge boon to you. A couple of things to note about this, your broker has to disclose it to you, I believe you are still required to get the NBBO price, & I believe you are required to be able to opt out by directing your order to be directly routed.
[edit] To clarify your question `jsprogrammer `tptacek is referencing #2 above. The reason the technical oddities & violations at NYSE don't have a huge impact on a retail investor is that unless otherwise stated, they are interacting with a wholesale market maker before their order hits any exchange, in a deal worked out with your broker and governed by legal obligations on both their parts.
1) A dark pool. This is just another name for an exchange, but one that is not public. I don't know if NYSE runs one, but it wouldn't be surprising. The theory behind these is that they are provided as a service to large institutional investors to trade with each others outside the vagaries of the public markets for lots of reasons (mostly advertised as a way to not impact prices). The reality of these has been mixed at best. You have to opt into using a dark pool.
2) Payment for order flow. In this case your broker (not the exchange) has an agreement with a third party to sell your trades to them so that they can either trade with you directly or route the trade to an exchange. The reason they want to do this is that the aggregate of all the trades they are buying is not price opinionated (as opposed to hedge funds say) so the market maker can make the aggregate difference between all of them. This mechanism is largely how all discount brokers offer cheap or free trading, so it is likely a huge boon to you. A couple of things to note about this, your broker has to disclose it to you, I believe you are still required to get the NBBO price, & I believe you are required to be able to opt out by directing your order to be directly routed.
[edit] To clarify your question `jsprogrammer `tptacek is referencing #2 above. The reason the technical oddities & violations at NYSE don't have a huge impact on a retail investor is that unless otherwise stated, they are interacting with a wholesale market maker before their order hits any exchange, in a deal worked out with your broker and governed by legal obligations on both their parts.
1)
A U.S. equities exchange has to be registered with and approved by the SEC. There are currently 12 of them. As a registered exchange you have to play by certain rules, and in exchange your displayed top of of book is subject to the order protection rule, meaning that no other venues (exchanges, dark pools, ECNs, or broker dealers) are allowed to trade through your top of book quote without first routing an order to you for the full amount of the displayed size. Exchanges also get a share of SIP revenue (which is important to exchange economics).
In it's most technical definition, a dark pool is an ATS (specifically not an exchange) that has no displayed quote i.e. no market data whatsoever. Most ATSs in the US fit this definition, though there are also ATSs that display a quote. For example, IEX is an ATS that has feed for their top of book quote... so technically not a "dark" pool.
NYSE doesn't run a dark pool and it would be weird if they did. Most US equity exchanges support fully hidden (completely non-displayed) orders, so if you want to trade in the "dark" on NYSE you can just send a hidden order. I think this is one area where futures exchanges and equity exchanges differ. In futures you have icebergs but not fully hidden.
2)
Wholesalers aren't allowed to give you a worse fill than the NBBO. That's the law. Often they will actually give you a tiny bit better than the NBBO on marketable orders. I think they do this so retail brokers can advertise price improvement. For most of the low cost retail platforms there is no ability to direct your order. You give it to the broker and I think basically if it's a marketable order they send it to the wholesaler and if not they post it to an exchange that offers a liquidity adding rebate (the brokerage keeps the rebate, natch). If you are willing to spend a little more on your brokerage account, you can use a platform like Interactive Brokers which does let you route orders directly.
A U.S. equities exchange has to be registered with and approved by the SEC. There are currently 12 of them. As a registered exchange you have to play by certain rules, and in exchange your displayed top of of book is subject to the order protection rule, meaning that no other venues (exchanges, dark pools, ECNs, or broker dealers) are allowed to trade through your top of book quote without first routing an order to you for the full amount of the displayed size. Exchanges also get a share of SIP revenue (which is important to exchange economics).
In it's most technical definition, a dark pool is an ATS (specifically not an exchange) that has no displayed quote i.e. no market data whatsoever. Most ATSs in the US fit this definition, though there are also ATSs that display a quote. For example, IEX is an ATS that has feed for their top of book quote... so technically not a "dark" pool.
NYSE doesn't run a dark pool and it would be weird if they did. Most US equity exchanges support fully hidden (completely non-displayed) orders, so if you want to trade in the "dark" on NYSE you can just send a hidden order. I think this is one area where futures exchanges and equity exchanges differ. In futures you have icebergs but not fully hidden.
2)
Wholesalers aren't allowed to give you a worse fill than the NBBO. That's the law. Often they will actually give you a tiny bit better than the NBBO on marketable orders. I think they do this so retail brokers can advertise price improvement. For most of the low cost retail platforms there is no ability to direct your order. You give it to the broker and I think basically if it's a marketable order they send it to the wholesaler and if not they post it to an exchange that offers a liquidity adding rebate (the brokerage keeps the rebate, natch). If you are willing to spend a little more on your brokerage account, you can use a platform like Interactive Brokers which does let you route orders directly.
How does priority work on fully hidden orders? Are they always at the back of the line?
Thanks for the clarification on the NBBO ramifications on wholesalers. I have never worked on a order flow trade and would love to see an insider account of them (its sort of what I was hoping Flash Boys would be about).
Thanks for the clarification on the NBBO ramifications on wholesalers. I have never worked on a order flow trade and would love to see an insider account of them (its sort of what I was hoping Flash Boys would be about).
Yeah for a given price, fully hidden and the non-displayed portion of iceberg orders will be ranked (generally by time priority) behind any displayed shares. Most of the rules are meant to incentivize displayed size.
Non-displayed orders and the non-displayed portion of iceberg orders are generally put in the back of the line behind all displayed liquidity. There are some exceptions, such as the BATS "hide not slide" order.
The case we are talking about however, I believe, is where NYSE operates an apparently public market with delayed quotes for some players. Tptacek claims that my orders cannot even make it into the supposedly public market because someone will scoop up my order on the way, before it even reaches the delayed quote market and therefore, the market is not special because I cannot access it (which is disputable).
Yes: if you, as an individual, place an order at an online brokerage, your order is most probably not going to make it to an exchange. It would be stupid to send it there, because wholesalers will give you better prices.
This is the case for retail orders, but not for trading firms. Prop trading firms don't tend to route their orders through wholesalers; they have deals with execution platforms (or implement their own) and route their orders to exchanges.
But there's a whole 'nother set of reasons why we're not especially concerned about competitions between prop trading firms on speed/latency/whatever: if they're impacted by HFT at all, they're competing in a zero-sum game to arb prices, and there's no moral reason why a slower trading firm is entitled to some equitable share of all the available profit from some arbitrage strategy.
This is the case for retail orders, but not for trading firms. Prop trading firms don't tend to route their orders through wholesalers; they have deals with execution platforms (or implement their own) and route their orders to exchanges.
But there's a whole 'nother set of reasons why we're not especially concerned about competitions between prop trading firms on speed/latency/whatever: if they're impacted by HFT at all, they're competing in a zero-sum game to arb prices, and there's no moral reason why a slower trading firm is entitled to some equitable share of all the available profit from some arbitrage strategy.
Tptacek seems quite convinced that the "tiny spread" pocketed by HFT arbitrageurs comes entirely out of the pocket of competing HFT shops and has no impact on the prices paid by other participants.
So, no? From 2008-2012, if you paid NYSE (one of like 15 different exchanges that trade NYSE-listed stocks) for a proprietary feed --- something that costs more than most idle home day traders can afford, but would be affordable by any YC startup --- you were getting an edge in competing with other electronic trading systems.
If you were a retail investor or a mutual fund, what NYSE was doing probably didn't impact you at all.
If you were a retail investor or a mutual fund, what NYSE was doing probably didn't impact you at all.
> what NYSE was doing probably didn't impact you at all.
Then where does all the money HFT firms make come from?
Then where does all the money HFT firms make come from?
First, HFT firms aren't especially lucrative compared to other finance specialties, so one answer to that question is "nowhere".
Second, HFT firms compete with other finance firms, so what money they do make comes from bidding down the costs those firms were imposing on the rest of the market.
If you're a retail trader, automated electronic traders make money off you by outbidding the markets to quote good prices to you for your dumb market orders, and then pocketing the spread --- in other words, they make money the same way market makers have always made money; they just make less of it at an instant, but at a far larger scale.
The same is true of mutual funds, which is why Vanguard --- the firm most people would say is the most trustworthy in all of finance --- has repeatedly claimed that HFT has lowered their cost of trades.
Second, HFT firms compete with other finance firms, so what money they do make comes from bidding down the costs those firms were imposing on the rest of the market.
If you're a retail trader, automated electronic traders make money off you by outbidding the markets to quote good prices to you for your dumb market orders, and then pocketing the spread --- in other words, they make money the same way market makers have always made money; they just make less of it at an instant, but at a far larger scale.
The same is true of mutual funds, which is why Vanguard --- the firm most people would say is the most trustworthy in all of finance --- has repeatedly claimed that HFT has lowered their cost of trades.
Somethings off. If all this is is market making, why all the trouble with the microsecond latencies? Microwave links? Shared colos?
That seems like a lot of cost everyone could save on by simply having the exchange enforce some minimum timings.
That seems like a lot of cost everyone could save on by simply having the exchange enforce some minimum timings.
Because speed is an implicit and intrinsic "figure of merit" in automated market making: if you are faster than other market makers, you outcompete them.
There are two straightforward problems with microsecond-speed electronic trading:
* At very small timescales, possibly as a sort of inevitable consequence of the CAP theorem, correlations between instruments that should trade in lock step start to break down. Since you can make money correcting these mispricings, time and energy gets sunk into doing that, and that imposes costs on the rest of the market. It's uncertain how high those costs are, but they are small, and clearly a pittance compared to non-automated market making.
* Electronic trading provides very nice paychecks for pretty interesting technical work, which means some talent gets attracted to the market that would otherwise get paid less money doing something with more social value. How big a deal this is to you depends a lot on your worldview.
To me, neither of these issues seems worth the galactically high cost of restructuring all the markets in such a way that we'd reliably avoid these problems.
There are two straightforward problems with microsecond-speed electronic trading:
* At very small timescales, possibly as a sort of inevitable consequence of the CAP theorem, correlations between instruments that should trade in lock step start to break down. Since you can make money correcting these mispricings, time and energy gets sunk into doing that, and that imposes costs on the rest of the market. It's uncertain how high those costs are, but they are small, and clearly a pittance compared to non-automated market making.
* Electronic trading provides very nice paychecks for pretty interesting technical work, which means some talent gets attracted to the market that would otherwise get paid less money doing something with more social value. How big a deal this is to you depends a lot on your worldview.
To me, neither of these issues seems worth the galactically high cost of restructuring all the markets in such a way that we'd reliably avoid these problems.
That's circular reasoning: right now speed is a figure of merit because that's how the game has been designed. I'm suggesting to change the game rules.
I don't mean to get rid of automatic market making, I'm saying the exchange could just move to a system where it checkpoints every 10ms or whatever, still much faster than humans can blink, but no detriment to the algorithms. Then everyone has 10ms to come up with orders that make sense in the current market situation, instead of the ridiculous timing game.
I don't mean to get rid of automatic market making, I'm saying the exchange could just move to a system where it checkpoints every 10ms or whatever, still much faster than humans can blink, but no detriment to the algorithms. Then everyone has 10ms to come up with orders that make sense in the current market situation, instead of the ridiculous timing game.
You don't actually have 10ms to act in the scenario you describe. Because there is new information entering the world at all times, you would want to wait until the last possible microsecond before the end of the 10ms window before entering any bids.
The window does nothing to remove the need for speed.
The world is a continuous system. No matter how hard you try you can't force it to be discrete.
(There are other problems in your proposed scenario to having to do with mismatched buy vs sell volume, but we'll ignore those for now.)
The window does nothing to remove the need for speed.
The world is a continuous system. No matter how hard you try you can't force it to be discrete.
(There are other problems in your proposed scenario to having to do with mismatched buy vs sell volume, but we'll ignore those for now.)
What new information?
I think the above poster meant to imply that the 10ms delay would cover all market transactions, so if you put in an order now, then no one knows it till the next 10ms stop post.
Essentially its buffering all the data then releasing it at once, so it's not a truly continuous system anymore.
I think the above poster meant to imply that the 10ms delay would cover all market transactions, so if you put in an order now, then no one knows it till the next 10ms stop post.
Essentially its buffering all the data then releasing it at once, so it's not a truly continuous system anymore.
> What new information?
Trades occurring at other locations. Even if the other location are operating in discretized time, there's going to be variability in how fast different market participants can transmit information from one location to another. From Wolfram Alpha, Tokyo to NYC is 36ms at speed of light in vacuum and 51ms at speed of light in fiber, so someone with a faster network could be multiple discretized ticks ahead.
Trades occurring at other locations. Even if the other location are operating in discretized time, there's going to be variability in how fast different market participants can transmit information from one location to another. From Wolfram Alpha, Tokyo to NYC is 36ms at speed of light in vacuum and 51ms at speed of light in fiber, so someone with a faster network could be multiple discretized ticks ahead.
> What new information?
Stuff that happens in the real world. The stuff that all security prices are eventually based on. It's not a closed system you know?
Stuff that happens in the real world. The stuff that all security prices are eventually based on. It's not a closed system you know?
Ah, that's why I wanted an example. Your answer is fairly obvious, but it's pretty absolute too so there's no way to ask questions like "is this information actually going to affect the bid size or price of a bidder?" or "is this event important but infrequent, like quarterly reporting?"
For example, infrequent important events will result in a race for people to submit at the last microsecond, but because they're infrequent 99% of the time the 10ms buffering will effectively change the behaviour of market participants.
For example, infrequent important events will result in a race for people to submit at the last microsecond, but because they're infrequent 99% of the time the 10ms buffering will effectively change the behaviour of market participants.
At the level of a penny, which is what the minimum price change is on most US stocks, almost anything can affect a sophisticated market-maker or proprietary trader's pricing: information in the stock's own order book, the price of related securities globally like the same stock trading in foreign markets, prices of correlated stocks, index futures prices around the world, his or her risk exposure in that stock and others in a portfolio, foreign exchange and interest rate market shifts, news feeds, sentiment analysis of online sources like twitter, company exposure to oil or other commodity price changes, etc. etc.
Exactly. Instead of the game being "who communicates fastest" it would be "who can time the sample rate the best"
There are plenty of solutions to that but it wouldn't even be a problem if exchanges weren't -selling their order flow before the trades happen-
Can you give an example of an exchange that sells the order flow before it happens?
Are you perhaps confusing exchange in this case with a broker?
Are you perhaps confusing exchange in this case with a broker?
By selling order flow brokers have reduced your commissions significantly (and all the way to zero in the case of robinhood.com).
This is good for you, not bad.
This is good for you, not bad.
That's silly, because it's an universal truth. Yet they still close the exchange and the dumb traders go home to sleep.
You can design the rules to optimize for one thing or another, continuous world notwithstanding. I'm suggesting a way to optimize for something more useful.
You can design the rules to optimize for one thing or another, continuous world notwithstanding. I'm suggesting a way to optimize for something more useful.
And I'm saying that your optimization wouldn't accomplish any goals that you might have.
Let's say that happens. The system checkpoints and everyone orders off those checkpoints.
What changes? What's the end result? How do normal people or society benefit? What would have to happen for the change to be declared a success. Conversely, what would have to happen for the change to be declared a failure?
What changes? What's the end result? How do normal people or society benefit? What would have to happen for the change to be declared a success. Conversely, what would have to happen for the change to be declared a failure?
The benefit? People don't have their order front runned by algorithm which will sell it back to them at a slightly higher price, skimming money off the top to no one's benefit.
Can you explain in as much detail as possible exactly how algorithms are front-running people's orders? Can you do it with an example of a specific kind of order, preferably down to where exactly the order is being placed? Then we can dig into the specifics and evaluate the claim being made here.
You've misunderstood what I'm saying. I agree with you: timescales far below anything that provide any meaningful social value are an intrinsic figure of merit (usual definition: "often arbitrary-seeming number people compete over") of the market microstructure we have now.
I'm not saying the markets must function this way, only that they do, and it's not evidence of a giant conspiracy or grift that they do.
The rest of my comment suggests that while we could potentially quantize the markets, it doesn't seem like we have much to win from doing that, and, further, that it's likely that the problems we're trying to solve by doing that would persist.
I'm not saying the markets must function this way, only that they do, and it's not evidence of a giant conspiracy or grift that they do.
The rest of my comment suggests that while we could potentially quantize the markets, it doesn't seem like we have much to win from doing that, and, further, that it's likely that the problems we're trying to solve by doing that would persist.
Exotic quote types and being paid for making the cross is where this explanation seems to hit market realities. HFTs are able to make money on fees by having special access not only in terms of latency but in terms of ordering.
All public exchanges are required to have public approval for all of their order types and you can look up how they behave at any time.
> Exotic quote types and being paid for making the cross is where this explanation seems to hit market realities.
What exactly is an "Exotic quote type"?
Do you mean order type because there is a huge difference. If you don't understand the difference it makes it hard to take your assertion seriously.
What exactly is an "Exotic quote type"?
Do you mean order type because there is a huge difference. If you don't understand the difference it makes it hard to take your assertion seriously.
The fundamental issue is Fisher consensus - arbitrage is a feature in other words.
A couple of things:
1) It is a common fallacy that there is "the exchange" or "the market". The market is an aggregation of exchanges (and non-exchange traded instruments) across a wide variety of jurisdictions and localities. But even though there is no central coordination between all that jumble, the instruments are correlated. That is, the change in price, demand, or availability of one is a signal about the pice, demand and availability of the others.
2) The market is made up of participants who exist on a band of price/time sensitivity. That is, someone like a retail trader doesn't have extreme price sensitivity but may have time requirements. In trading that style of participant is known as "dumb" (oh traders, you so PC).
3) Market makers make all their money trading spread for time but thats not where the latency games come into play. The latency is because...
4) Smart money is price sensitive (or price informed) and they tend to be where market makers lose money. Smart money tends to be either prop traders with particular algos, large block hedge funds or other market makers. Those are the people who cause market makers to play latency games, because its a vector a market maker can control that keeps them smart. They will never be smarter than the hedge fund that knows its going to buy a bunch of shares and therefore drive up the price, but they can at least jump out of the way quicker than the other smart money.
1) It is a common fallacy that there is "the exchange" or "the market". The market is an aggregation of exchanges (and non-exchange traded instruments) across a wide variety of jurisdictions and localities. But even though there is no central coordination between all that jumble, the instruments are correlated. That is, the change in price, demand, or availability of one is a signal about the pice, demand and availability of the others.
2) The market is made up of participants who exist on a band of price/time sensitivity. That is, someone like a retail trader doesn't have extreme price sensitivity but may have time requirements. In trading that style of participant is known as "dumb" (oh traders, you so PC).
3) Market makers make all their money trading spread for time but thats not where the latency games come into play. The latency is because...
4) Smart money is price sensitive (or price informed) and they tend to be where market makers lose money. Smart money tends to be either prop traders with particular algos, large block hedge funds or other market makers. Those are the people who cause market makers to play latency games, because its a vector a market maker can control that keeps them smart. They will never be smarter than the hedge fund that knows its going to buy a bunch of shares and therefore drive up the price, but they can at least jump out of the way quicker than the other smart money.
It is a common fallacy that retail orders ever go to stock exchanges. This is a good way to determine someone telling the truth. Here's proof: http://www.nanex.net/aqck2/4704.html
If you're doing market making, note that you're more likely to get a buy order filled when the market is going down, and more likely to get a sell order filled when the market is going up. This is the flip side of getting to "capture the spread".
So in order to be successful, especially when your margins are razor thin (and they pretty much always are), you have to be very, very good at canceling your orders as quickly as possible any time it looks like the market is likely to move against you.
So in order to be successful, especially when your margins are razor thin (and they pretty much always are), you have to be very, very good at canceling your orders as quickly as possible any time it looks like the market is likely to move against you.
Can you define "not especially lucrative"?
I.E. how much money is a good HFT firm making at scale?
I.E. how much money is a good HFT firm making at scale?
Virtu, one of the very largest HFT firms, had revenues of ~$750MM, with net income around $120mm, in 2014. Goldman Sachs made $34bn.
http://ir.virtu.com/results.cfm
Actual results for 2015. Your numbers are a fair bit off of those. What you can see from the results is that their revenue/profit numbers are small in real terms, but their margins are big compared to other businesses, even technology ones.
Personally, I'm not offended by that and think that the entire conversation about HFT is funny given how little money is actually involved, but if you view profitability in any context as evil or rent seeking, then you have ammunition for that argument with Virtu.
[edit]
Another good one to look at is Knight (Getco) http://investors.kcg.com/phoenix.zhtml?c=105070&p=irol-SECTe....
Had a much better year last year vs the previous 2 and it still was a fundamentally little amount of money.
[edit again]
That Knight statement is very interesting because it shows that the vast amount of their profit came from execution services and that their market making business was very low margin. Now that could be because their market making business is taking the full shot of the infrastructure accounting that enables the execution services business, but its interesting none the less.
Actual results for 2015. Your numbers are a fair bit off of those. What you can see from the results is that their revenue/profit numbers are small in real terms, but their margins are big compared to other businesses, even technology ones.
Personally, I'm not offended by that and think that the entire conversation about HFT is funny given how little money is actually involved, but if you view profitability in any context as evil or rent seeking, then you have ammunition for that argument with Virtu.
[edit]
Another good one to look at is Knight (Getco) http://investors.kcg.com/phoenix.zhtml?c=105070&p=irol-SECTe....
Had a much better year last year vs the previous 2 and it still was a fundamentally little amount of money.
[edit again]
That Knight statement is very interesting because it shows that the vast amount of their profit came from execution services and that their market making business was very low margin. Now that could be because their market making business is taking the full shot of the infrastructure accounting that enables the execution services business, but its interesting none the less.
From the source you gave:
> Full Year 2015:
> Net Income of $197.5 million
> Adjusted Net Trading Income of $500.7 million
His numbers seem pretty accurate.
> Full Year 2015:
> Net Income of $197.5 million
> Adjusted Net Trading Income of $500.7 million
His numbers seem pretty accurate.
He edited it down an order of magnitude (and rightly so, it was clearly a typo).
That said, whenever any trading entity talks about their trading income, be suspicious, because that is usually just the difference between bought/sold (and fees) and doesn't account for the infrastructure costs of trading.
Thats why I like these public SEC documents so much, because they have to use GAAP so you know exactly what they mean.
That said, whenever any trading entity talks about their trading income, be suspicious, because that is usually just the difference between bought/sold (and fees) and doesn't account for the infrastructure costs of trading.
Thats why I like these public SEC documents so much, because they have to use GAAP so you know exactly what they mean.
You are wrong. I'm sure you mean $750M, but they only have 200 employees, Goldman has 40,000. HFT is extremely lucrative.
Thanks for correcting; yes, I meant that Virtu makes a small fraction of what Goldman makes. Goldman, of course, is not principally an HFT firm.
Does that list include the HFT arms of the bigger traditional firms? Goldman bought into Perseus[1], so I assume if Perseus counts for them, but I imagine there may be some larger firms with HFT divisions that don't report separately.
1: http://www.bloomberg.com/news/articles/2015-04-21/goldman-sa...
1: http://www.bloomberg.com/news/articles/2015-04-21/goldman-sa...
> First, HFT firms aren't especially lucrative compared to other finance specialties, so one answer to that question is "nowhere".
Total bullshit. If there was so little money to be made why is every exchange catering to high frequency traders? Why was the building across from the NYSE hollowed out to become a data center? Why are custom fiber optic cables being layed down specifically for it?
> If you're a retail trader, automated electronic traders make money off you by outbidding the markets to quote good prices to you for your dumb market orders, and then pocketing the spread --- in other words, they make money the same way market makers have always made money; they just make less of it at an instant, but at a far larger scale.
This seems to be typical of HFT apologists. They start to use terms that are defined in other terms that they've essentially made up, much like trying to get to the first principles of a religion. There are no 'market makers', there are people selling with a miminum price and people buying with a maximum price. They aren't outbidding the markets, they are front running by seeing an order, buying it, and then selling it to the original buyer. Everything else is there to obscure the truth. I've seen so many of these posts here and none of them has ever approached any sort of justification or validity.
Total bullshit. If there was so little money to be made why is every exchange catering to high frequency traders? Why was the building across from the NYSE hollowed out to become a data center? Why are custom fiber optic cables being layed down specifically for it?
> If you're a retail trader, automated electronic traders make money off you by outbidding the markets to quote good prices to you for your dumb market orders, and then pocketing the spread --- in other words, they make money the same way market makers have always made money; they just make less of it at an instant, but at a far larger scale.
This seems to be typical of HFT apologists. They start to use terms that are defined in other terms that they've essentially made up, much like trying to get to the first principles of a religion. There are no 'market makers', there are people selling with a miminum price and people buying with a maximum price. They aren't outbidding the markets, they are front running by seeing an order, buying it, and then selling it to the original buyer. Everything else is there to obscure the truth. I've seen so many of these posts here and none of them has ever approached any sort of justification or validity.
First, let me gingerly remind you that strong arguments don't appear stronger when they include things like "total bullshit" and name-calling.
Second: I didn't say HFT wasn't lucrative at all. I said it wasn't lucrative compared to other finance specialties. I would feel worse about this misunderstanding if the thread you were commenting on didn't include actual numbers backing this up.
Third: the ultimate reason exchanges "cater" to "HFT" is that they get paid based on order volume. That's what it means to be an exchange. Exchanges compete with each other. If nobody trades on your exchange, you don't make much money with it. Exchanges would rather you do business with them than not. It's not complicated.
Fourth: if you're in the normal demographic of HN (age 20-35 or so), you've never bought or sold a share of stock that went through a human market maker. Market-making is automated now and has been for something like two decades.
Fifth: if you're an individual buying and selling stocks for your own account by clicking buttons in an online brokerage, you are placing market orders. There aren't "minimum" and "maximum" prices. There's a current best bid, and a current best offer, and a gap between those two prices. We have decades worth of research on what automated trading is doing to that gap: it is slashing it from dollars to pennies.
Sixth: if you are placing limit orders, there is nothing an HFT or anyone else can do to "front-run" you to cheat on your order limit. Incoming orders are priced according to the price on the order resting on the book. When you place a limit order that doesn't immediately execute (in other words: when you place a limit order that is actually meaningful), that's the price your order is going to trade for, if it ever trades.
If you disagree with any of this, it would be helpful if you could lay out a hypothetical sequence of events in which this "front-running" is actually occurring. I place a LIMIT SELL order for 1000 CSCO on BATS at $27.90. An HFT decides to front-run me. What exactly happens next?
Second: I didn't say HFT wasn't lucrative at all. I said it wasn't lucrative compared to other finance specialties. I would feel worse about this misunderstanding if the thread you were commenting on didn't include actual numbers backing this up.
Third: the ultimate reason exchanges "cater" to "HFT" is that they get paid based on order volume. That's what it means to be an exchange. Exchanges compete with each other. If nobody trades on your exchange, you don't make much money with it. Exchanges would rather you do business with them than not. It's not complicated.
Fourth: if you're in the normal demographic of HN (age 20-35 or so), you've never bought or sold a share of stock that went through a human market maker. Market-making is automated now and has been for something like two decades.
Fifth: if you're an individual buying and selling stocks for your own account by clicking buttons in an online brokerage, you are placing market orders. There aren't "minimum" and "maximum" prices. There's a current best bid, and a current best offer, and a gap between those two prices. We have decades worth of research on what automated trading is doing to that gap: it is slashing it from dollars to pennies.
Sixth: if you are placing limit orders, there is nothing an HFT or anyone else can do to "front-run" you to cheat on your order limit. Incoming orders are priced according to the price on the order resting on the book. When you place a limit order that doesn't immediately execute (in other words: when you place a limit order that is actually meaningful), that's the price your order is going to trade for, if it ever trades.
If you disagree with any of this, it would be helpful if you could lay out a hypothetical sequence of events in which this "front-running" is actually occurring. I place a LIMIT SELL order for 1000 CSCO on BATS at $27.90. An HFT decides to front-run me. What exactly happens next?
>They aren't outbidding the markets, they are front running by seeing an order, buying it, and then selling it to the original buyer.
Please step me through this scenario. Exactly what do you think is happening, in what order and how?
Please step me through this scenario. Exactly what do you think is happening, in what order and how?
He's right - if you weren't part of the group that was paying them over $100M/year in fees for "real-time" data and you weren't buying or selling stock in 2008, 2009, 2010 and 2011, it probably didn't impact you at all.
The people making money on the HFT stuff are the exchanges. The HFT people are making realtively easy money, but require vast bulk to do so.
In 2010, internalizers ABSOLUTELY, WITHOUT QUESTION, priced retail investor orders with SIP prices (the feed that got way behind). What the NYSE was doing:
ABSOLUTELY. POSITIVELY. Impacted retail investors and mutual funds.
ABSOLUTELY. POSITIVELY. Impacted retail investors and mutual funds.
Shouldn't internalizers be required to use direct feeds instead of the SIP? The SIP, by construction, will always be lagging the 'real' (if that even means anything) NBBO.
Internalizers use both. The give retail the slower price, and buy and sell on their own account at the faster price. Nanex has uncovered a document from the NYSE stating that this is the case.
Let me be more clear -- if internalizers are regulated to use anything, why aren't they regulated to use the direct feeds specifically? If the regulation specifically allows them to use the SIP, that seems to be a failure of the regulation itself.
What NYSE was doing seems to have increased frothiness of the markets. Your median individual investor might not pay a direct price for this, but it certainly could increase costs in subtle ways. Or simply make the market more frothy in the macro view as well as the micro (nano?) view.
Further, some whales (pension funds) might get hurt and that hurts your median citizen. Many hedge funds seem to have a strategy of whale-hunting. Thus dark pools and all that.
Further, some whales (pension funds) might get hurt and that hurts your median citizen. Many hedge funds seem to have a strategy of whale-hunting. Thus dark pools and all that.
Is this the same thing that was discussed in the documentary Wall Street Code (http://topdocumentaryfilms.com/wall-street-code/) where certain types of order/purchase types actually bumped your position in the queue?
No.
its pretty amazing how few people seem to have base domain knowledge of core market infrastructure considering how many people work in that area and that it isn't particularly tricky. It certainly colours the opinions of everyone here jumping to entirely silly conclusions.
I can read (and have read) Volumes 1, 2, and 3 of The Art of Computer Programming and thereby acquire a considerable amount of "base domain knowledge" in that area.
I'm not aware of a correspondingly simple way of acquiring base domain knowledge of the topics we are discussing here. Perhaps you can suggest some books?
I'm not aware of a correspondingly simple way of acquiring base domain knowledge of the topics we are discussing here. Perhaps you can suggest some books?
This is OK as a book, I read much of it many years ago:
The Complete Guide to Capital Markets for Quantitative Professionals (McGraw-Hill Library of Investment & Finance) Hardcover – 1 Dec 2006 by Alex Kuznetsov
A book on trading is prob more useful. There are many public specs of exchanges. For example:
http://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3...
The main thing to learn is orderbook building mechanics
The Complete Guide to Capital Markets for Quantitative Professionals (McGraw-Hill Library of Investment & Finance) Hardcover – 1 Dec 2006 by Alex Kuznetsov
A book on trading is prob more useful. There are many public specs of exchanges. For example:
http://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3...
The main thing to learn is orderbook building mechanics
You can also come to the same conclusion if you see how NYSE conducts opening and closing auctions. The presence of human specialists and d-quotes is in the same vein...
And $5mm seems like a very small penalty to pay for systematically ripping off retail investors. I wonder if there is a valid basis for a class action that could recoup more of the theoretical losses due to the delayed quotation...
Can you explain exactly how this is an instance of NYSE "systematically ripping off retail investors"? Please be as specific as you can.
The article detailed how NYSE provided delayed quotations to it public service. This means that investors without access to the real-time info unknowingly traded at a disadvantage to those who did.
In exactly what way is a retail investor harmed by the kinds of delays that this problem introduced? Please be specific.
Note: despite the fact that retail investors virtually never trade directly on exchanges, let's stipulate that they always do.
I don't think it's a good thing that the NYSE's feeds were this crappy; that's not what I'm debating.
Note: despite the fact that retail investors virtually never trade directly on exchanges, let's stipulate that they always do.
I don't think it's a good thing that the NYSE's feeds were this crappy; that's not what I'm debating.
I don't see a need to pretend that retail investors trade directly on an exchange to show harm. The reason that some firms pay for access to current information is that it gives them an advantage. An advantage against whom? Is this only a matter of HFT firms trading against each other, and they all have access to the same info? Or, is the info asymmetric and someone is disadvantaged in a trade? I have to think that at some level, HFT will result in higher prices paid by brokers and their clients, and directly or indirectly by mutual funds. It's a bit of a stretch, and I think that may explain why we haven't heard of any civil lawsuits based on these facts. Also statute of limits on some related claims may have expired.
> Or, is the info asymmetric and someone is disadvantaged in a trade?
There is no expectation in the markets that everyone is working with the same information, quite the contrary, the markets wouldn't provide value if everyone was working with the same data. One of the chief reasons the markets are valuable is that they give people incentives to surface (in the form of market activities) pricing information they might otherwise keep secret.
> I have to think that at some level, HFT will result in higher prices paid by brokers and their clients
Quite the contrary, it has led to a race to the bottom such that I can trade for free on my phone from anywhere with internet access at spreads that are nearly nothing. Robinhood is only possible because HFT has made it so.
> and directly or indirectly by mutual funds.
Vanguard, the gold standard for low priced mutual funds, completely disagrees with you.
https://www.sec.gov/comments/s7-02-10/s70210-122.pdf
There is no expectation in the markets that everyone is working with the same information, quite the contrary, the markets wouldn't provide value if everyone was working with the same data. One of the chief reasons the markets are valuable is that they give people incentives to surface (in the form of market activities) pricing information they might otherwise keep secret.
> I have to think that at some level, HFT will result in higher prices paid by brokers and their clients
Quite the contrary, it has led to a race to the bottom such that I can trade for free on my phone from anywhere with internet access at spreads that are nearly nothing. Robinhood is only possible because HFT has made it so.
> and directly or indirectly by mutual funds.
Vanguard, the gold standard for low priced mutual funds, completely disagrees with you.
https://www.sec.gov/comments/s7-02-10/s70210-122.pdf
How do you know Vanguard disagrees with me when I haven't even stated a position. I'm posing a loaded question hoping that someone can make an intelligent and factual argument as to why the sanctioned behavior that the SEC believes is illegal does or does not in fact cause any harm to retail investors. The Nanex commenter does that pretty well, and I think frames the proper issue (if we care about the "morality" of this) as not having to do with HFT at all. If we accept this, it means that my earlier comments here were irrelevant to the topic, as are yours and the comments by other HFT defenders in this thread.
Perhaps he is referring to your comment I have to think that at some level, HFT will result in higher prices paid by brokers and their clients
From the Vanguard comments: This roughly 25% decrease in the end value of the investment demonstrates the impact of reduced transaction costs on longterm investors.
So perhaps you didn't state a position, but you posed a thought (I have to think that at some level) or question and Vanguard rather decisively answers that question.
From the Vanguard comments: This roughly 25% decrease in the end value of the investment demonstrates the impact of reduced transaction costs on longterm investors.
So perhaps you didn't state a position, but you posed a thought (I have to think that at some level) or question and Vanguard rather decisively answers that question.
Vanguard is talking about electronic trading. Vanguard specifically states the time horizon is 10-15 years. HFT began in U.S. Stock in 2007. The biggest impact Vanguard experienced was from decimalization (going from eights to pennies, well before HFT). More info/proof: http://www.nanex.net/aqck2/3532.html
[deleted](1)
What he's saying there was that due to inefficiencies in NYSE's infrastructure, the feed that went to the consolidation (which provides the NBBO) was delayed. Firms were able to then exploit those inefficiencies by dumping a ton of quotes into the market to cause it slow down those feeds. This is called quote-stuffing and is illegal and against exchange rules. (I don't know if this was illegal then.)
From a technical perspective, as orders are sent to the exchange, NYSE sends them out on a direct line to subscribers who listen to that feed. This is not private information, but it is relatively expensive, given you'd need to have servers at a colo, and pay for fast networking, boxes, software, etc.
They also then aggregate that data and send them out on a feed to the NBBO which is much more accessible to the majority of traders. Obviously, it is somewhat delayed, as they need to aggregate, and a firm that can aggregate the data faster than NYSE thereby has an advantage. This aggregated data gets set out as soon as possible, and NBBO defines a time range that they need to be accurate. For what its worth, NBBO means "National Best Bid and Offer" and they will aggregate bids and offers from all exchanges and provide the best one in a single consolidated feed.
Nanex determined that at certain times during high volume, Nasdaq's aggregate feed was delayed by up to 30 seconds, and this was cause by certain entities quote stuffing (which like I said earlier, is against the rules).
That is all that was determined. There is nothing about trading on private information (which there is none here) or front-running (which HFT firms can't do) or any of the other nonsense that gets spewed out.
If anything, other firms with good infrastructure would have been able to arb out the inefficiency and make the markets right again, quickly.
From a technical perspective, as orders are sent to the exchange, NYSE sends them out on a direct line to subscribers who listen to that feed. This is not private information, but it is relatively expensive, given you'd need to have servers at a colo, and pay for fast networking, boxes, software, etc.
They also then aggregate that data and send them out on a feed to the NBBO which is much more accessible to the majority of traders. Obviously, it is somewhat delayed, as they need to aggregate, and a firm that can aggregate the data faster than NYSE thereby has an advantage. This aggregated data gets set out as soon as possible, and NBBO defines a time range that they need to be accurate. For what its worth, NBBO means "National Best Bid and Offer" and they will aggregate bids and offers from all exchanges and provide the best one in a single consolidated feed.
Nanex determined that at certain times during high volume, Nasdaq's aggregate feed was delayed by up to 30 seconds, and this was cause by certain entities quote stuffing (which like I said earlier, is against the rules).
That is all that was determined. There is nothing about trading on private information (which there is none here) or front-running (which HFT firms can't do) or any of the other nonsense that gets spewed out.
If anything, other firms with good infrastructure would have been able to arb out the inefficiency and make the markets right again, quickly.
Your summary is incorrect; the 30 second delay was six years ago and was caused by selling access to real-time data vs delaying the data for the public. Quote stuffing resulted in delays of hundreds of milliseconds:
> Once again, we detected sizable delays between time-stamps
> in the public quote and Open Book. These delays ranged in
> the hundreds of milliseconds. Though the delay magnitude was
> far lower than the tens of seconds discovered during the flash
> crash, it was still hundreds of times higher than expected.Regular quote stuffing generally delayed by hundreds of milliseconds, which is a pretty bad sign regarding their infrastructure.
The 30 second delay was only during the flash crash. For an aggregate feed that's not aggregating very efficiently, that would make sense, as there was an abnormally high amount of volume during the crash.
The 30 second delay was only during the flash crash. For an aggregate feed that's not aggregating very efficiently, that would make sense, as there was an abnormally high amount of volume during the crash.
I've seen a number of comments here suggesting that the data feeds are exhorbanently expensive.
FWIW I'm getting the 'real-time NYSE data' (not the level two data but real time level one) for $2 / month on my tradingview membership. Not exactly breaking the bank
FWIW I'm getting the 'real-time NYSE data' (not the level two data but real time level one) for $2 / month on my tradingview membership. Not exactly breaking the bank
while that might be true, the data is only part of what you pay for.
The speed and format of the data is the other, and major component of what you pay for.
You are paying for what we'd call display only quotes, meaning you get these quotes so a human can visualize where the market is. If your strategies are based on trading once a day then this is fine for you.
The other way in which you will be throttled is how many quotes you can view at once.
If you trade options you won't be happy with only being able to view, say, 20 SPY contracts at a time as there are 1000's of contracts for sale.
Ideally you'd be able to view 20 contracts per date( once a week) multiplied by 2 for both puts and calls over the next 3 months, which is 20 * 2(put and call) * 12 (3 months of 4 week contracts).
This is alot of data to stream over the internet considering that those quotes probably change 200 time a second each.
The speed and format of the data is the other, and major component of what you pay for.
You are paying for what we'd call display only quotes, meaning you get these quotes so a human can visualize where the market is. If your strategies are based on trading once a day then this is fine for you.
The other way in which you will be throttled is how many quotes you can view at once.
If you trade options you won't be happy with only being able to view, say, 20 SPY contracts at a time as there are 1000's of contracts for sale.
Ideally you'd be able to view 20 contracts per date( once a week) multiplied by 2 for both puts and calls over the next 3 months, which is 20 * 2(put and call) * 12 (3 months of 4 week contracts).
This is alot of data to stream over the internet considering that those quotes probably change 200 time a second each.
Ya. I don't have any exposure to options currently. Really I'm barely even a swing trader more of an investor. I did notice that the price could go up substantially if you were looking for data on futures / e minis etc.
"That is all that was determined."
Do you work for the SEC? Because only the SEC knows what I sent them. If you do work for the SEC, then you have some explaining to do. If you don't work for the SEC, you have some explaining to do. I thought Hacker News was a place for logic and reasoning. Yet it is so easy to pick these inconsistencies out. Why do you suppose that is?
Do you work for the SEC? Because only the SEC knows what I sent them. If you do work for the SEC, then you have some explaining to do. If you don't work for the SEC, you have some explaining to do. I thought Hacker News was a place for logic and reasoning. Yet it is so easy to pick these inconsistencies out. Why do you suppose that is?
So essentially the exchanges offer multiple data feeds. These feeds are differentiated by a couple of points.
1) How much market data they give out. Typically this is broken into 3 levels:
The other way they differentiate is the speed at which you get updates.
Have a look here to see what just the Nasdaq offers. http://www.nasdaqtrader.com/Trader.aspx?id=FeedMIPS
You can spend anywhere from $500/month to $75,000/month to receive market data from just the Nasdaq, one of 50 trading venues in the US.
As you can imagine the difference is speed. The slower methods will use the text based FIX protocol and the fastest methods give you a feed and an FPGA to parse the feed with.
It's this later feed that is getting the exchanges into trouble as these feeds notify the user of trades before the SIP can be updated which lets firms have a 300 millisecond second peek at the market before everyone else does.
Couple these faster feeds with many dark pools not reporting trades in a timely manner and new order types that allow firms to sweep liquidity at one exchange only( ie not rout-able orders) and you get a situation where latency arb can thrive.
1) How much market data they give out. Typically this is broken into 3 levels:
- Level 1, Just the top of book quote, this is the best bid and offer and their corresponding number of shares at this price level.
- Level 2 Depth of market at each price level, same as above but you can see all liquidity offered.
- Level 3 same as Level 2 but you can see each individual order that contributes to each price level
The other way they differentiate is the speed at which you get updates.
Have a look here to see what just the Nasdaq offers. http://www.nasdaqtrader.com/Trader.aspx?id=FeedMIPS
You can spend anywhere from $500/month to $75,000/month to receive market data from just the Nasdaq, one of 50 trading venues in the US.
As you can imagine the difference is speed. The slower methods will use the text based FIX protocol and the fastest methods give you a feed and an FPGA to parse the feed with.
It's this later feed that is getting the exchanges into trouble as these feeds notify the user of trades before the SIP can be updated which lets firms have a 300 millisecond second peek at the market before everyone else does.
Couple these faster feeds with many dark pools not reporting trades in a timely manner and new order types that allow firms to sweep liquidity at one exchange only( ie not rout-able orders) and you get a situation where latency arb can thrive.
> It's this later feed that is getting the exchanges into trouble as these feeds notify the user of trades before the SIP can be updated which lets firms have a 300-700 nano second peek at the market before everyone else does.
From the article:
Specifically, we found that stock quotes from the NYSE were delayed by more than 30 seconds to the public quotation feed (chart 1), relative to Open Book, which is NYSE's expensive direct feed product used mostly by High Frequency Traders (HFT).
That indicates it was quite a bit more than 300-700 nano seconds at it's worst, by a few orders of magnitude. The chart showing the delay over time in their testing period shows a delay of 200ms routinely. That's way more than a few hundred nanoseconds.
From the article:
Specifically, we found that stock quotes from the NYSE were delayed by more than 30 seconds to the public quotation feed (chart 1), relative to Open Book, which is NYSE's expensive direct feed product used mostly by High Frequency Traders (HFT).
That indicates it was quite a bit more than 300-700 nano seconds at it's worst, by a few orders of magnitude. The chart showing the delay over time in their testing period shows a delay of 200ms routinely. That's way more than a few hundred nanoseconds.
Yes, but to be clear that was a very special situation that hadn't happened before or since. They were essentially fined for not having good developers.
The NYSE has always had a reputation of being awfully inept from a technology standpoint, though they have made strides in the past few years.
That particular time was made even worse by the direct feed holding up much better that their SIP feed, though the direct feed was also slower than normal.
An accurate title might be NYSE fined for having a software bug that slowed down one feed occurred during the busiest time in the markets history.
The NYSE has always had a reputation of being awfully inept from a technology standpoint, though they have made strides in the past few years.
That particular time was made even worse by the direct feed holding up much better that their SIP feed, though the direct feed was also slower than normal.
An accurate title might be NYSE fined for having a software bug that slowed down one feed occurred during the busiest time in the markets history.
> Yes, but to be clear that was a very special situation that hadn't happened before or since.
If by a special situation, you mean directly observable at multiple time periods. Here's what the article says:
To satisfy a curiosity of whether the NYSE public quote delay was unique to May 6, 2010, we ran another quote-by-quote comparison of time-stamps from the public quote and Open Book for a 30 minute trading period in General Electric stock (GE) on July 21, 2010 (see chart 2). We chose this period because there was a noticeable lag in the public quote from the NYSE versus quotes from other exchanges, allowing us to rule out the consolidation process as a source of the delay.
So they were able to find another time period exhibiting the same behavior as the flash crash at a later date, and it doesn't appear they had to look very hard (but there aren't a lot of details on that). Thirty seconds is a crazy amount of lag, but that was during the "special situation". That they look to have routinely had lag well into the hundreds of milliseconds is more troubling, because there's no longer a special situation to point towards as an excuse.
If by a special situation, you mean directly observable at multiple time periods. Here's what the article says:
To satisfy a curiosity of whether the NYSE public quote delay was unique to May 6, 2010, we ran another quote-by-quote comparison of time-stamps from the public quote and Open Book for a 30 minute trading period in General Electric stock (GE) on July 21, 2010 (see chart 2). We chose this period because there was a noticeable lag in the public quote from the NYSE versus quotes from other exchanges, allowing us to rule out the consolidation process as a source of the delay.
So they were able to find another time period exhibiting the same behavior as the flash crash at a later date, and it doesn't appear they had to look very hard (but there aren't a lot of details on that). Thirty seconds is a crazy amount of lag, but that was during the "special situation". That they look to have routinely had lag well into the hundreds of milliseconds is more troubling, because there's no longer a special situation to point towards as an excuse.
Umm, I think what you posted proves my point. The 30 second delay was a complete aberration. 100 millisecond delays happen all the time at every exchange.
To be fair to you I originally posted nano second when I meant millisecond. I've corrected my post.
As I've said before, I'm not at an HFT firm but we routinly see latency changes in messagesfrom all excahnges and dark pools.
Just like Google can't guarantee each query will be served in 250 milliseconds, each exchange can't guarantee every message will be delivered in nano seconds.
I'm good with this.
To be fair to you I originally posted nano second when I meant millisecond. I've corrected my post.
As I've said before, I'm not at an HFT firm but we routinly see latency changes in messagesfrom all excahnges and dark pools.
Just like Google can't guarantee each query will be served in 250 milliseconds, each exchange can't guarantee every message will be delivered in nano seconds.
I'm good with this.
> To be fair to you I originally posted nano second when I meant millisecond. I've corrected my post.
Ah, that makes a big difference. :)
> As I've said before, I'm not at an HFT firm but we routinly see latency changes in messagesfrom all excahnges and dark pools.
From separate feeds to the same exchange? My understanding from the article is that that is not allowed:
A crucial sub-ruling in the regulations prohibits exchanges from giving stock quotes to special groups faster than to the public.
So as long as you're measuring from the start of the request, not the end, you shouldn't see delays in the hundreds of milliseconds to the same exchange, regardless of feed. At least that's how I understand it, but you sound like you have more practical experience, and may be able to correct where I'm misinterpreting something. Is that not how it works in real life?
Ah, that makes a big difference. :)
> As I've said before, I'm not at an HFT firm but we routinly see latency changes in messagesfrom all excahnges and dark pools.
From separate feeds to the same exchange? My understanding from the article is that that is not allowed:
A crucial sub-ruling in the regulations prohibits exchanges from giving stock quotes to special groups faster than to the public.
So as long as you're measuring from the start of the request, not the end, you shouldn't see delays in the hundreds of milliseconds to the same exchange, regardless of feed. At least that's how I understand it, but you sound like you have more practical experience, and may be able to correct where I'm misinterpreting something. Is that not how it works in real life?
It is technology, and it broke, badly. You cant blame them for doing that on purpose if it was a side effect of a black swan event
The charts are not reporting what happened during the flash crash, they are for a separate, later event that they chose to see if the behavior was regular. It was, but not to the levels seen during the flash crash, as is to be expected from the cause.
This is not about a time when some technology broke under extreme load, this is about a broken system that persisted in that state for an extended period of time, and it was broken in a way that made it no longer compliant with the law.
This is not about a time when some technology broke under extreme load, this is about a broken system that persisted in that state for an extended period of time, and it was broken in a way that made it no longer compliant with the law.
> As you can imagine the difference is speed. The slower methods will use the text based FIX protocol and the fastest methods give you a feed and an FPGA to parse the feed with.
Err, No. You can parse FIX with an FPGA if you feel like it. It is up to you how you consume the data. The differential will be CoLocation vs dedicated line vs internet delivery on a base level and then differing subsets of feeds to varying liquidity pools.
So really you are paying whomever owns the DC for the privilege of a seat in the same DC as the exchange and for the cross connect right into their cage.
Err, No. You can parse FIX with an FPGA if you feel like it. It is up to you how you consume the data. The differential will be CoLocation vs dedicated line vs internet delivery on a base level and then differing subsets of feeds to varying liquidity pools.
So really you are paying whomever owns the DC for the privilege of a seat in the same DC as the exchange and for the cross connect right into their cage.
>Err, No. You can parse FIX with an FPGA if you feel like it.
That's true but no one said you couldn't. What I said was that FIX based feeds would be slower than binary ones. I stand behind this.
I also said that some binary feeds come with an FPGA card for parsing them, you are of course free to use your own FPGA to parse a fix feed or binary feed, but the binary feed will be faster due to it being smaller.
>The differential will be CoLocation vs dedicated line vs internet delivery on a base level and then differing subsets of feeds to varying liquidity pools.
This is wrong, but its not your fault, its clear you aren't in the industry or really understand it at all and that's ok.
Colocation is part of what you pay for, but the latency and bandwidth of the feed itself also is part of what you pay for. You'll also pay for the type of data you receive and the SLA contract that dictates how fast you'll receive your data at the 99.99% level.
Feel free to mail me if you'd like to learn more!
That's true but no one said you couldn't. What I said was that FIX based feeds would be slower than binary ones. I stand behind this.
I also said that some binary feeds come with an FPGA card for parsing them, you are of course free to use your own FPGA to parse a fix feed or binary feed, but the binary feed will be faster due to it being smaller.
>The differential will be CoLocation vs dedicated line vs internet delivery on a base level and then differing subsets of feeds to varying liquidity pools.
This is wrong, but its not your fault, its clear you aren't in the industry or really understand it at all and that's ok.
Colocation is part of what you pay for, but the latency and bandwidth of the feed itself also is part of what you pay for. You'll also pay for the type of data you receive and the SLA contract that dictates how fast you'll receive your data at the 99.99% level.
Feel free to mail me if you'd like to learn more!
new order types that allow firms to sweep liquidity at one exchange only( ie not rout-able orders)
You say that like it's a bad thing. My understanding is that an "intermarket sweep order"[1] (aka ISO) is essential to the proper functioning of the market. Otherwise, here's what used to happen all the time:
- the "true" market for XYZ is $10.00 bid, $10.01 offer. (Let's ignore the mantra of relativity that "simultaneity does not exist").
- in this case lets assume NASDAQ prices currently reflect that "true" market, and prices disseminated by NYSE are stale.
- a buy order arrives at NASDAQ, willing to buy at $10.01
- NYSE is supplying stale data, claims it has stock for sale at $10.00
- without ISO, NASDAQ can't allow the transaction to occur at $10.01, but is obligated to respect the stale data it is receiving from NYSE. NASDAQ must reject the order, or route it to NYSE.
- SUCKERS! the NYSE quote was stale, but the order was not transacted on NASDAQ. And, there is no longer any stock for sale at $10.00 at NYSE, so it won't be filled there either!
Here's an article that claims that ISOs are a form of protection against high frequency traders:[2]
[1] https://en.wikipedia.org/wiki/Intermarket_sweep_order [2] http://tabbforum.com/opinions/why-hfts-have-an-advantage-par...
You say that like it's a bad thing. My understanding is that an "intermarket sweep order"[1] (aka ISO) is essential to the proper functioning of the market. Otherwise, here's what used to happen all the time:
- the "true" market for XYZ is $10.00 bid, $10.01 offer. (Let's ignore the mantra of relativity that "simultaneity does not exist").
- in this case lets assume NASDAQ prices currently reflect that "true" market, and prices disseminated by NYSE are stale.
- a buy order arrives at NASDAQ, willing to buy at $10.01
- NYSE is supplying stale data, claims it has stock for sale at $10.00
- without ISO, NASDAQ can't allow the transaction to occur at $10.01, but is obligated to respect the stale data it is receiving from NYSE. NASDAQ must reject the order, or route it to NYSE.
- SUCKERS! the NYSE quote was stale, but the order was not transacted on NASDAQ. And, there is no longer any stock for sale at $10.00 at NYSE, so it won't be filled there either!
Here's an article that claims that ISOs are a form of protection against high frequency traders:[2]
Brokers that don’t use ISOs in fast market
conditions disadvantage their clients and
subsidize the profits of HFTs
[1] https://en.wikipedia.org/wiki/Intermarket_sweep_order [2] http://tabbforum.com/opinions/why-hfts-have-an-advantage-par...
>> new order types that allow firms to sweep liquidity at one exchange only( ie not rout-able orders).
> You say that like it's a bad thing.
Umm I said no such thing. Please don't make thing up.
The order types are useful to market makers and help prevent locked markets, which REG NMS prohibits.
I use them every day and I"m not at an HFT firm.
> You say that like it's a bad thing.
Umm I said no such thing. Please don't make thing up.
The order types are useful to market makers and help prevent locked markets, which REG NMS prohibits.
I use them every day and I"m not at an HFT firm.
Nuclear reactors are planned in a way that their response time is in human scale e.g. they take something like 20 seconds to increase or decrease activity. This leaves room for human intervention.
IMHO markets should do something similar: create something like 30-second barriers for trades. Trades that cross the bid/ask are carried out. Trades submitted cannot be canceled until the next 30s cycle. Trades outside of the range (bid too low/ask too high) stay open. The open trade book is sent to everybody every 30s.
Allowing µs-range trades (and particularly adding and removing bids and asks so fast) is asking for trouble, it is obvious that at some point of time two or three HFT robots will react on each other's trades and do something veeeeery stupid.
IMHO markets should do something similar: create something like 30-second barriers for trades. Trades that cross the bid/ask are carried out. Trades submitted cannot be canceled until the next 30s cycle. Trades outside of the range (bid too low/ask too high) stay open. The open trade book is sent to everybody every 30s.
Allowing µs-range trades (and particularly adding and removing bids and asks so fast) is asking for trouble, it is obvious that at some point of time two or three HFT robots will react on each other's trades and do something veeeeery stupid.
Even people who support quantized markets don't propose markets running on human timescales.
Meanwhile, if you're the kind of trader that is impacted by trade latency (read: a competitive market maker or decently well capitalized prop trading firm), there are already venues you can trade in that are quantized. The problem is that nobody wants to trade in them.
Meanwhile, if you're the kind of trader that is impacted by trade latency (read: a competitive market maker or decently well capitalized prop trading firm), there are already venues you can trade in that are quantized. The problem is that nobody wants to trade in them.
Not sure what you mean by "quantized market" (discrete-time auction?). But the opening and closing sessions at the NYSE run at human timescales (twice per day), and most of the daily volume is traded at open and close iirc.
edit: "Across Wall Street, less trading is taking place at 3 p.m. or, in fact, most any time but the opening minutes and the final half hour." http://www.wsj.com/articles/traders-pile-in-at-the-close-143...
edit: "Across Wall Street, less trading is taking place at 3 p.m. or, in fact, most any time but the opening minutes and the final half hour." http://www.wsj.com/articles/traders-pile-in-at-the-close-143...
Actually there have been several proposals for exchanges to work via, eg, every-N-seconds Dutch auctions.
The thing about many of our markets today is that they are international, with different rules in each jurisdiction. Supposing you implemented this system in the entire US, the HFT firms would simply shift their efforts to trading venues where order books are not quantized. Then they would take the real-time information and take advantage of the US exchanges.
Think about the billions (trillions?) of trades that occur on minescule timescales now and how often we have flash crashes. Sure, HFT algos interact with one another constantly, but flash crash investigations have uncovered both human (manual entry) and algo error as their root causes. Part of what makes flash crashes more severe is when HFT safety protocols kick in and they remove their orders from the market, causing liquidity to dry up and exacerbate the crash.
Think about the billions (trillions?) of trades that occur on minescule timescales now and how often we have flash crashes. Sure, HFT algos interact with one another constantly, but flash crash investigations have uncovered both human (manual entry) and algo error as their root causes. Part of what makes flash crashes more severe is when HFT safety protocols kick in and they remove their orders from the market, causing liquidity to dry up and exacerbate the crash.
Pretty sure this is the initial analysis from 2010 - http://www.nanex.net/FlashCrash/FlashCrashAnalysis_LOD.html
"If the average or base quote rate is around 10,000/second, then it only takes an additional 10,000 quotes/second to reach the magic 20,000 quotes/seconds where a corresponding delay is seen in NYSE quote from CQS."
This seems like a bug related to system load, so a $5M seems pretty harsh unless they knowingly ignored the issue for the benefit of their high-paying CQS customers.
"If the average or base quote rate is around 10,000/second, then it only takes an additional 10,000 quotes/second to reach the magic 20,000 quotes/seconds where a corresponding delay is seen in NYSE quote from CQS."
This seems like a bug related to system load, so a $5M seems pretty harsh unless they knowingly ignored the issue for the benefit of their high-paying CQS customers.
I don't think it's really that harsh if you look at it as a bug in a critical system. Certain fields with safety critical systems (such as medical or aviation software) face similarly strict regulations and enforcement for the good of the public. It seems to me with that with the volume of money being affected by this system that the potential danger for faults in the software would be as high as potential damages from medical software or something similar.
I have two questions about HFT:
- Is HFT a healthy or unhealthy part of the economy? Meaning does it help flatten out the highs/lows, or does it emphasise them?
- Aside from a micro-tax on trading (e.g. 1c/trade) is there any other mitigations/solutions to HFT?
- Is HFT a healthy or unhealthy part of the economy? Meaning does it help flatten out the highs/lows, or does it emphasise them?
- Aside from a micro-tax on trading (e.g. 1c/trade) is there any other mitigations/solutions to HFT?
Your first question asks whether HFT increases or decreases volatility. That's an open question, and the answer also depends on the time scale that you're looking at.
But volatility isn't the only axis on which to measure the health of the markets. The advent of electronic automated market making is also tied to drastic reduction of spreads, which are a tax that anyone trading has to pay to the market for the privilege of buying at the market.
A lot of people talk about spreads as if they were a meaningless abstraction, in sort of the same way as they talk about "liquidity". But in the 1990s, crooked human market makers (read: all market makers) rigged spreads to skim extra money off trades; look up the "odd eighths scandal". That doesn't happen anymore, and, more importantly, spreads (and their attendant cost to trading) are razor thin.
That's not the only thing automated electronic trading does to improve the markets. For instance, in the 80s and 90s, simply placing an order with a broker was extremely expensive. Today, there are no-fee brokers. Why? Because automated trading systems purchase the rights to internalize orders from retail brokers; if you're an automated market maker, retail order flow is practically risk free revenue, and all you have to do to earn it is hook your trading system up to brokers.
But volatility isn't the only axis on which to measure the health of the markets. The advent of electronic automated market making is also tied to drastic reduction of spreads, which are a tax that anyone trading has to pay to the market for the privilege of buying at the market.
A lot of people talk about spreads as if they were a meaningless abstraction, in sort of the same way as they talk about "liquidity". But in the 1990s, crooked human market makers (read: all market makers) rigged spreads to skim extra money off trades; look up the "odd eighths scandal". That doesn't happen anymore, and, more importantly, spreads (and their attendant cost to trading) are razor thin.
That's not the only thing automated electronic trading does to improve the markets. For instance, in the 80s and 90s, simply placing an order with a broker was extremely expensive. Today, there are no-fee brokers. Why? Because automated trading systems purchase the rights to internalize orders from retail brokers; if you're an automated market maker, retail order flow is practically risk free revenue, and all you have to do to earn it is hook your trading system up to brokers.
IIRC, HN's yummyfajitas goes into a lot of details in his blog posts on HFT[1]. My own understanding is that it provides liquidity and reduces bid/ask spreads, resulting in a more accurate stock valuation. I'm not sure the negatives, but I'm not a trader nor do I follow such things all that closely.
1: https://news.ycombinator.com/item?id=3852341
1: https://news.ycombinator.com/item?id=3852341
It's a mixed bag. On the one hand, we really do get a lot more liquidity thanks to having tons of fast market makers. You can get subsecond fills on most marketable or close-to-marketable (eg; limit orders within the spread) on most common stock listed on the NYSE/NASDAQ.
But it's often not "real" liquidity. Let's say there's 100 shares available at $100.00, 200 at $100.50, etc. Such that there's a total of 5k shares on the book at this time at any price. But we want to buy 10k shares.
The average fill price assuming no intervention and only marketable orders would be (100$100 + 200$100.50 + ...)/5000.
An intervening HFT firm will only be adding useful liquidity if the average fill price drops somehow. Shittier algos will lose some money by lowering the average fill price so they match your order and get the liquidity rebate from the exchange. This adds true liquidity. But intelligent market makers generally will just add identical liquidity to what's already available but at 1 or 2 cents higher (and a few slots higher on the order book). As a result, you just end up paying more for the same quantity of shares. That's not true liquidity because it's really only triggered by your order, doesn't move the price significantly, and is limited to exactly the size of your order.
But it's often not "real" liquidity. Let's say there's 100 shares available at $100.00, 200 at $100.50, etc. Such that there's a total of 5k shares on the book at this time at any price. But we want to buy 10k shares.
The average fill price assuming no intervention and only marketable orders would be (100$100 + 200$100.50 + ...)/5000.
An intervening HFT firm will only be adding useful liquidity if the average fill price drops somehow. Shittier algos will lose some money by lowering the average fill price so they match your order and get the liquidity rebate from the exchange. This adds true liquidity. But intelligent market makers generally will just add identical liquidity to what's already available but at 1 or 2 cents higher (and a few slots higher on the order book). As a result, you just end up paying more for the same quantity of shares. That's not true liquidity because it's really only triggered by your order, doesn't move the price significantly, and is limited to exactly the size of your order.
Wait, that is still providing liquidity. Its just doing it at a different price, which is the rational thing to do in the face of a multi-level clear.
Why wouldn't the sudden appearance of a big buyer that wanted more liquidity at near levels than is available indicate the price should go up? That's supply and demand. Am I missing something about your argument?
Why wouldn't the sudden appearance of a big buyer that wanted more liquidity at near levels than is available indicate the price should go up? That's supply and demand. Am I missing something about your argument?
We'd have liquidity without this endless war on getting closer to zero. They are adding nothing and soaking up qualified grads who could be building something useful.
This is a great example of how the modern economy fails to link wealth created with wealth extracted.
This is a great example of how the modern economy fails to link wealth created with wealth extracted.
That's true, but it's true of a lot of other things (for instance: huge fractions of all the devs working at Google and Facebook). Meanwhile, trading tech is (IMO) much more fun to work on than a lot of the work Googlers do, lucrative, and not concentrated in the Bay Area.
I'd rather we found a way to get everyone to work on drug discovery and civic engagement, but unfortunately, we don't generally put regulatory brakes on technologies simply because they attract developers that would otherwise be working on our pet issues.
I'd rather we found a way to get everyone to work on drug discovery and civic engagement, but unfortunately, we don't generally put regulatory brakes on technologies simply because they attract developers that would otherwise be working on our pet issues.
We should put breaks on it because it's not creating wealth. If it's not creating wealth but people are becoming wealthy it's because money is being extracted via economic rent (due to monopoly).
When firms make lots of money from trading platforms and then pay staff who go and consume real resources that is ultimately coming by transferring from wealth producers.
When firms make lots of money from trading platforms and then pay staff who go and consume real resources that is ultimately coming by transferring from wealth producers.
Economic rent? Monopoly? That was far more true of the firms HFT preys on, and not true of HFT firms at all.
The big exchanges are showing a little leg to the HFT players in exchange for kick-backs via fees.
EDIT: cannot reply due to HN rulz, hence inline...
The exchanges compete. The more asymmetric you make the platform for HFT vs the rest the more business you get.
Where would you trade? Somewhere you can win or not? Do I want a piece of that? I might let you know the incantations required if you pay me for the "service".
In theory other liquidity might move away but many ECNs have a very dominant position. For example NYSE.
EDIT: cannot reply due to HN rulz, hence inline...
The exchanges compete. The more asymmetric you make the platform for HFT vs the rest the more business you get.
Where would you trade? Somewhere you can win or not? Do I want a piece of that? I might let you know the incantations required if you pay me for the "service".
In theory other liquidity might move away but many ECNs have a very dominant position. For example NYSE.
What does this have to do with my comment, or your comment that preceded it?
You understand that exchanges compete with each other, right? The markets are now decentralized; there are like 15 different lit exchanges on which you can trade stocks.
You understand that exchanges compete with each other, right? The markets are now decentralized; there are like 15 different lit exchanges on which you can trade stocks.
From that blog post:
"Most HFTs run a market making strategy. What this means is they play both sides of the table - they take no position on whether a stock will go up or down. Instead, they try to offer securities both to buy and sell. If you want to buy, they will sell to you at $20.10. If you want to sell, they'll buy from you at $20. As long as their buys and sells match don't get too out of whack, the HFT will collect $0.10 = $20.10 - 20.00."
That does provide more immediate trades for both the buyer and seller, which the pro-HFT camp calls liquidity. Unfortunately it's usually not needed - both the buyer and seller were already in the market and willing to trade, and now they both just had an additional $0.10/share sucked out of their trade by some HFT.
"Most HFTs run a market making strategy. What this means is they play both sides of the table - they take no position on whether a stock will go up or down. Instead, they try to offer securities both to buy and sell. If you want to buy, they will sell to you at $20.10. If you want to sell, they'll buy from you at $20. As long as their buys and sells match don't get too out of whack, the HFT will collect $0.10 = $20.10 - 20.00."
That does provide more immediate trades for both the buyer and seller, which the pro-HFT camp calls liquidity. Unfortunately it's usually not needed - both the buyer and seller were already in the market and willing to trade, and now they both just had an additional $0.10/share sucked out of their trade by some HFT.
That's why market makers are also called scalpers, because they skim the $0.10 spread. But in return for paying the spread, what traders get is immediacy.
The problem is, how much immediacy do traders actually want? HFT provides immediacy in microseconds; its needed for continuous-time auctions because buyers sellers usually aren't in the market at the exact same microsecond interval (while the HFT is there at every interval).
Frequent Batch Auctions[1], where orders are matched in batches with single-price clearing (call auction style) - just like the opening and closing sessions of NYSE, but much more frequently with shorter periods of say 30 seconds, 1 second, or even 100ms per batch. Their models show that frequent batch auctions will reduce spreads even further (because it eliminates mechanical/latency arbitrage rents). In theory, retail traders should get even better prices (because the spreads will be smaller) if they are willing to give up microsecond immediacy for 1 second order matching.
http://faculty.chicagobooth.edu/eric.budish/research/HFT-Fre...
The problem is, how much immediacy do traders actually want? HFT provides immediacy in microseconds; its needed for continuous-time auctions because buyers sellers usually aren't in the market at the exact same microsecond interval (while the HFT is there at every interval).
Frequent Batch Auctions[1], where orders are matched in batches with single-price clearing (call auction style) - just like the opening and closing sessions of NYSE, but much more frequently with shorter periods of say 30 seconds, 1 second, or even 100ms per batch. Their models show that frequent batch auctions will reduce spreads even further (because it eliminates mechanical/latency arbitrage rents). In theory, retail traders should get even better prices (because the spreads will be smaller) if they are willing to give up microsecond immediacy for 1 second order matching.
http://faculty.chicagobooth.edu/eric.budish/research/HFT-Fre...
Current continuous-time trading already provides you the ability to specify very precisely how much immediacy you are willing to pay for. Thats precisely what the market types are about.
I don't particularly have an opinion about the Budish paper, other than it has some serious impacts to globally traded products and it is way more disruptive than people seem to suggest.
I do worry that fundamentally altering the way the markets work, because some participants don't understand order types is particularly troublesome.
I don't particularly have an opinion about the Budish paper, other than it has some serious impacts to globally traded products and it is way more disruptive than people seem to suggest.
I do worry that fundamentally altering the way the markets work, because some participants don't understand order types is particularly troublesome.
Simplish order types (limit/market, good till cancel, immediate or cancel, etc.) can only do so much. When you place a limit order on a continuous-time market, you'll pay exactly your limit price; but on a batch market you'll pay the clearing price (which may be less than your limit price).
The complicated types (hide-n-slide, NBBO pegs, and conditional pegs[1]) may help some players at the expense of others. The only players who wouldn't benefit from batch auctions (according to Budish et. al.) are latency arbitrageurs.
1. https://mechanicalmarkets.wordpress.com/2015/10/05/iex-peg-o...
The complicated types (hide-n-slide, NBBO pegs, and conditional pegs[1]) may help some players at the expense of others. The only players who wouldn't benefit from batch auctions (according to Budish et. al.) are latency arbitrageurs.
1. https://mechanicalmarkets.wordpress.com/2015/10/05/iex-peg-o...
> you'll pay exactly your limit price
Thats not quite true. Price improvement happens all the time on limit orders, but I'll concede you'll pay close to it if it fills.
> The complicated types (hide-n-slide, NBBO pegs, and conditional pegs[1]) may help some players at the expense of others.
This is true of limit/market/stops as well.
The Budish paper is very interesting.
What we don't know is what it would do to the spread, which is very important especially given that it will be harder to predict the clearing price. It is also a major change to the markets, virtually impossible to implement and its not clear to me what that risk is buying us, especially given that I fully expect most participants to go through an intermediate first.
Thats not quite true. Price improvement happens all the time on limit orders, but I'll concede you'll pay close to it if it fills.
> The complicated types (hide-n-slide, NBBO pegs, and conditional pegs[1]) may help some players at the expense of others.
This is true of limit/market/stops as well.
The Budish paper is very interesting.
What we don't know is what it would do to the spread, which is very important especially given that it will be harder to predict the clearing price. It is also a major change to the markets, virtually impossible to implement and its not clear to me what that risk is buying us, especially given that I fully expect most participants to go through an intermediate first.
What are some circumstances in which you'd get price improvement on a resting limit order?
You wouldn't on a resting order, but if you hit the other side you will.
In the past, either the buyer and seller had to wait longer to find a mutually acceptable price, or pay a lot more than $0.10/share. HFT is actually an improvement for both sides, because they can either 1) buy/sell faster or 2) pay lower transaction costs (and probably both!).
In the given example if a buyer wanted to buy at $20 instead of $20.10 they could easily put in a limit order for that price and wait for a counter party. That would be taking a risk though that either:
A) a counter-party never showed up B) a counter-party only showed up when the stock dropped so you end up buying on the way down.
Taking this risk is a perfectly fine thing to do and many people do this. Others would prefer to pay the market maker a small fee for providing liquidity. That fee is not being sucked out by some HFT. It's a fee for providing a service.
And BTW, thanks to automation and sophisticated trading algorithms that service is provided at EXTREMELY low cost these days. Bid/ask spreads are tiny.
A) a counter-party never showed up B) a counter-party only showed up when the stock dropped so you end up buying on the way down.
Taking this risk is a perfectly fine thing to do and many people do this. Others would prefer to pay the market maker a small fee for providing liquidity. That fee is not being sucked out by some HFT. It's a fee for providing a service.
And BTW, thanks to automation and sophisticated trading algorithms that service is provided at EXTREMELY low cost these days. Bid/ask spreads are tiny.
I don't see why we couldn't put a time delay in. Something like this: when you commit to a trade, the actual trade delayed for some N minutes, where N is a randomly-chosen value from some distribution. Wouldn't this eliminate the incentive to do HFT?
Another disincentive might be the trade tax that Bernie Sanders has proposed since it would basically affect HFT way more than anyone else.
No, it would affect everyone - that's why it's so bad. HFT traders would just adjust their minimum spreads and continue on their merry way. Sanders' FTT doesn't attack the mechanisms of HFT, it attacks the markets as a whole - HFT would suffer, but so would everyone else. If you want to hit HFT specifically you need to hit the HFT-specific behaviors - either add a per-message charge (like Canada did) or add a tax to cancelled orders.
It's worth noting that after the Canadian regulatory authorities added a per-message fee, bid-ask spreads rose, liquidity dropped, and it was retail traders who suffered the most. http://marginalrevolution.com/marginalrevolution/2014/04/a-s...
It's worth noting that after the Canadian regulatory authorities added a per-message fee, bid-ask spreads rose, liquidity dropped, and it was retail traders who suffered the most. http://marginalrevolution.com/marginalrevolution/2014/04/a-s...
I understand that a trade tax would affect everyone, but for someone like me who makes maybe a handful of trades per month it wouldn't really do that much to dissuade me from making those trades. I also don't particularly care one way or the other what happens with HFT, I was just mentioning this as a thing people are thinking about.
My big concern with the transaction tax is how it applies to indexed funds/etfs. Those trade all the time so the price impact can be significant for those. Further, depending on implementation, you could easily get taxed multiple times for that as there is a transaction to get in/out of the index, and the internal index transactions.
No, it would just raise the cost of trading for everyone else. If HFT's risk goes up they have to raise their prices.
While I am fully pro-HFT, and in fact work as a consultant for a few HFT shops, this is certainly illegal and anti-competitive.
(Somewhat akin to using game engine bugs for artificial lag creation in MMOs on the opponent side).
(Somewhat akin to using game engine bugs for artificial lag creation in MMOs on the opponent side).
Note, the SEC has a protocol for all whistleblowers. Here's the largest ever award [1] and the wiki article on the topic [2] (sorry for mobile link).
I asked my accountant friend about whether he'd ever consider whistleblowing if he saw wrongdoing. His decision criteria was based on whether he'd A) Ever be able to get hired again or B) If the size of the prize was so big he'd never have to worry about it.
[1] https://www.sec.gov/News/PressRelease/Detail/PressRelease/13...
[2] https://en.m.wikipedia.org/wiki/SEC_Office_of_the_Whistleblo...
I asked my accountant friend about whether he'd ever consider whistleblowing if he saw wrongdoing. His decision criteria was based on whether he'd A) Ever be able to get hired again or B) If the size of the prize was so big he'd never have to worry about it.
[1] https://www.sec.gov/News/PressRelease/Detail/PressRelease/13...
[2] https://en.m.wikipedia.org/wiki/SEC_Office_of_the_Whistleblo...
Anyone know what this quote means?
When the SEC called Hunsader in June 2015 to tell him about the award, he said to them "I would have accepted $1 if you simply acknowledged me at the time."
Was he initially ignored?
When the SEC called Hunsader in June 2015 to tell him about the award, he said to them "I would have accepted $1 if you simply acknowledged me at the time."
Was he initially ignored?
I was ignored by the Trading and Markets people at the SEC, but not ignored (obviously) by the Enforcement people.
This might be of interest to people:
A NYSE Speed Bump You Weren't Aware Of https://www.iextrading.com/about/press/op-ed/
A NYSE Speed Bump You Weren't Aware Of https://www.iextrading.com/about/press/op-ed/
The "speed bump" here being that NYSE's ancient creaky FIX gateway is slower than the NYSE ARCA gateway that they tell everyone to use instead. Quelle surprise! 80s text network protocol slower than 90s binary protocol: film at 11.
This is almost right. NYSE and Arca are two separate exchanges (both owned by NYSE). Both exchanges offer FIX and binary gateway protocols. Annoyingly, the Arca binary protocol and the NYSE binary protocol are not the same protocol. NYSE does not suggest using the Arca gateway to access NYSE. They suggest you use a NYSE binary gateway to access NYSE.
Dammit. I even looked this up before I wrote the comment, and got the opposite impression. (I'm a little familiar with the Arca protocol for other reasons).
Thanks for correcting me.
Thanks for correcting me.
Pretty sure NYSE's ancient creaky FIX gateway is infinitely faster than the NYSE ARCA gateway - as the latter, while more network efficient, isn't going to ever take your packets to NYSE.
NYSE and NYSE ARCA are different exchanges.
NYSE and NYSE ARCA are different exchanges.
Ultimately, the market has been setup to almost mandate participation. If you work someplace that provides a matching 401k program, and you choose not to contribute, then you're passing-up on matched dollars if you don't contribute to a 401k. If you're a part of any endowment, pension plan, or other managed retirement plan there's a good chance that plan (and your money) is being invested in the market.
So what are your options? Do you not contribute to a 401k or IRA, pay the full tax rate on all of your income today, and invest in real estate or some other investment?
So what are your options? Do you not contribute to a 401k or IRA, pay the full tax rate on all of your income today, and invest in real estate or some other investment?
Nasdaq does go through great pains to ensure that public & private feeds go out "equally"-ish on the wire whether it's a quote going out of the prop or public data feeds. At the packet level, a public quote to the SIP or a private quote via Itch are within the sub ~10u level. But once the packet leaves, the fate of that packet is surrendered to the speed of the SIP system.
There's a couple of inherent challenges.
The SIP that aggregates the data is by design a separate system from the trading systems. This is for obvious reasons, as every participant provides their quote to the SIP. So no matter how fast the SIP the is, even with a 99.999% latency of 20 microseconds, the SIP will always be 20 micros + travel time to the SIP (1 to 500 to 7000 micros) as opposed to a direct feed. Nasdaq is in Carteret. NYSE is in Mahwah. BATS is at NY4 in Secaucus, etc. So direct feeds will always be the fastest, even with a SIP that can defy the speed of light a little bit.
Nasdaq does actually read marketdata directly from other exchange participants and falls back to using the "slower SIP" data as a fall back in the event of any issues. This allows for the latest quotes to be respected.
The challenge with the SIPs being hundreds of millis behind is partly due to the criticality & member-driven consensus system in place. Each of the marketcenters NYSE, Nasdaq, Bats, etc have a stake and voting power on how the SIP would be run & operated. Since it's a system that "works" there's really no motivation to continually improve the technology/latency of the SIP. There's also a fear to change the SIP and lead to it going down. If a SIP goes down, by rule, a regulatory halt must be issued and all trading in the securities that particular SIP handles must stop. This is what occurred during the 3 hour outage several years back at Nasdaq. The trading system was fine, but since the SIP was down, everything had to be halted. Halts are detrimental to business since the respective marketcenters make money on trades. Halts = No Trades = Lost Revenue, so there is a lot of motivation to not halt by all the member exchanges.
That ambivalence to a tech refresh has since changed since the scathing critique of Flash Boys and the Nasdaq SIP is in process of being refreshed to the same architecture as the trading systems, which should bring median latencies of the SIP down into the tens of microseconds and 99% latencies into the hundreds of microseconds. Current medians and 90th percentiles are at 450/810 microseconds, which is considered an eternity.
http://www.wallstreetandtech.com/infrastructure/nasdaq-omx-w...
http://www.utpplan.com/DOC/UTP%202015-Q4%20Stats%20with%20Pr...
On the technical side, one of the challenges with synchronizing packet delivery between a private & public feed is that the absolute fastest NICs available today with kernel bypass can get a packet from wire to memory or memory to wire in about 2-3u. So every hop of a computer costs a minimum of 4-6u. Usually this is Order Port Ingress -> Matching Engine -> Order Port Egress. An FPGA NIC that does the IP Header processing and delivers data to memory can do wire to memory in about 1-2u. One-way PCI-E latency is around 800 nanos. So there's about a 500-1000 nano budget to process a packet in the FPGA, then another 800 nanos to deliver it over PCI-E to memory to be processed. The fastest cut-through switches have a port-to-port latency of around 250 nanos, so switch hops are pretty negligible in the overall latency of a system, but they do play a part. Anyway, making sure that everything is super duper optimized and running as it should be in a semi-continuously deployed environment makes for interesting and quite crazy/bull-shit level challenges.
There's a couple of inherent challenges.
The SIP that aggregates the data is by design a separate system from the trading systems. This is for obvious reasons, as every participant provides their quote to the SIP. So no matter how fast the SIP the is, even with a 99.999% latency of 20 microseconds, the SIP will always be 20 micros + travel time to the SIP (1 to 500 to 7000 micros) as opposed to a direct feed. Nasdaq is in Carteret. NYSE is in Mahwah. BATS is at NY4 in Secaucus, etc. So direct feeds will always be the fastest, even with a SIP that can defy the speed of light a little bit.
Nasdaq does actually read marketdata directly from other exchange participants and falls back to using the "slower SIP" data as a fall back in the event of any issues. This allows for the latest quotes to be respected.
The challenge with the SIPs being hundreds of millis behind is partly due to the criticality & member-driven consensus system in place. Each of the marketcenters NYSE, Nasdaq, Bats, etc have a stake and voting power on how the SIP would be run & operated. Since it's a system that "works" there's really no motivation to continually improve the technology/latency of the SIP. There's also a fear to change the SIP and lead to it going down. If a SIP goes down, by rule, a regulatory halt must be issued and all trading in the securities that particular SIP handles must stop. This is what occurred during the 3 hour outage several years back at Nasdaq. The trading system was fine, but since the SIP was down, everything had to be halted. Halts are detrimental to business since the respective marketcenters make money on trades. Halts = No Trades = Lost Revenue, so there is a lot of motivation to not halt by all the member exchanges.
That ambivalence to a tech refresh has since changed since the scathing critique of Flash Boys and the Nasdaq SIP is in process of being refreshed to the same architecture as the trading systems, which should bring median latencies of the SIP down into the tens of microseconds and 99% latencies into the hundreds of microseconds. Current medians and 90th percentiles are at 450/810 microseconds, which is considered an eternity.
http://www.wallstreetandtech.com/infrastructure/nasdaq-omx-w...
http://www.utpplan.com/DOC/UTP%202015-Q4%20Stats%20with%20Pr...
On the technical side, one of the challenges with synchronizing packet delivery between a private & public feed is that the absolute fastest NICs available today with kernel bypass can get a packet from wire to memory or memory to wire in about 2-3u. So every hop of a computer costs a minimum of 4-6u. Usually this is Order Port Ingress -> Matching Engine -> Order Port Egress. An FPGA NIC that does the IP Header processing and delivers data to memory can do wire to memory in about 1-2u. One-way PCI-E latency is around 800 nanos. So there's about a 500-1000 nano budget to process a packet in the FPGA, then another 800 nanos to deliver it over PCI-E to memory to be processed. The fastest cut-through switches have a port-to-port latency of around 250 nanos, so switch hops are pretty negligible in the overall latency of a system, but they do play a part. Anyway, making sure that everything is super duper optimized and running as it should be in a semi-continuously deployed environment makes for interesting and quite crazy/bull-shit level challenges.
Congrats on being rewarded and recognized for the very important work you do in bringing transparency to the financial markets. The whole story of how this went down is just epic!
My question is about the business side of Nanex. Are sales of your products such as NxCore enough to sustain the company? Do you also actively trade? And what opportunities do you see currently around data science and machine learning for entrepreneurs in the quantitative finance world?
My question is about the business side of Nanex. Are sales of your products such as NxCore enough to sustain the company? Do you also actively trade? And what opportunities do you see currently around data science and machine learning for entrepreneurs in the quantitative finance world?
Is there any good reason that the SEC shouldn't ban direct feeds? If everyone were forced to use the SIP, what issues would remain here?
I never realised the NBBO could cross because of HFT. (Technically a market type order is required to prevent this, but the quote stuffing lags price) Definitely seems wrong.
For someone who knows next to nothing about trading, but has read "Flash Boys", any other recommended books on the topic?
Yes: read "Flash Boys: Not So Fast", a very readable and very technical takedown of "Flash Boys", which is a terrible, incoherent book --- without a doubt Lewis' worst.
For a better version of "Flash Boys", read "Dark Pools"; Dark Pools also takes a gimlet look at high-speed trading, but also does a great job of documenting the birth of the NASDAQ ECNs and the transition from the early-90s centralized markets to modern, decentralized, electronic trading.
If you want to get into the technical detail, the best book is still Larry Harris' _Trading and Exchanges_; it is almost exactly the "TCP/IP Illustrated" of trading.
For a better version of "Flash Boys", read "Dark Pools"; Dark Pools also takes a gimlet look at high-speed trading, but also does a great job of documenting the birth of the NASDAQ ECNs and the transition from the early-90s centralized markets to modern, decentralized, electronic trading.
If you want to get into the technical detail, the best book is still Larry Harris' _Trading and Exchanges_; it is almost exactly the "TCP/IP Illustrated" of trading.
TheNanex(2)
[deleted]
Google should open a stock exchange.
Sure, they'd be weary of the immense regulation involved. But I imagine they're much more qualified to build a fair, fast and accurate trading system than any of the existing exchanges are.
Sure, they'd be weary of the immense regulation involved. But I imagine they're much more qualified to build a fair, fast and accurate trading system than any of the existing exchanges are.
What would be more "fair" or "fast" or "accurate" about Google's exchange? I've gotten to do software pentests of several huge exchanges. They work pretty much the way you'd expect them to. A lot of things people are just starting to do in web software (message-mediated microservices) were idiomatic in finance in the 1990s.
Look no further than the linked article.
> a high quote rate from any NYSE stock will cause a corresponding delay in all NYSE public quotes! Anyone with this knowledge can easily cause latency on demand across many stocks, by simply blasting quotes in any one stock
Or another link from this thread.
> U.S. stock exchanges have invested huge sums of money creating two-tier markets - building and offering faster data and technology infrastructures at a price that only a small niche of traders can benefit from or afford, while at the same time continuing to offer slower products to everybody else.
https://www.iextrading.com/about/press/op-ed/
> a high quote rate from any NYSE stock will cause a corresponding delay in all NYSE public quotes! Anyone with this knowledge can easily cause latency on demand across many stocks, by simply blasting quotes in any one stock
Or another link from this thread.
> U.S. stock exchanges have invested huge sums of money creating two-tier markets - building and offering faster data and technology infrastructures at a price that only a small niche of traders can benefit from or afford, while at the same time continuing to offer slower products to everybody else.
https://www.iextrading.com/about/press/op-ed/
That was true on one exchange in 2012. It's no longer true today on NYSE, and probably wasn't true at any other venue (Hunsader is able to explain how he found this in a relatively small blog post; dollars to donuts he looked everywhere else for it too).
So: not delaying public quotes is table stakes. What could Google do better than that?
I'm not so much challenging you as I think it's an interesting question, and would like to see it hashed out in technical detail.
So: not delaying public quotes is table stakes. What could Google do better than that?
I'm not so much challenging you as I think it's an interesting question, and would like to see it hashed out in technical detail.
kpkp(3)
Now that shows you how far the whistleblower laws have gone ever since Enron. A big step up from what there was before.
First of all, this has nothing to do with High-Frequency Trading. It's about the NYSE not delivering a product (SIP real-time data) while collecting $100M a year for that service. This was happening for at least 3 years.
I am a champion of free markets. The term "High-Frequency Trading critic" is a label others use when they either can't understand and/or refute solid evidence.
I'm happy to answer questions.