Python For Finance: Algorithmic Trading(medium.com)
medium.com
Python For Finance: Algorithmic Trading
https://medium.com/@kacawi/python-for-finance-algorithmic-trading-60fdfb9bb20d
163 comments
An economist and a normal person are walking down the street together. The normal person says “Hey, look, there’s a $20 bill on the sidewalk!” The economist replies by saying “That’s impossible- if it were really a $20 bill, it would have been picked up by now.”
This quiq seems to support the original posters thesis, though, no? No one's going to make a living wandering the streets in search of $20 bills.
I think you might be extending this model beyond its boundaries of usefulness.
Well it's a theoretical economics concept. The $20 bill is an opportunity, nothing more. A pretty good deal of course, but if you are pedantic you could defend the standpoint that you did have to do something to get it: first, get lucky enough to be there, notice it, then bend over and maybe clean it. So it's not "free", just a pretty good deal.
The same thing applies to getting a job for instance. Take job X. If job X was available and worth doing at wage $, someone would be doing it. So why bother applying ?
So what it really means is that the semi-strong and strong form of the efficient market hypothesis is bullshit: there are plenty of opportunities in the market, you're just not seeing most of them.
I would argue that nearly everyone doesn't even try to see opportunities.
The same thing applies to getting a job for instance. Take job X. If job X was available and worth doing at wage $, someone would be doing it. So why bother applying ?
So what it really means is that the semi-strong and strong form of the efficient market hypothesis is bullshit: there are plenty of opportunities in the market, you're just not seeing most of them.
I would argue that nearly everyone doesn't even try to see opportunities.
This is why models are just for starting a conversation and not for predicting the future. Life has too many variables.
My favorite response to this:
"OK, how many times have you seen a $20 bill on the sidewalk???"
"OK, how many times have you seen a $20 bill on the sidewalk???"
I agree with most of what you're saying in principle, but it is very possible to identify alpha or acquire an edge that institutional market participants don't have. If you have access to data that most of the market does not, you can effectively trade on it. You can also effectively trade on a novel insight on a combination of data sources.
There is a lot of information asymmetry, and an individual is capable of capturing that without requiring a PhD or the resources of a large firm. That's not to say it's easy per se, but it's not hopeless. It requires special expertise or an unconventional approach.
Otherwise I agree that most people probably shouldn't attempt it (for risk tolerance reasons).
There is a lot of information asymmetry, and an individual is capable of capturing that without requiring a PhD or the resources of a large firm. That's not to say it's easy per se, but it's not hopeless. It requires special expertise or an unconventional approach.
Otherwise I agree that most people probably shouldn't attempt it (for risk tolerance reasons).
> If you have access to data that most of the market does not, you can effectively trade on it.
Yes, but you aren't going to have any such data. If the information is available to you, it's available to other participants of the market as well and you're not the only one trading on it.
Yes, but you aren't going to have any such data. If the information is available to you, it's available to other participants of the market as well and you're not the only one trading on it.
There is still a lot of opportunity out there. The thing is, as smart and loaded with resources the quants on wall street are, they have to work on opportunities that can provide returns with very large amounts of capital. Smaller opportunities that could bring in a few tens of thousands a month would not even make it on their radar. They need opportunities that can provide a return on hundreds of millions in capital.
I used to believe that as well. But it's fully possible to have data no one else does if you source it yourself (and I do). Furthermore, it's alright if a small number of other participants have the data as long as it's not yet priced in to market consensus.
If you can source it, so can others, what makes you think only you have thought about trading on that data which must necessarily be public for you to have it legally?
> If you can source it, so can others
Yes, agreed: the longer the data is available, the more likely it is that others catch and start using it. That's a constant battle of course - continually coming up with new sources of data and new ways of getting it before most of the market.
> what makes you think only you have thought about trading on that data which must necessarily be public for you to have it legally?
I don't think I'm the only one who has thought about trading on it; on the contrary, I know there are many parties interested in trading on material information that is only technically public, but very obscure. However, after consistently profiting on this data, I can be fairly well assured that there are relatively few parties with the same informational edge, because the market consensus clearly doesn't price it in until it becomes explicitly widely known.
As I said, I don't need to have certainty that I am literally the only party with the information, so long as it is known by so few parties that the market consensus doesn't price in the sentiment. All material data has a half-life, which decays with time and market awareness. In some cases, the nature of the data or the methodology for sourcing it is so novel that it is conceivable to be the sole party with the information for a short and meaningful time period (where information means the exact same dataset acquired in a categorically similar way, not the downstream sentiment derived from the dataset). But that is a much higher bar, and it's completely unnecessary for profit. It's nice for pride though.
You'll have to forgive me for not going into any greater detail than that, both because any more than this is strictly a trade secret, and because I'd rather not have a competitor for my personal trading operations (or have someone turn around and sell this data, ruining my alpha in the process). The best I can tell you is to look into the analysis of satellite imagery, but with greater human ingenuity and a more specialized approach. That's not going to get you all the way there, but at least that much of the methodology is publicly known thanks to the old article about Walmart parking lots. You can't collect actual information about a company, but there is a lot of room for human creativity in finding strong proxies for that data that the rest of the market is mostly ignoring.
Yes, agreed: the longer the data is available, the more likely it is that others catch and start using it. That's a constant battle of course - continually coming up with new sources of data and new ways of getting it before most of the market.
> what makes you think only you have thought about trading on that data which must necessarily be public for you to have it legally?
I don't think I'm the only one who has thought about trading on it; on the contrary, I know there are many parties interested in trading on material information that is only technically public, but very obscure. However, after consistently profiting on this data, I can be fairly well assured that there are relatively few parties with the same informational edge, because the market consensus clearly doesn't price it in until it becomes explicitly widely known.
As I said, I don't need to have certainty that I am literally the only party with the information, so long as it is known by so few parties that the market consensus doesn't price in the sentiment. All material data has a half-life, which decays with time and market awareness. In some cases, the nature of the data or the methodology for sourcing it is so novel that it is conceivable to be the sole party with the information for a short and meaningful time period (where information means the exact same dataset acquired in a categorically similar way, not the downstream sentiment derived from the dataset). But that is a much higher bar, and it's completely unnecessary for profit. It's nice for pride though.
You'll have to forgive me for not going into any greater detail than that, both because any more than this is strictly a trade secret, and because I'd rather not have a competitor for my personal trading operations (or have someone turn around and sell this data, ruining my alpha in the process). The best I can tell you is to look into the analysis of satellite imagery, but with greater human ingenuity and a more specialized approach. That's not going to get you all the way there, but at least that much of the methodology is publicly known thanks to the old article about Walmart parking lots. You can't collect actual information about a company, but there is a lot of room for human creativity in finding strong proxies for that data that the rest of the market is mostly ignoring.
No no, good explanation.
[deleted]
You are oversimplifying and overcomplicating at the same time. If it was an easy/obvious zero-sum game, we'd not have people going into finance at all.
MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".
MIT does not teach "HFT", those "MIT PHD Quants" are just as untrained in finance as you are. The rest is math, something that requires time to study but not necessarily a certificate. And creativity. I've only seen a single HFT-algo that made me say "wow, that's creative"; all others were just straight implementations of finance "wisdom".
1.) Bet what you can afford to lose
2.) The less you know the better
2.) The less you know the better
Why do you say it's a zero sum game? If I've learnt anything from my time in finance, it's that the market is definitely not zero sum. The prices you see represent sentiment, not a hard valuation, and someone isn't necessarily losing when you gain. If you really think about it, even entire economies aren't really zero sum as our method of valuation is intrinsically subjective!
If you aren't changing the sentiment, then it is a zero sum game based on the current sentiment.
Adjusted to average growth of stock market (and inflation), it is a zero sum game. You aren't creating value by predicting the future. Just like you don't create value when you predict which lottery ticket will win.
Adjusted to average growth of stock market (and inflation), it is a zero sum game. You aren't creating value by predicting the future. Just like you don't create value when you predict which lottery ticket will win.
As it turns out, there are at least 2 ways you can create value to society here:
1) Reduce spreads.
2) Stabilise prices.
Now, whether it makes any sense to put to work all those bright minds for these purposes, is a question the society should have asked some time ago...
1) Reduce spreads.
2) Stabilise prices.
Now, whether it makes any sense to put to work all those bright minds for these purposes, is a question the society should have asked some time ago...
I think that the OP meant that the speculation part (trying to buy low and sell high to beat the market) is a zero-sum game.
On the other hand, this zero-sum game helps companies raise capital to make real investments with real returns, and that part isn't zero-sum.
On the other hand, this zero-sum game helps companies raise capital to make real investments with real returns, and that part isn't zero-sum.
What I meant was HFT as I (and how I think the 'common person') understands it is that trading is basically a zero sum game since trading doesn't really help a company raise money over very short time periods, but investing is not a zero sum game where you're looking for stocks to go up over the long term.
Where I think HFT is zero sum is your competing against other HFT people to more accurately and quickly predict the future in the very near term, then buy & sell in a very short period. And since the market changes all the time, these algos must change to stay profitable over time as the competition improves their algos to beat you. Algo trading might work in markets where HFT isn't big yet, but most people won't know which markets that is or how to actually trade there, and if you get good enough to make successful models, you're basically in the industry, it's probably way more than a hobby at that point.
Where I think HFT is zero sum is your competing against other HFT people to more accurately and quickly predict the future in the very near term, then buy & sell in a very short period. And since the market changes all the time, these algos must change to stay profitable over time as the competition improves their algos to beat you. Algo trading might work in markets where HFT isn't big yet, but most people won't know which markets that is or how to actually trade there, and if you get good enough to make successful models, you're basically in the industry, it's probably way more than a hobby at that point.
You are painting with an extremely broad brush. Before you give up on this idea, just try this exercise: enumerate the markets and financial instruments that you could potentially trade in, and characterize the current trading environment in that market.
An ultra-HFT liquidity provider being profitable has little bearing on the potential profitability of quantitative trading on significantly larger timescales. There isn't sufficient volatility in most products for these actors to eat the lunch of actors who have alpha on trades that can hold for significant periods.
An ultra-HFT liquidity provider being profitable has little bearing on the potential profitability of quantitative trading on significantly larger timescales. There isn't sufficient volatility in most products for these actors to eat the lunch of actors who have alpha on trades that can hold for significant periods.
Even if it's a zero sum game (which it's not), most participants are not quant/algo based. As long as you can beat 50% percentile, you can make a profit.
Sure you may not be as profitable as top quant companies, but do you really mind that much?
Sure you may not be as profitable as top quant companies, but do you really mind that much?
Why on earth do you think the top quant companies will leave alpha on the table for you to snatch up?
That sounds too much like wishful thinking to me. Furthermore, if someone is really smart enough to beat the market consistently why on earth would they trade just on their own personal account? Work for a hedge fund and use other people's money to leverage your bets.
That sounds too much like wishful thinking to me. Furthermore, if someone is really smart enough to beat the market consistently why on earth would they trade just on their own personal account? Work for a hedge fund and use other people's money to leverage your bets.
Strategies have varying degrees of scalability. It's possible to find consistent returns on a personal scale that would be unfeasible to scale to $100M+ portfolios
> Furthermore, if someone is really smart enough to beat the market consistently why on earth would they trade just on their own personal account?
Alternatively, if you have a strategy that empirically works, and a strategy for identifying such strategies, why take on investors to share the risk? You can scale up your own investment and leverage yourself with less regulatory oversight.
Why on earth do you think the top quant companies will leave alpha on the table for you to snatch up?
It's not so much that they leave alpha on the table, rather that there is so much alpha available, and the capacity constraints and reward profiles are so different for many of them, that individuals can prosper outside of a firm. They just normally don't, because they lack the same training (and because it's very competitive).
Alternatively, if you have a strategy that empirically works, and a strategy for identifying such strategies, why take on investors to share the risk? You can scale up your own investment and leverage yourself with less regulatory oversight.
Why on earth do you think the top quant companies will leave alpha on the table for you to snatch up?
It's not so much that they leave alpha on the table, rather that there is so much alpha available, and the capacity constraints and reward profiles are so different for many of them, that individuals can prosper outside of a firm. They just normally don't, because they lack the same training (and because it's very competitive).
Some strategies only work with small investment sizes or require a lot of effort, so don't scale well.
Big firms leave tons of (for them) small opportunities on the table, because it's not worth their time.
Similarly, there are people who can regularly beat the market that don't have much to offer large funds because their methods don't scale.
Big firms leave tons of (for them) small opportunities on the table, because it's not worth their time.
Similarly, there are people who can regularly beat the market that don't have much to offer large funds because their methods don't scale.
Like Buffet who said it's easy to invest even you only have a couple million, and much harder when you have billions.
Trading is a zero sum game. You are making nothing. You win, someone else loses.
Trading is not a zero sum game. It doesn't matter if nothing's being made, that's not the point. It's an important service providing pricing and liquidity which facilitates capital allocation, diversification, and risk management. Trading is an absolutely essential part of the economy.
As a simple example, a farmer hedges his wheat crop selling wheat futures. That allows him to reduce some of his risk, thereby allowing him to plant more (i.e. growing the economy). The market marker who bought the wheat futures may be trading multiple commodities. This trade diversifies his risk, allowing him to trade more of other commodities, allowing other farmers to offset more of their risks (i.e. growing the economy). Investment companies or hedge funds enter the market purchasing futures, to diversify their risks, allowing them to invest more in equity markets, which fund companies (i.e. growing the economy).
If my point isn't already clear, markets and trading facilitate the diversification and allocation of risk capital to market participants thereby growing the economy. It's the conduit for capital which does make something.
As a simple example, a farmer hedges his wheat crop selling wheat futures. That allows him to reduce some of his risk, thereby allowing him to plant more (i.e. growing the economy). The market marker who bought the wheat futures may be trading multiple commodities. This trade diversifies his risk, allowing him to trade more of other commodities, allowing other farmers to offset more of their risks (i.e. growing the economy). Investment companies or hedge funds enter the market purchasing futures, to diversify their risks, allowing them to invest more in equity markets, which fund companies (i.e. growing the economy).
If my point isn't already clear, markets and trading facilitate the diversification and allocation of risk capital to market participants thereby growing the economy. It's the conduit for capital which does make something.
Sorry, but this is like saying those CalTech PHD in Computers Science are the best, so no point for me to go into that field...
This is like everything in life, not easy, but if you work on it it'll pay out in the end
>but if you work on it it'll pay out in the end
unless its one of the many things in your life that don't
unless its one of the many things in your life that don't
Not really, since trading is a winner take all game, whereas work in other fields does not have to be.
You can get an advantage if you focus on smaller markets or industries where it's not worth the time for the big quants to play in.
The is the only place the small guy can get alpha. Too small for the big guys to worry about and lots of fun to boot. The downside is the insider trading problem is really bad.
If the market is a zero sum game, then for every winning trade, there must be a losing trade. Obviously some teams must be winning consistently, which implies that some are losing consistently. This says that the consistent losers go out of business. So who are the winners going to trade with? By contradiction, it is not zero sum.
> Obviously some teams must be winning consistently, which implies that some are losing consistently.
That does not necessarily follow; markets are not static and players are free to enter and leave.
That does not necessarily follow; markets are not static and players are free to enter and leave.
People like to say the market is a zero sum game, but I have always been suspicious of this truism.
Its fair to say that participants can leave the market, but in practice that isn't what we see. In practice there are firms doing this trading, and they are staying in business, and obviously making money.
Are they fleecing the little guy then? This explanation falls flat for me, i.e. for the amount of money they seem to be making, it would take a lot of small time participants losing everything every day. Most people I know aren't even active traders.
So why is it an accepted truism that the market is zero sum?
Its fair to say that participants can leave the market, but in practice that isn't what we see. In practice there are firms doing this trading, and they are staying in business, and obviously making money.
Are they fleecing the little guy then? This explanation falls flat for me, i.e. for the amount of money they seem to be making, it would take a lot of small time participants losing everything every day. Most people I know aren't even active traders.
So why is it an accepted truism that the market is zero sum?
>Are they fleecing the little guy then?
Yes, more or less (although that's just one way they make money, and probably not the most lucrative) And fortunately for them there are plenty of "little guys" ready to enter the market on a regular basis.
Yes, more or less (although that's just one way they make money, and probably not the most lucrative) And fortunately for them there are plenty of "little guys" ready to enter the market on a regular basis.
My point exactly, not the most lucrative. So what remains is the implicit assertion they are fleecing the big time guys. And somehow the big time guys remain in business... so what's going on?
The missing puzzle piece here are dividends. Sure some people buy high and sell low and thus take a loss. But the market as whole can increase in value due to the external growth and profit.
Not quite. You need money to enter. Which was the parent poster's point - eventually, the losers will all be gone and there will be nobody left to trade with.
New suckers. Also a zero sum game doesn't mean one side always winning.
I'm curious to know who the loser was with the cryptocurrency rally this year?
You've forgotten you can introduce new units of whatever is being traded and quite often is & there are additional complications such as dividends, stock options, etc which makes your oversimplification lacking substance.
You've forgotten you can introduce new units of whatever is being traded and quite often is & there are additional complications such as dividends, stock options, etc which makes your oversimplification lacking substance.
This. But it does give python weenies an excuse to fool themselves into believing they are "programmers."
Personally, I struggle to see the competitive advantage Quantopian brings. They use retail brokerage platforms to facilitate trading, which rules out anything close to HFT. Then, they are tied to any financial data vendor (Morningstar in this case) to not offer too much visibility on the underlying data. As others have mentioned, this makes it tough to validate aspects like adjusted vs. as reported earnings, how delistings are handled, etc. From my experience, getting/making sure the data is accurate is a ton of work, even if it is from good sources and you can see all the actual data. The moment an investor/trader on the platform gets traction is the moment they want Compustat data, exchange data, Bloomberg for fixed income, and will trade through Instinet/Flextrade, etc. The moment the platform is successful is the moment Morningstar could pull the rug out from them. If someone has better understanding or knowledge on Quantopian in particular, I'd be interested to hear why.
My understanding is that Quantopian has no interest in being a platform for HFT. It's there to democratize trading strategies and test them in a sound way. They do a lot to have a good backtesting platform and clean the data that is available in the platform. If I wanted to automate a strategy, I'd have to figure out the Robinhood API and basically recreate what they have. Instead, I can use their platform, their backtests, their free data, and build strategies that efficiently lose money.
I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.
I think the other goal being that if you create a profitable strategy, you can enter it into competitions, trade with other people's money, and make a profit. I think the platform is a value add, I have an account and use the research notebooks. Some day when I more spare time I'll make some trading algorithms.
It sounds like day trading for script kiddies. There's no way to develop or execute a sound strategy with that level of access.
It's like a slot machine tournament. No one has an edge but some percentage of people will do well enough over a finite period to convince them to put real money behind their lucky algorithm.
It's like a slot machine tournament. No one has an edge but some percentage of people will do well enough over a finite period to convince them to put real money behind their lucky algorithm.
I understand they don't, but the execution options could matter. Retail brokerage platforms don't actually send your orders to the exchange: http://www.schwab.com/public/schwab/nn/legal_compliance/impo.... If your turnover is very low, and your trade size is small, this shouldn't make an impact in liquid products, but only having market and limit orders can be very limiting. Granted, IB is probably the best retail platform, but it's not perfect. Even a smart beta strategy would likely have 4X turnover to an index. I do hope they succeed, and perhaps they begin offering a more open, premium offering to compete with Capital IQ, etc. I just have a hard time seeing how operating as a fund-of-funds essentially is a good business model considering the CAPEX they have to build the software. I could see it as a way to build, test, and then offer a premium back-testing platform or portfolio analysis/attribution tool once they've ironed out the bugs. There's a lot more money in that business than in fund-of-funds.
The main problem with Quantopian is that the data missing especially for delisted stocks, here are some examples you can test yourself: AA, CWH, NIHD, PANL, HTZ, DSL - I have a whole list I found very fast, just imagine what other mistakes could be there... And the other problem is that they are my competitor, so I would never give them my trading strategy or ideas I am testing at the moment...
The trading strategy stuff has been brought up before. They say when evaluating algorithms they only look at the alpha, beta, etc. and not the algorithm itself. While the system can see what you're running, I think they work pretty hard to let your code be yours. I'm definitely not very advanced in algorithmic trading, so I see how there could be some data issues that I wouldn't come across.
This is false. Quantopian sources their historical and real time data from Nanex, and the data is explicitly free of survivorship bias (i.e. they maintain data for delisted equities).
I don't know what you verified, but it directly contradicts my own experience and the FAQ: https://www.quantopian.com/faq#data-sources
To your second point, as another commenter said Quantopian doesn't see your algorithm unless you give them permission, as outlined in the terms of service (which are very readable, by the way).
NB: I am not unaffiliated with Quantopian and don't use the platform for my own trading, but I have tried it out before.
I don't know what you verified, but it directly contradicts my own experience and the FAQ: https://www.quantopian.com/faq#data-sources
To your second point, as another commenter said Quantopian doesn't see your algorithm unless you give them permission, as outlined in the terms of service (which are very readable, by the way).
NB: I am not unaffiliated with Quantopian and don't use the platform for my own trading, but I have tried it out before.
Please go back and check and you'll se the data is missing for the symbols stated above, and I am sur for many more based on my limited test... everyone can see I am right by trying to request the data, nothing false about that...
> everyone can see I am right by trying to request the data, nothing false about that...
Oh, they can? Well that's funny, because I just went onto Quantopian and checked for myself.
http://imgur.com/a/VwMUJ
http://imgur.com/a/VV68C
Gee, would you look at that...they are there.
This took me all of five minutes. Do you have anything else I can easily disprove while I'm at it?
Oh, they can? Well that's funny, because I just went onto Quantopian and checked for myself.
http://imgur.com/a/VwMUJ
http://imgur.com/a/VV68C
Gee, would you look at that...they are there.
This took me all of five minutes. Do you have anything else I can easily disprove while I'm at it?
I like your script naming scheme...
The problem is the data is missing for specific dates... I just can imagine what else is missing or it is wrong...
This is really trying my patience. You made a clearly incorrect claim without mentioning specific dates and challenged me to disprove it. When I did disprove the claim, you shifted the goalposts with this backpedaling remark about how only specific dates are missing. You continue to make these claims and put the burden of proof on everyone else because you can't be bothered to defend them when you're challenged.
I'm not searching for a needle in a haystack for you. Make a falsifiable claim, and make the entire claim, with the specific dates you're talking about, or stop this astroturfing against Quantopian and QuantConnect that you're doing.
I'm not searching for a needle in a haystack for you. Make a falsifiable claim, and make the entire claim, with the specific dates you're talking about, or stop this astroturfing against Quantopian and QuantConnect that you're doing.
My goal is to show people how start to think for themselves and hopefully create lifestyle business around trading or invest for themselves instead of using "buy and hope strategy" and how to do it without losing a single penny while learning and to show them the barrier for entry is pretty low...
But you are right, why bother when every time I gave an idea, the day traders with the thousands of dollars data feeds are all over my posts... no good deed goes unpunished here at HN I guess :-(
Shitting on other people products with vague and inaccurate criticisms isn't a good deed, it's just bad form. People don't need you to "show them how to think for themselves." The other guy is right, either make a sound and specific criticism or don't say anything. You did exactly what he said, you made a bad argument which he refuted, then you moved the goal post so as not to appear wrong, while still leaving out the necessary detail to disprove your new vague claim, rather than admitting your mistake and making a better argument that included specifics about what is missing.
What dates do you see missing data for?
They're searching for "alpha" strategies. Then they buy/share profit with the creator... I have a hunch that they are feeding these good strategies to some type of ML in order to become more efficient and profitable.
If you want to build a real algorithmic trading strategy.. please pick up a real textbook such as Qian's Quantitative Equity Portfolio Management.
You should understand the following concepts at a minimum:
- Markowitz portfolio optimization (mean-variance analysis)
- Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors, etc)
- Alpha decay
- Time series analysis (autocorrelation, GARCH, ARMA processes)
- Basic price-based signals (momentum, volatility, value, etc) - etc, etc.
You should understand the following concepts at a minimum:
- Markowitz portfolio optimization (mean-variance analysis)
- Beta-neutral portfolios (i.e. using MSCI BARRA, sector ETFs or PCA factors, etc)
- Alpha decay
- Time series analysis (autocorrelation, GARCH, ARMA processes)
- Basic price-based signals (momentum, volatility, value, etc) - etc, etc.
Any others you (or anyone else) recommend?
Evidence-based Technical Analysis by David Aronson is a great intro.
thanks for the recommendation
So, I got semi-seriously interested in this around the end of last year.
My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios.
I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations?
How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat?
Me:
1) Build some simple forecasting NNs.
2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure.
3) Get impatient -- get some "bright" ideas, do some manual trading.
4) Make some money on the first day, lose double that on the next two days.
5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading.
6) Back to coding infrastructure. Get bored ... do some manual trades.
7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies.
8) Enjoy my 25% returns.
My takeaway is: it's not about implementing a couple of trading strategies. It's about implementing a pipeline that rapidly allows you to test what-if scenarios.
I might have like 10 ideas a day for strategies. How many of those can I rigorously validate per week? What about variations? I.e. tuning various hyper-parameters? Combinations?
How quickly can I recombine data from various sources into exactly the layout I need to train and test this or that strat?
Me:
1) Build some simple forecasting NNs.
2) Realize I need to be able to generate and test ideas waaay faster. Start working on infrastructure.
3) Get impatient -- get some "bright" ideas, do some manual trading.
4) Make some money on the first day, lose double that on the next two days.
5) Decide to ban myself from manual trading as obviously I'm an addict. Resolve to only do algorithmic trading.
6) Back to coding infrastructure. Get bored ... do some manual trades.
7) After losing around $20k, stop the trading madness, and go back to buying and holding great tech companies.
8) Enjoy my 25% returns.
> 5) Decide to ban myself from manual trading as obviously I'm an addict.
> 7) After losing around $20k
> 7) After losing around $20k
QuantConnect recently announced full python library support; and we have launched https://www.quantconnect.com/tutorials to help people write quantiative strategies in Python.
QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started.
The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.
QuantConnect & LEAN gives you ability to do tick->daily resolutions; for equity, morning-star, future, option, forex and cfd trading - all with a fully open source project which includes samples of data to get you started.
The grunt work is still done in C# so its faster than other full python based backtesting engines. Edit: I'm the Founder @ QC.
Hey Jared, two questions:
1. Any chance of a Robinhood integration à la Quantopian?
2. How are the architectural revisions[0] coming along?
Also some totally unsolicited feedback:
If I'm being completely honest, I found it difficult to get going with QC. The documentation is decent, but there's not enough to avoid having to review LEAN source right off the bat. The examples also tend to mix helper classes with lower-level functionality, and that can create confusion.
The framework itself feels a bit over-reliant on OOP. Some aspects feel too tightly coupled, others too little. Obviously LEAN has been around for many years now, so architectural baggage is perfectly understandable.
A total rewrite I'm sure isn't feasible, though I'd suggest the following ethos in any case:
a) Design primitive user-accessible data structures with virtually no inbuilt functionality.
b) Build low-level components that operate using those data structures.
c) Build high-level components that compose low-level components.
d) Allow users to author their own components, and to compose components of any type however they see fit.
Pretty sure you're already on that track in a sense, so it's good to see things headed in the right direction. What keeps me from writing a custom framework is the data, the fact QC does a ton of grunt work, and that it's well-tested.
tl;dr Please break apart the monolithic QCAlgorithm class as much as you can! :)
[0] https://www.quantconnect.com/forum/discussion/1816/qc-algori...
1. Any chance of a Robinhood integration à la Quantopian?
2. How are the architectural revisions[0] coming along?
Also some totally unsolicited feedback:
If I'm being completely honest, I found it difficult to get going with QC. The documentation is decent, but there's not enough to avoid having to review LEAN source right off the bat. The examples also tend to mix helper classes with lower-level functionality, and that can create confusion.
The framework itself feels a bit over-reliant on OOP. Some aspects feel too tightly coupled, others too little. Obviously LEAN has been around for many years now, so architectural baggage is perfectly understandable.
A total rewrite I'm sure isn't feasible, though I'd suggest the following ethos in any case:
a) Design primitive user-accessible data structures with virtually no inbuilt functionality.
b) Build low-level components that operate using those data structures.
c) Build high-level components that compose low-level components.
d) Allow users to author their own components, and to compose components of any type however they see fit.
Pretty sure you're already on that track in a sense, so it's good to see things headed in the right direction. What keeps me from writing a custom framework is the data, the fact QC does a ton of grunt work, and that it's well-tested.
tl;dr Please break apart the monolithic QCAlgorithm class as much as you can! :)
[0] https://www.quantconnect.com/forum/discussion/1816/qc-algori...
When promoting your company, please say so, also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...
Sorry SirL! I edited it within 10 sec to be explicit but you must've refreshed before I'd updated it =)
We provide FX/CFD data for free download; the other data is restricted by the exchanges sadly so we can't make it available. Instead we put the tools we used to make it into LEAN format into LEAN (/Toolbox) so you can purchase it and convert it yourself.
We provide FX/CFD data for free download; the other data is restricted by the exchanges sadly so we can't make it available. Instead we put the tools we used to make it into LEAN format into LEAN (/Toolbox) so you can purchase it and convert it yourself.
Thanks jaredbroad, just out of curiosity, what are the average yearly returns on your top 3 users, how much are you investing with them and how do you split the profits?
>When promoting your company, please say so ...
His bio and pronoun usage said as much.
>also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...
Can you not load up the data and test it however you wish? They even say where they get their data from.
His bio and pronoun usage said as much.
>also on QuantConnect one cannot actually see the data, so there is no way to verify how good the data is...
Can you not load up the data and test it however you wish? They even say where they get their data from.
No you cannot
First of all, you're criticizing QuantConnect's data when you claim to get your data from ebay.
Second, yes you absolutely can verify QuantConnect's data. You can use it as much as you want within the context of their platform, you just can't download the data en masse from their platform and use it on your own. But if you have tick data (equities) or minute data (options) yourself, you can certainly verify it (which of course, you don't, because you think ebay is a good source for financial data).
I am going to continue griefing you in threads like this where you spread blatant misinformation, because at this point I'm convinced you have an ulterior motive or are in fact selling this data you keep talking about on ebay.
Second, yes you absolutely can verify QuantConnect's data. You can use it as much as you want within the context of their platform, you just can't download the data en masse from their platform and use it on your own. But if you have tick data (equities) or minute data (options) yourself, you can certainly verify it (which of course, you don't, because you think ebay is a good source for financial data).
I am going to continue griefing you in threads like this where you spread blatant misinformation, because at this point I'm convinced you have an ulterior motive or are in fact selling this data you keep talking about on ebay.
Unfortunately you cannot display QC tic data to be able to very it against you broker for example. As I said before if you have a good and cheap source, please share it with everyone...
> good and cheap source
Pick one. You don't get both. I'm happy to share where you can get real data from.
https://www.tickdata.com
https://datashop.cboe.com
http://www.nanex.net/nxcore.html
https://quantquote.com
No affiliation with any of these.
Pick one. You don't get both. I'm happy to share where you can get real data from.
https://www.tickdata.com
https://datashop.cboe.com
http://www.nanex.net/nxcore.html
https://quantquote.com
No affiliation with any of these.
On a side note, the market continues to do well and I've been noticing this trend of active-trading, real-estate investing gurus crawl out of the wood work selling services.
Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.
Please do not try to trade actively unless that's your full-time job. Passive investing using index funds is definitely not sexy, but it gets the job done.
Don't worry, we have cryptocurrency now. A new, unregulated frontier.
> Passive investing using index funds is definitely not sexy, but it gets the job done.
Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.
Only if you have a significant amount of capital to play with. People looking into active trading are doing it because they don't have the necessary capital to make passive investing meaningful; they're looking for much much larger returns that you can get from an index fund, which necessarily comes with more risk. Traders are gamblers.
This is extremely reckless investment advice.
Active trading, by definition, incurs higher transaction costs than passive investing. Since transaction costs are generally priced per trade, active trading is more costly on a percentage basis for individuals with less capital.
Active trading, by definition, incurs higher transaction costs than passive investing. Since transaction costs are generally priced per trade, active trading is more costly on a percentage basis for individuals with less capital.
> This is extremely reckless investment advice.
It wasn't investment advice, nor did I say it was a good idea; I'm merely telling you why people choose active investing: they believe they can do better and there's enough stories out there of the "lucky" person who did better to keep that hope alive. In all likelihood they're going to lose their money, but you'll never convince people trying to win a lottery that they're better off accepting they can't. I literally said traders are gamblers, obviously that means I think active trading is gambling, that's not investment advice.
It wasn't investment advice, nor did I say it was a good idea; I'm merely telling you why people choose active investing: they believe they can do better and there's enough stories out there of the "lucky" person who did better to keep that hope alive. In all likelihood they're going to lose their money, but you'll never convince people trying to win a lottery that they're better off accepting they can't. I literally said traders are gamblers, obviously that means I think active trading is gambling, that's not investment advice.
I went to the Quantopian conference for their basic training on algorithmic trading. This blog post was pretty much what they covered, intro to pandas and a simple strategy. There is a lot of educational material on their site too (which is what you ended up getting in the paid training).
My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python.
1. https://lectures.quantecon.org/py/
My biggest thing with the Python for Finance books - I know Python, I want to learn finance. All these books are the inverse of that, for people who know finance and want to learn Python. There is a good site for quantitative economics [1] that has tutorials in Python and Julia. I would love a mathematics of finance book that had the examples in Python.
1. https://lectures.quantecon.org/py/
those are indeed good lectures but they are not math for finance per se - more like dynamic programming for econ/finance. my impression is that finance as such is a hodge-podge of econ/accounting/business.
The problem with learning finance that it takes a lot og effort. A good place to start would be taking the 3 CFA exams. After about 1000 hours of study, you will have a basic grasp of finance.
This is utterly the wrong way to learn finance for quantitative trading. That would be an unproductive use of time.
Read Options, Futures and Other Derivatives and Algorithmic Trading and DMA and you basically know everything you'll get from public sources that could be meaningful for trading.
If he knows how to code and is looking for "mathematics of finance" he should start with those, not the CFA.
Read Options, Futures and Other Derivatives and Algorithmic Trading and DMA and you basically know everything you'll get from public sources that could be meaningful for trading.
If he knows how to code and is looking for "mathematics of finance" he should start with those, not the CFA.
That is your opinion, but if you look at the resumes of scientific active equity fund managers and quants, they all have it. Just understanding the quantitative side leads to vast underperformance over the course of market cycles...
Quants don't usually have CFAs. A phD in physics is more common to see. CFAs help you working in a real financial environment where you need to be aware of a lot of stuff. If you want to day trade at home, it's close to useless. Please nobody gets a CFA if what you want to do is learning day trading.
If you are just gambling with math at home, dont bother. If you want to understand finance like the op, better start learning. They have phds and cfas...
If you want to understand finance 360, from compliance to settlements, from middle office to government bonds structuring, sure, get a CFA. But somebody learning to trade not only doesn't need one, but very little of what's in it is useful. I've been in finance for ~10 years, trust me, very few traders and quants have CFAs, that's more product management, research analysts, compliance, controllers, risk managers, even IT. We are not talking about learning those jobs.
The CFA doesn't cover settlements at all, nor does it have anything to do with gov bond structuring, and very, very little on compliance. (Basically the bare minimum you need to know on ethics and asset manager code of conduct/GIPS).
The vast majority of the CFA curriculum covers investing fundamentals, including accounting (100% necessary for building models), quant (obviously necessary), econ (pretty important).
The vast majority of the CFA curriculum covers investing fundamentals, including accounting (100% necessary for building models), quant (obviously necessary), econ (pretty important).
What resumes? Can you show me these vast numbers of resumes with CFAs? Are you talking about fund managers or quants? If your goal is develop quantitative trading strategies, you shouldn't be getting a CFA because it's superfluous.
http://news.efinancialcareers.com/us-en/150266/why-the-cfa-w...
On another note, even if what you were saying were true (and it isn't), it wouldn't demonstrate that it's useful to have the CFA. Certifications are usually helpful for HR or for regulation, not for actual knowledge. And everything you need to know about finance can be learned in far less than 1000 hours for trading if you're in a research or development role.
http://news.efinancialcareers.com/us-en/150266/why-the-cfa-w...
On another note, even if what you were saying were true (and it isn't), it wouldn't demonstrate that it's useful to have the CFA. Certifications are usually helpful for HR or for regulation, not for actual knowledge. And everything you need to know about finance can be learned in far less than 1000 hours for trading if you're in a research or development role.
Linkedin. Go there, search for scientific active equity (that is what a professional quant manager is called).
Sort by who manages real money and who doesnt. Anyone managing $1bn or more all has it. That is not very much money to manage.
Sort by who manages real money and who doesnt. Anyone managing $1bn or more all has it. That is not very much money to manage.
I feel like we're talking past each other. I'm talking about a role where you develop strategies as either a researcher or a trader. Higher up the chain, sure, I can admit there's a utility to the CFA.
I think the main issue is that you need a solid grasp of the qualitative side of finance to build working quant models. Pms are responsible for quant model construction and they all get cfas because that is base level finance understanding.
Did you find the conference worth it ? I was thinking of going to it as well or even going to the online version since it's a lot cheaper but ended up not taking part.
I went to the conference 1 or 2 years ago and it was good. Worth the student pricing. I only did the training this year, also on student pricing, and it was pretty disappointing.
Here's a dataset on Kaggle that can be used in this process https://www.kaggle.com/biomimic/periodic-table-of-elements-m...
This is nothing more than gambling. Let's say you have 20k to play with. You would be far better off in the long run to put 15k in Wealthfront and use the other 5k as a bankroll of 25 buyins for 1/2 no limit holdem to learn the game.
The article mentions a few of the pitfalls of backtesting, but it does not mention one of the best tools at your disposal in backtesting: Walk Forward Optimization/Analysis
https://en.wikipedia.org/wiki/Walk_forward_optimization
See also: "The Evaluation and Optimization of Trading Strategies" by Robert Pardo
https://en.wikipedia.org/wiki/Walk_forward_optimization
See also: "The Evaluation and Optimization of Trading Strategies" by Robert Pardo
I'm wondering... why not scala?
scala gives me type-checking tools (eg dependent types) so I don't shoot myself on foot
Scala is absolutely used in finance, especially for interoperability with Java in Java-heavy firms. But it's data analysis libraries aren't as mature as the pandas/scipy stack. You'd use them for different parts of a trading pipeline.
I use R for almost all quantitative tasks but not as a trading platform. It's great for research and testing though.
I have a simple manual algorithm that took me from 650€ to 1450€ now (as test) in 2 months. Does someone here have experience with algo trading in c#?
I won't say I have "experience" but quantconnect.com is as good a platform as I've found for C#. Their engine is open source so you don't have to rely on them either, though you'll need to provide your own data if you run it yourself.
QuantConnect also offers tick-level data, though not full resolution if memory serves. Still way ahead of Quantopian in any case.
Futures have millisecond timestamps (trades/quotes), equity trade ticks are rounded to the nearest second, cfd/forex are millisecond quote bars. For options we have minute resolution data =)
Worth noting though that Options data is super hard to get running on QuantConnect due to the memory limits and lack of index options.
Agree, we're working on making it easier now. We patched a bug today which should make it more efficient.
Do you have a public email for getting in touch with you? I'm interested in chatting.
Ninjatrader
How is algorithmic trading not equivalent to astrology?
Nothing can be predicted because there's way too many confounding factors.
Nothing can be predicted because there's way too many confounding factors.
I have a mean reversion strategy based on the comparing the results of several types of sentiment analysis in real time. It earns between 3 - 15% in options trading every few days with a win rate of around 70% and an average holding time of a few hours. Stop losses are at -70% or so.
Algorithmic trading is very difficult, but it is empirical. Information asymmetry exists in the market, and if you can capture it you profit. I don't know how capacity constrained my strategy is (I think it would be difficult to work this strategy profitably with over $10M depending on how much effort you put into execution; my execution is unsophisticated and I'm working with two orders of magnitude less than that), but it's working.
Algorithmic trading is very difficult, but it is empirical. Information asymmetry exists in the market, and if you can capture it you profit. I don't know how capacity constrained my strategy is (I think it would be difficult to work this strategy profitably with over $10M depending on how much effort you put into execution; my execution is unsophisticated and I'm working with two orders of magnitude less than that), but it's working.
> Stop losses are at -70% or so.
Am I reading that right, you lose one trade you lose 70% of your capital? Sounds like a very very dangerous strategy with a near 100% certainty of blowing up the account. The optimal trade size for equity growth (which is itself hugely risky) is vastly smaller than that.
Am I reading that right, you lose one trade you lose 70% of your capital? Sounds like a very very dangerous strategy with a near 100% certainty of blowing up the account. The optimal trade size for equity growth (which is itself hugely risky) is vastly smaller than that.
Hah, no that's a typo, sorry my mind was a bit scattered in this thread. I meant to type that stop losses are at -15% or so (of the position, not the portfolio).
There is a separate risk management metric in play that automatically closes positions and pauses the strategy after a specific number of losses in a row, or after the strategy's capital allocation drops below a certain percentage (whichever comes first).
The other metrics are as stated.
There is a separate risk management metric in play that automatically closes positions and pauses the strategy after a specific number of losses in a row, or after the strategy's capital allocation drops below a certain percentage (whichever comes first).
The other metrics are as stated.
> -15% or so (of the position, not the portfolio).
Oh... HUGE difference. What percentage of the portfolio is the position? How long have you run this strategy; I ask because those are huge returns that indicate to me you're taking huge risk which can work short term but nearly always wipes you out over a sufficient amount of time.
> It earns between 3 - 15% in options trading every few days with a win rate of around 70% and an average holding time of a few hours.
If those returns could be held for any length of time you'd be wealthy in no time.
Oh... HUGE difference. What percentage of the portfolio is the position? How long have you run this strategy; I ask because those are huge returns that indicate to me you're taking huge risk which can work short term but nearly always wipes you out over a sufficient amount of time.
> It earns between 3 - 15% in options trading every few days with a win rate of around 70% and an average holding time of a few hours.
If those returns could be held for any length of time you'd be wealthy in no time.
> Oh... HUGE difference. What percentage of the portfolio is the position? How long have you run this strategy; I ask because those are huge returns that indicate to me you're taking huge risk which can work short term but nearly always wipes you out over a sufficient amount of time.
No single position can be more than 10% of the portfolio, there is no limit on concurrent positions, and the strategy cannot use more than 20% of the account's capital. Each position has a profit target, normally a 5% increase on the cost basis, and when it's opened a GTC order is immediately submitted to flip it. But this isn't a market-making algorithm or something that requires serious latency considerations - the target holding period for each position is less than five days, after which if the position is still open it's flagged. The strategy has been running continuously for only eight months; prior to that it was backtested with historical data before launching live. It used to be that the mean holding time was around 2 days, but lately it's been flipping positions in fairly short intraday scales (an hour or less), which means I'll be adding more risk accountability to it.
> If those returns could be held for any length of time you'd be wealthy in no time.
Yes...however this strategy has generated less than $30,000 in profits since its launch (on an initial outlay of $10,000), and I don't reinvest profits.
This is an options trading algorithm, and I don't think there is enough liquidity available in the targets to significantly ramp up capital from here without adding on more leverage (and risk). The other difficulty is that a confluence of factors needs to happen somewhat simultaneously in order for a candidate equity to become a target.
Due to this, it's more accurate to say that the strategy generates a few hundred dollars weekly to biweekly (on average). I may be overcautious, but this is primarily a research project for me. I would not actually be trading with real capital were it not for the fact that I'd like to demonstrate the results in the future.
No single position can be more than 10% of the portfolio, there is no limit on concurrent positions, and the strategy cannot use more than 20% of the account's capital. Each position has a profit target, normally a 5% increase on the cost basis, and when it's opened a GTC order is immediately submitted to flip it. But this isn't a market-making algorithm or something that requires serious latency considerations - the target holding period for each position is less than five days, after which if the position is still open it's flagged. The strategy has been running continuously for only eight months; prior to that it was backtested with historical data before launching live. It used to be that the mean holding time was around 2 days, but lately it's been flipping positions in fairly short intraday scales (an hour or less), which means I'll be adding more risk accountability to it.
> If those returns could be held for any length of time you'd be wealthy in no time.
Yes...however this strategy has generated less than $30,000 in profits since its launch (on an initial outlay of $10,000), and I don't reinvest profits.
This is an options trading algorithm, and I don't think there is enough liquidity available in the targets to significantly ramp up capital from here without adding on more leverage (and risk). The other difficulty is that a confluence of factors needs to happen somewhat simultaneously in order for a candidate equity to become a target.
Due to this, it's more accurate to say that the strategy generates a few hundred dollars weekly to biweekly (on average). I may be overcautious, but this is primarily a research project for me. I would not actually be trading with real capital were it not for the fact that I'd like to demonstrate the results in the future.
What platform are you using? Do you pay for each trade?
I use Interactive Brokers (the Python TWS API), with a trading system I've written in mostly Python and C++. Yes, I'm charged per trade (occasionally there are rebates, but the strategy doesn't optimize for adding liquidity). I don't have the trading volume to negotiate flat commission rates (nor is that really high on my list of priorities).
Algo trading typically utilises technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons, or fundamental analysis (e.g. algorithmically valuing and pricing options based on underlying fundamental data, sentiment analysis, etc.), and buying/shorting as appropriate. It is based on scientific methods: empirical evidence being used to validate hypothesis that produce results that can be shared and repeated by others.
Astrology is story-telling with no scientific grounding.
Astrology is story-telling with no scientific grounding.
> technical analysis which is basically patterns proven to repeat in markets for a variety of fundamental reasons
Technical analysis is merely another name for hindsight bias. Those patterns only look like they repeat in hindsight because you're ignoring all the failed patterns that setup right but failed to play out and thus don't look like the pattern in hindsight.
Technical analysis is exactly like astrology and is in no way grounded in science. Quantitative analysis is based in the scientific process, technical analysis is superstition created by visual traders who think they see patterns but fail to understand biases. You are confusing the two, technical analysis is a pattern library of nonsense like "head and shoulders" or "dead cat bounce" based purely on visual patterns in price data that only "appear" to repeat due to hindsight bias. It is in no way remotely scientific.
I've spend thousands of hours testing trading patterns in Forex, they're all nonsense that lose as much as they win over any decent sample size which equates to random.
Edit: to the guy below, You should ask for sources that back up the claims that it does work. You won't find any, it's trader superstition, not science. You don't have to prove things don't work; you have to prove they do. At least if you want to call it based in science. However, the field of quantitative analysis itself is proof technical analysis doesn't work; it's what happened with people who actually knew math got into trading and discovered the technical analysis superstition that visual traders who lack math skills made up.
Technical analysis is merely another name for hindsight bias. Those patterns only look like they repeat in hindsight because you're ignoring all the failed patterns that setup right but failed to play out and thus don't look like the pattern in hindsight.
Technical analysis is exactly like astrology and is in no way grounded in science. Quantitative analysis is based in the scientific process, technical analysis is superstition created by visual traders who think they see patterns but fail to understand biases. You are confusing the two, technical analysis is a pattern library of nonsense like "head and shoulders" or "dead cat bounce" based purely on visual patterns in price data that only "appear" to repeat due to hindsight bias. It is in no way remotely scientific.
I've spend thousands of hours testing trading patterns in Forex, they're all nonsense that lose as much as they win over any decent sample size which equates to random.
Edit: to the guy below, You should ask for sources that back up the claims that it does work. You won't find any, it's trader superstition, not science. You don't have to prove things don't work; you have to prove they do. At least if you want to call it based in science. However, the field of quantitative analysis itself is proof technical analysis doesn't work; it's what happened with people who actually knew math got into trading and discovered the technical analysis superstition that visual traders who lack math skills made up.
Can you provide sources that back up your claim with studies of inefficiency of technical analysis?
I'm not busting your balls, I actually agree with you but I haven't seen it really proven.
I'm not busting your balls, I actually agree with you but I haven't seen it really proven.
> Can you provide sources that back up your claim with studies of inefficiency of technical analysis?
Not the original guy but I just want to point out that Warren Buffett, Peter Lynch do not believe in technical analysis.
Not the original guy but I just want to point out that Warren Buffett, Peter Lynch do not believe in technical analysis.
Well they are obviously fundamental type of investors, and I have no doubts this type of investment is superior to pattern matching and chart gazing. But I see all those guys doing their pretty technical analysis charts in stocks and bitcoins and even though I think it's all BS, I wonder, is it? I mean, can you get a small edge? why would they be at it all day if it wasn't making them money?
> I mean, can you get a small edge? why would they be at it all day if it wasn't making them money?
It's gambling, they win randomly and falsely attribute the wins to tech analysis and dismiss all the losses as them "reading" it wrong. The random wins attribute to an addiction and superstition but they'd win just as much flipping a coin to decide whether to buy or sell. These people are very bad at math, if they were any good they'd be doing quantitative analysis and looking for actual patterns in the data that stand up to real scientific inquiry. Technical analysis is to quantitative analysis what alternative medicine is to actual medicine.
It's gambling, they win randomly and falsely attribute the wins to tech analysis and dismiss all the losses as them "reading" it wrong. The random wins attribute to an addiction and superstition but they'd win just as much flipping a coin to decide whether to buy or sell. These people are very bad at math, if they were any good they'd be doing quantitative analysis and looking for actual patterns in the data that stand up to real scientific inquiry. Technical analysis is to quantitative analysis what alternative medicine is to actual medicine.
If this was the case...if there was way to make money off of scientifically proven, reliably reproducible 'stock market patterns' everyone would be a billionaire.
So how do you explain this:
When filing for its IPO in March 2014, it was disclosed that during five years Virtu Financial made profit 1,277 out of 1,278 days, losing money just one day.
When filing for its IPO in March 2014, it was disclosed that during five years Virtu Financial made profit 1,277 out of 1,278 days, losing money just one day.
They don't rely on technical analysis. They are arbitrage trading across different markets, where there's zero chance of losing money if you're fast enough and doing it right. That's why HFT is so popular, if you're doing it right you can't lose money.
And that one day was due to a programming error, not a trading error
The net gain in stock trading is usually positive in the long run, even if you just buy and sell randomly. Just look at the DJIA.
It is a great article, but why on earth someone will use a service like Quantopian or similar service?
They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy?
Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc...
The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...
They are your competitor and who will prevent a disgruntle employee or a hacker to steel your successful trading strategy?
Just buy some data from eBay, you can get 20 years of historical stock market data for less than $100 and you can test any trading strategy or idea imaginable, including trend following, buy and hold ETFs, etc...
The barrier of entry is pretty low and you can develop a great lifestyle business with no customers, employees and investors around that...
Because they do so much work upfront for you. I worked in algorithmic trading for a bit over 10 years but left in 2014. I spent a few months in my spare time building out a trading and backtesting platform, pulling data off of yahoo finance, connecting to Interactive Brokers via their C++ API, etc. It was all very slow and tedious, and during my googling one day, I come across quantopian. I felt like an idiot- they already had free access to morningstar with a decent API around it, tons of other datasets you could subscribe to, and a fully fleshed out backtesting platform with risk measures, pretty graphs, etc.
It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's.
Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets.
One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well.
If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.
It would have taken me months-years to get to that point. Shortly after they also included integrations to IB and Robinhood so you could very easily take your ideas and actually live trade them. I immediately stopped development on my trading platform and just started using Q's.
Q could potentially take my algorithm and start using it themselves, though they actually put money into algorithms they think are promising and share the profits with you. That's a small risk IMHO. Hackers and the like... I did this stuff for 10+ years. People WAY over estimate the value of this code- you have an idea, you backtest, hope it works, and then you put it in the wild. I have never seen an algo that is a real guaranteed money maker (aside from arbitrage/hft stuff). What are these hackers going to do when they take a 10% loss one month and don't understand why? Some "signals" can be used in-line fairly generically with other algorithms to enhance their performance, but I will still bet on the guy that knows/understands the algorithms to not lose his shirt, then some hacker dude that likely has no capital to begin with and may not even understand the basics of financial markets.
One thing you seem to overlook is that it is one thing to go buy a bunch of historical data off ebay, but you need to constantly update that if you are going to live trade- Q handles that part as well.
If I was looking to make a business out of this, I would build my own platform. But using it as a research platform to invest my own money, Quantopian is awesome.
My experience is unfortunately different, I found the data for some delisted stocks like HTZ missing which will massively screw the past results due to the survivorship bias...
Can you please prove this or stop saying it? Quantopian's data is from Nanex and is free of survivorship bias, as I mentioned to you in another comment.
Mostly a matter of trust. I trust Quantopian more than I trust some random asshole on eBay purporting to sell accurate historical data. Plus, Quantopian is convenient.
This is the cheapest data I can find and you can do random checks to verify the data...
if you know a cheaper or even better a free place please advice...
if you know a cheaper or even better a free place please advice...
Have you done this? If so it would be great to get your perspective
This commenter comes into every single thread about trading and talks about buying data from ebay, then consistently demonstrates that he doesn't know the first thing about due diligence on financial data. Each time I try to ask him about his data quality or methodology at even a high level, he responds by accusing me of wanting to steal his work or stop the democratization of data.
I'm going to reiterate this right now: you are not going to find data valuable enough for trading insights on ebay. Go look at the prices of historical data for various time resolutions at TickData or CBOE Livevol and then ask yourself if cheap, competitive data on ebay is too good to be true.
If the data is competitive, it's expensive. If it's not competitive, you shouldn't be paying for it, especially not from ebay. The first time I read about him doing this I was puzzled, now I'm getting increasingly frustrated because it's blatantly false information.
I'm going to reiterate this right now: you are not going to find data valuable enough for trading insights on ebay. Go look at the prices of historical data for various time resolutions at TickData or CBOE Livevol and then ask yourself if cheap, competitive data on ebay is too good to be true.
If the data is competitive, it's expensive. If it's not competitive, you shouldn't be paying for it, especially not from ebay. The first time I read about him doing this I was puzzled, now I'm getting increasingly frustrated because it's blatantly false information.
Unless you are day trading you don't need such a granular data format, so please stop saying you need to spend thousands and thousands of dollars to be able to back test a trading strategy
If your data doesn't need to operate on an intraday scale, you can get perfectly accurate real time or historical data for free just by using a reputable broker like Interactive Brokers. At that point there's no need to buy the data on ebay, so I still don't understand why you ever would.
Actually I have checked IB API and it is very restrictive, e.g. you cannot request let say all daily data for all symbols for 2007 for example or even for one day for that matter...
As far as I understand historical daily data is easy to get from yahoo, google, etc. Data is only difficult to find/expensive when you need tick data.
Yes, I am doing it as a side project and I am looking to retire in a few years and live off that...
I second the above comment; you should write about the process and how it is working out for you.
I'll third this comment. Some more details on what you trade, where you trade, how long you've been doing it, performance metrics, etc... would be fascinating.
As you can see I am paranoid, because the more people trading the same strategy, the less effective it becomes, not to mention, if someone knows your trading strategy it can trade against you... it’s a zero sum game after all...
All that said, here are the 2 most important advises I wish someone told me when I started years ago:
Have an iron clad risk management strategy for your portfolio, e.g. if you are willing to risk 1% of your capital on any given day and tomorrow happens to be the Black Monday and all stock go down 50% you still have to be sure you’ll loose only 1% if you have to sell everything... This is where the back data comes into play – the goal is to back test not how much money you are going to make, but how much money you are going to loose in the worst case scenario if you have to liquidate everything and move to all cash. If you have this working 100%, you just need a trading strategy with a minimal edge and you’ll make money in the long run in the market and the compounding effect will really make you a rich person (rich person to me is someone who makes more money then he spends, WITHOUT working more than 1 hour a day)
And the second advice is never ever trust and buy a trading strategy from someone, the fact he is selling or "teaching" a trading strategy means it is not working and he is trying to make some money by scamming people and selling the dream, all those trading educators cannot make money trading and that’s why they are coming with up with those fake trading courses... just check this website to see the magnitude of this scam and how people are loosing their life savings... I just don’t know why FBI, SEC and the government is not doing something to stop those guys...
https://www.tradingschools.org/
Good luck and as I said before, just treat is as a hobby at first and spend the time testing and you’ll find something that is working for you...
EDIT: some spelling errors fixed.
All that said, here are the 2 most important advises I wish someone told me when I started years ago:
Have an iron clad risk management strategy for your portfolio, e.g. if you are willing to risk 1% of your capital on any given day and tomorrow happens to be the Black Monday and all stock go down 50% you still have to be sure you’ll loose only 1% if you have to sell everything... This is where the back data comes into play – the goal is to back test not how much money you are going to make, but how much money you are going to loose in the worst case scenario if you have to liquidate everything and move to all cash. If you have this working 100%, you just need a trading strategy with a minimal edge and you’ll make money in the long run in the market and the compounding effect will really make you a rich person (rich person to me is someone who makes more money then he spends, WITHOUT working more than 1 hour a day)
And the second advice is never ever trust and buy a trading strategy from someone, the fact he is selling or "teaching" a trading strategy means it is not working and he is trying to make some money by scamming people and selling the dream, all those trading educators cannot make money trading and that’s why they are coming with up with those fake trading courses... just check this website to see the magnitude of this scam and how people are loosing their life savings... I just don’t know why FBI, SEC and the government is not doing something to stop those guys...
https://www.tradingschools.org/
Good luck and as I said before, just treat is as a hobby at first and spend the time testing and you’ll find something that is working for you...
EDIT: some spelling errors fixed.
> more people trading the same strategy, the less effective it becomes
you should pick a more scalable strategy, the kind of strategy that becomes more profitable the more you talk your own book.
you should pick a more scalable strategy, the kind of strategy that becomes more profitable the more you talk your own book.
Not everyone can build their own backtesting tool. I played around with quantopian for a little while and did some live trading with quantopian + robinhood and I never would have been able to do that without quantopian
I am not a developer, but using my brokers API, examples and python, it took me few months to build a very robust and automatic trading system... I did learn python by back testing my trading ideas... To use Quantopian you still have to learn python, so I am not sure where is the benefit, considering all the problems mentioned here...
Or C#
So despite the fact that the subject is interesting, I'd consider it a waste of time to try and gain anything but a basic understanding of the industry and how algo trading works.