Why building profitable trading bot is hard?
I wants know why 90% of trading bots are fails. What's the reason behind it.
45 comments
What's the reason behind it.
Lol! Because you only hear about the bad bots.
Think about this --- if you had a really good bot capable of actually being consistently profitable, why would you ever advertise it or tell anyone about it or even acknowledge it's existence?
Lol! Because you only hear about the bad bots.
Think about this --- if you had a really good bot capable of actually being consistently profitable, why would you ever advertise it or tell anyone about it or even acknowledge it's existence?
Yep, every bot for sale is proof it doesn’t work. They do exist (at least for a length of time) but they are not for sale.
Paying $49 a month for access to a money making machine is kinda like buying the title to the Golden Gate Bridge --- it's just a slightly more sophisticated con targeted at the internet crowd.
I just recently got into a trading firm. I also had this sort of question before and its relative question, that is “can individual trader really make money without having to be a fortune teller”?
What I found out that my trading firm is a market maker, and this trading firm, alongside with other non individual (corporate) players in trading world, have moats in their cutting of fees.
To make a profitable trade, you need to take into account many of the fees (and taxes) or else you get into death by thousand cuts. Trading firm simply operates in a different fee structure than individual traders. Some market makers are actually getting paid instead, just by providing liquidity. That’s why they can be profitable.
If an individual stock trader buys stock A for $100 and sell it later for $100, it is considered loss (due to fees) or at least break even. Not so much for trading firms, they actually make profit.
Since trading bots are just individual traders, same rules apply.
CMIIW
What I found out that my trading firm is a market maker, and this trading firm, alongside with other non individual (corporate) players in trading world, have moats in their cutting of fees.
To make a profitable trade, you need to take into account many of the fees (and taxes) or else you get into death by thousand cuts. Trading firm simply operates in a different fee structure than individual traders. Some market makers are actually getting paid instead, just by providing liquidity. That’s why they can be profitable.
If an individual stock trader buys stock A for $100 and sell it later for $100, it is considered loss (due to fees) or at least break even. Not so much for trading firms, they actually make profit.
Since trading bots are just individual traders, same rules apply.
CMIIW
Much of this is true. However it doesn't really answer their question of why most trading bots aren't profitable. So you covered fees which really only affect mostly breakeven trades. That leaves out the trades with any real gains. If you have a bot making positive trades above the fee level (usually minor, like $2) then it's not a concern. The question becomes, why can't the bot achieve these profitable trades?
I don't think it's an issue of bots not being profitable but the simple strategies most of the off the shelf trading bots use not being profitable. There are a ton of financial firms using algorithmic trading bots that make plenty of money doing it. The difference is they're experts and developing their own complex strategies. When you try to run your own trading bot using just the basic strategies most of them use that's who you're competing against. Of course your poor little bot is going to get trounced.
However doesn't a market maker run the risk of holding an unbalanced bag when the market moves? Also curious is there any true market maker these dayd? Aren't they all doing trading actually or just my fantasy? Thanks!
What do you think the distinction between market making and trading is?
Also, yes market makers can get left with bad positions if the market moves against them, but when I was in the industry operational failures were the much more troubling issues.
Also, yes market makers can get left with bad positions if the market moves against them, but when I was in the industry operational failures were the much more troubling issues.
I think MM doesn't take sides and try to balance the book while making a lot of pennies from both sides. However I heard (not in industry) that many MM are actively taking sides so the question.
Most market making strategies are run by firms that have a portfolio of other strategies that will be directional. Those mm strats, besides making the spread when possible, also allow the firm to maintain fee structures which make other strats possible and order inventory that allows them to have priority in the market. Most of the time the overall firm is mostly neutral while individual strategies/desks are directional.
All of its trading though and none of this is new.
All of its trading though and none of this is new.
Thanks! This makes a lot of sense~
I run https://feetr.io. I do stock discovery as opposed to automated trading (at this point). Almost a year into recording stock picks, it is averaging 4.0624% per day across all stocks, and 5.9948% for the best stock of that day.[0]
With the above, you'd think that a trading bot would be a piece of cake. The easiest part of the process, probably. As of right now, I've found that it's hard to model intuition using code. It's easy to watch a graph and say "I think it's about time to sell", but it's hard to give that understanding to a computer. I do believe that it's a problem that can be solved, and certainly you can look at Aladdin[1] as a system that has been very successful in this space.
Don't take negativity as a reason to not do something though. If you're passionate and interested, then go prove everyone wrong.
[0] "per day" references per day that Feetr finds a stock
[1] https://www.blackrock.com/aladdin
With the above, you'd think that a trading bot would be a piece of cake. The easiest part of the process, probably. As of right now, I've found that it's hard to model intuition using code. It's easy to watch a graph and say "I think it's about time to sell", but it's hard to give that understanding to a computer. I do believe that it's a problem that can be solved, and certainly you can look at Aladdin[1] as a system that has been very successful in this space.
Don't take negativity as a reason to not do something though. If you're passionate and interested, then go prove everyone wrong.
[0] "per day" references per day that Feetr finds a stock
[1] https://www.blackrock.com/aladdin
4% daily?! That seems crazy high. If it found a stock even 100 days, you would have quadrupled your money over the year.
Well the 4% is across all stocks. Multiple can be found per day. Sometimes none will be found.
All data is 100% available on feetr.io after market close and, when it launches, Monday will be given for free on Reddit, Twitter, and to anyone who has a free tier account at feetr. Monthly subscription will be $8.99.
All data is 100% available on feetr.io after market close and, when it launches, Monday will be given for free on Reddit, Twitter, and to anyone who has a free tier account at feetr. Monthly subscription will be $8.99.
This sounds too good to be true.
Yes, hindsight is 20/20.
It's easy to pick the high/sell price for the day after the day is over. Predicting this on the fly is a whole lot trickier. It's very unlikely you will catch the day's high. Unless you're psychic, you'll probably sell either too early or too late with diminished returns..
It's easy to pick the high/sell price for the day after the day is over. Predicting this on the fly is a whole lot trickier. It's very unlikely you will catch the day's high. Unless you're psychic, you'll probably sell either too early or too late with diminished returns..
Absolutely correct. Feetr doesn't invest on people's behalf at this point, it's purely stock discovery. The idea of showing the difference between the open price and the highest is that this is how much profit could've been made that day.
The concept that I try to push is to focus on compound interest. To start with a small amount and just keep rolling it over into (near) daily profits. Don't try to time the top, sell when you're comfortable.
The concept that I try to push is to focus on compound interest. To start with a small amount and just keep rolling it over into (near) daily profits. Don't try to time the top, sell when you're comfortable.
The big guys are already using market sentiment analysis, so I think these are their scraps at best. I wouldn't be surprised if there is a secondary or deeper pattern in the data. For example, the highest gainers could be considered too risky for the big players.
Thank you very much, I appreciate that!
I would offer it for free but I believe that it's immoral (and potentially illegal) for me to be able to invest in the stocks when it launches, and so I've priced it at a point that I think people can justify. I tried to match Netflix on price, but they increased it so now I can say I'm cheaper than them.
If investing on Monday is making people more than $9 then it's worth it to subscribe, and if it isn't then I need to rethink the algorithm.
I would offer it for free but I believe that it's immoral (and potentially illegal) for me to be able to invest in the stocks when it launches, and so I've priced it at a point that I think people can justify. I tried to match Netflix on price, but they increased it so now I can say I'm cheaper than them.
If investing on Monday is making people more than $9 then it's worth it to subscribe, and if it isn't then I need to rethink the algorithm.
> I believe that it's immoral (and potentially illegal) for me to be able to invest in the stocks when it launches
Great marketing here, the tool is so good that it would be "immoral/illegal" for the creator to use it.
Great marketing here, the tool is so good that it would be "immoral/illegal" for the creator to use it.
Please don't misunderstand, I absolutely have been using it: https://www.indiehackers.com/post/ive-made-over-700-return-w...
But when it launches, I have a competitive advantage over everyone using the service and that doesn't sit right with me.
But when it launches, I have a competitive advantage over everyone using the service and that doesn't sit right with me.
I think comparing the open price to the days high price is a little disingenuous. Stocks fluctuate so I'm not sure what metric would be more appropriate, but it would be interesting to see the discrepancy if you just took the close price.
I don't know that I agree. It's an algorithm geared towards day trading so the highest price of the day is a much better metric than the close price, as you should've sold by then.
This leaves only one tiny little problem --- predicting the highest price of the day before the day is over.
Any suggestions for doing this?
If you can't do this; if you sell too early or too late, your return will potentially be a lot less than picking the high price after the fact.
In other words, the stated returns are maximized assuming perfect clairvoyance --- which no one has. Basing return on closing price assumes zero clairvoyance --- which the average person has --- and is thus a more realistic measure.
Any suggestions for doing this?
If you can't do this; if you sell too early or too late, your return will potentially be a lot less than picking the high price after the fact.
In other words, the stated returns are maximized assuming perfect clairvoyance --- which no one has. Basing return on closing price assumes zero clairvoyance --- which the average person has --- and is thus a more realistic measure.
The best thing you can do is leave the idea that you will time it correctly. Take profit when you are comfortable. Feetr provides stocks almost every day and, to me, the goal is compound interest.
The metrics are as they are because, right now[0], there are no better options to validate how Feetr performed. These values are, like you say, theoretical maximums and have not been matched by humans in almost a year. You get the top every so often but not consistently.
However, I wouldn't say that this makes the service less useful than people think. I have made 700% returns and this allowed me to quit my job and work on Feetr full time. Other beta testers have made more than their salary using this.
[0] As above, there will be an investment service offered at some point
The metrics are as they are because, right now[0], there are no better options to validate how Feetr performed. These values are, like you say, theoretical maximums and have not been matched by humans in almost a year. You get the top every so often but not consistently.
However, I wouldn't say that this makes the service less useful than people think. I have made 700% returns and this allowed me to quit my job and work on Feetr full time. Other beta testers have made more than their salary using this.
[0] As above, there will be an investment service offered at some point
Finally a question I can answer (but ppl might not like it)! :)
It’s my personal opinion, based on my time and experience (and several books about the field), “90% fail” in the same way “90% of business fail.”
Tons of reasons but I truly believe lack of alpha is overestimated. I think finding alpha is not nearly as hard as people make it out to be. Execution and continuously innovating is the real challenge. Design your systems, knowing you will have to change up your strategy and focus on a system that will help you find new alpha.
I believe after lots of research, this is how Rentech did it.
And to touch on the business ananology again, you could have the best donuts in town but if the rest of your business is fundamentally flawed, it will become part of the “90%.”
Now, I’ll say this, I don’t manage some insane portfolio or anything. However, I run a private firm (myself and family) that makes for a comfortable life. I work more than 9-5, but I enjoy the challenges and freedom.
If you’re asking about more complex bots used in managing larger, institutional funds, I’m afraid I can’t answer that… but I imagine the challenges are quite different.
I try to post things that have helped me with building these systems from time to time.
http://www.twitter.com/robswc
Lmk if there’s anything else, I can try my best to answer.
It’s my personal opinion, based on my time and experience (and several books about the field), “90% fail” in the same way “90% of business fail.”
Tons of reasons but I truly believe lack of alpha is overestimated. I think finding alpha is not nearly as hard as people make it out to be. Execution and continuously innovating is the real challenge. Design your systems, knowing you will have to change up your strategy and focus on a system that will help you find new alpha.
I believe after lots of research, this is how Rentech did it.
And to touch on the business ananology again, you could have the best donuts in town but if the rest of your business is fundamentally flawed, it will become part of the “90%.”
Now, I’ll say this, I don’t manage some insane portfolio or anything. However, I run a private firm (myself and family) that makes for a comfortable life. I work more than 9-5, but I enjoy the challenges and freedom.
If you’re asking about more complex bots used in managing larger, institutional funds, I’m afraid I can’t answer that… but I imagine the challenges are quite different.
I try to post things that have helped me with building these systems from time to time.
http://www.twitter.com/robswc
Lmk if there’s anything else, I can try my best to answer.
What books and advice would you give someone with zero knowledge that would make them a good investor such that they would be highly competent in managing a portfolio?
I’m afraid I can’t say I have a good answer for that, as I don’t consider myself an investor.
I actually don’t know fundamentals well at all and rarely keep up with news.
The systems I designed are for trades lasting no more than a few hours, so much of that stuff never comes into play.
I’ll just say, I think investors like Warren Buffett are good to emulate. They play the long game and play it well… I think doing something like that is a full time job though.
Last note, just knowing a few ppl personally, I’ve seen people grow their wealth by just sticking it into the market and forgetting about it ;)
I actually don’t know fundamentals well at all and rarely keep up with news.
The systems I designed are for trades lasting no more than a few hours, so much of that stuff never comes into play.
I’ll just say, I think investors like Warren Buffett are good to emulate. They play the long game and play it well… I think doing something like that is a full time job though.
Last note, just knowing a few ppl personally, I’ve seen people grow their wealth by just sticking it into the market and forgetting about it ;)
Watch the Warren Buffet videos on YouTube. I did that and followed his advise. It works.
Short term trading, via bot or not, is similar to a zero sum game. Building profitable bots is hard because you are competing for profit with very good ones.
Building profitable bot isn't the hard part. The problem is that markets go through regime changes and a bot that had been profitable in previous regime could turn into hole digger...
Just look at compound results of hedge funds (published by Goldman Sachs). These smart quants are consitently buying high and selling low for last couple of months and years.
[] GSTHHVIP vs S&P500
Just look at compound results of hedge funds (published by Goldman Sachs). These smart quants are consitently buying high and selling low for last couple of months and years.
[] GSTHHVIP vs S&P500
The stock market is very efficient - aka, the price reflects most, if not all available information at the time.
Therefore, a trading bot would need to trade on information that is not available to other bots, for it to be profitable. This is hard to achieve, and each bot that _do_ achieve it is making the market even more efficient.
Therefore, a trading bot would need to trade on information that is not available to other bots, for it to be profitable. This is hard to achieve, and each bot that _do_ achieve it is making the market even more efficient.
Two investors can look at the same balance sheet and come to opposite conclusions. 90% of actively managed investment funds underperform the market. It's a hard problem, full stop. If you're good at solving hard problems, you could solve something less boring and probably make more.
Managed funds must pay managers; probably out of the returns.
ETFs (Electronically Traded Funds) have no fund management fees; like Class B stock, ETFs typically are not voting shares (which you don't have when you buy a mutual fund anyways).
Algotraders reference e.g. an S&P 500 ETF as the default benchmark for comparing a portfolio's performance.
Quarterly earnings reports are filed as XBRL XML. A value investor might argue that you don't need to rebalance a portfolio until there is new data about technical fundamentals for technical analysis.
The average bear trades on sentiment and comparatively doesn't at all appropriately hedge; this is part of Behavioral economics, the technical reason why some people actually can outperform the market, imho.
ETFs (Electronically Traded Funds) have no fund management fees; like Class B stock, ETFs typically are not voting shares (which you don't have when you buy a mutual fund anyways).
Algotraders reference e.g. an S&P 500 ETF as the default benchmark for comparing a portfolio's performance.
Quarterly earnings reports are filed as XBRL XML. A value investor might argue that you don't need to rebalance a portfolio until there is new data about technical fundamentals for technical analysis.
The average bear trades on sentiment and comparatively doesn't at all appropriately hedge; this is part of Behavioral economics, the technical reason why some people actually can outperform the market, imho.
If the financial analyst does not have a (possibly piecewise) software function to at least test with backtesting and paper trading, do they even have an objective relative performance statistic? Your notebook or better should also model fees and have a parametrizable initial balance.
Here's the awesome-quant link directory: https://github.com/wilsonfreitas/awesome-quant
Here's the awesome-quant link directory: https://github.com/wilsonfreitas/awesome-quant
Only over the very long term. Short term there is a lot of noise and randomness.
Market efficiency. Trades are all public as well.
So any regularly profitable trade of any substantial volume would be copied pretty quickly, squeezing out any profit.
No doubt one could find an obscure trade, keep volumes low and stay under the radar long enough to make a tidy sum.
So any regularly profitable trade of any substantial volume would be copied pretty quickly, squeezing out any profit.
No doubt one could find an obscure trade, keep volumes low and stay under the radar long enough to make a tidy sum.
Because if it was that easy, everyone would be making a trading bot.
[deleted]
“Short term trading is speculation not investing.”
Warren Buffet
Sock markets area a gamble than commonsense.