Cryptocurrency trading bot in ~40 lines of JavaScript(github.com)
github.com
Cryptocurrency trading bot in ~40 lines of JavaScript
https://github.com/thomasdavis/blockbid-ccxt-tutorials
83 comments
FWIW, I just took a look at our HFT trading system. We specialize in colocated low latency market making in an equity market in a mid-sized country. The following line counts are only for the C++ code directly used in the production system. Doesn't count research, model fitting, ETL pipelines, build systems, etc.
75 kloc - Total line count
10 kloc - Common libraries, utilities, data structures, etc.
10 kloc - Reconstitutes the limit order book from the data feed
10 kloc - General framework (order management, margin tracking, logging, runtime control hooks, etc.)
7 kloc - Safety checks (pre/post trade, price, circuit breakers, etc.)
13k - Adapter for the exchange's datafeed protocol
5k - Adapter for the exchange's order gateway protocol
3k - System for correlating our orders within the datafeed
10k - Signals/alphas
8k - Trading logic
Not claiming that other people's systems will necessarily look anything like ours, but my experience jibes with this too. Trading code is only about 10% of the total codebase for a production-ready trading system. Overall strategy-specific code (signals and trading logic) only make up 25% of the codebase, with the rest being all general-purpose "plumbing".
Even a tiny inconsequential function that most people never think about- reliably identifying which orders in the data feed belong to us- takes nearly half the amount of code as nearly all the trading logic put together.
75 kloc - Total line count
10 kloc - Common libraries, utilities, data structures, etc.
10 kloc - Reconstitutes the limit order book from the data feed
10 kloc - General framework (order management, margin tracking, logging, runtime control hooks, etc.)
7 kloc - Safety checks (pre/post trade, price, circuit breakers, etc.)
13k - Adapter for the exchange's datafeed protocol
5k - Adapter for the exchange's order gateway protocol
3k - System for correlating our orders within the datafeed
10k - Signals/alphas
8k - Trading logic
Not claiming that other people's systems will necessarily look anything like ours, but my experience jibes with this too. Trading code is only about 10% of the total codebase for a production-ready trading system. Overall strategy-specific code (signals and trading logic) only make up 25% of the codebase, with the rest being all general-purpose "plumbing".
Even a tiny inconsequential function that most people never think about- reliably identifying which orders in the data feed belong to us- takes nearly half the amount of code as nearly all the trading logic put together.
It definitely should scare you =D
The script is a tutorial to help complete beginners have an understanding of how to write a bot.
Though you're right, I should have really made it more clear because it is real money.
The script is a tutorial to help complete beginners have an understanding of how to write a bot.
Though you're right, I should have really made it more clear because it is real money.
Well, "real" money since we're talking about cryptocurrencies here.
Agreed.
The bot I am currently working on is pushing 6,000 lines of code, and I feel like it might double in size before it's perfect.
The bot I am currently working on is pushing 6,000 lines of code, and I feel like it might double in size before it's perfect.
"40 lines of JS definitely scare me"
Me too.
I suppose it depends how you want to treat this though.
As a bot programming challenge this seems fair enough. As something to entrust your or others life savings with? No.
Me too.
I suppose it depends how you want to treat this though.
As a bot programming challenge this seems fair enough. As something to entrust your or others life savings with? No.
> As something to entrust your or others life savings with? No.
As first rule, you shouldn't entrust anyone's life savings with a single trader.
Forty lines of Javascript to me is far less scary than 40,000 lines. At least with forty lines, I can easily tell what the hell it is trying to do.
In this case, it's just example code on how to use the API. Its "strategy" is to attempt to buy low and sell high. It doesn't really get any simpler than that.
As first rule, you shouldn't entrust anyone's life savings with a single trader.
Forty lines of Javascript to me is far less scary than 40,000 lines. At least with forty lines, I can easily tell what the hell it is trying to do.
In this case, it's just example code on how to use the API. Its "strategy" is to attempt to buy low and sell high. It doesn't really get any simpler than that.
I was thinking more of the concept than this particular implementation.
This is basically a minimum viable product for others to play around with.
If that playing leads to bot trading competitions, fine. If it leads to someone building a 40k line monster and thinking it might be a good idea to entrust it with a meaningful proportion of anyone's savings.... No.
This is basically a minimum viable product for others to play around with.
If that playing leads to bot trading competitions, fine. If it leads to someone building a 40k line monster and thinking it might be a good idea to entrust it with a meaningful proportion of anyone's savings.... No.
You might want to replace the `openOrders.forEach ...` call with a call to `await Promise.all(openOrders.map ...)` if you want to catch errors and stop the program when cancelling previous orders, and more importantly make sure you cancel orders before doing anything else.
Thanks for the tip, I didn't actually know that technique.
You're welcome. Alternatively, you could simply use "for of" but the requests won't be sent in parallel.
It used ccxt, which is a very big library with most of the infrastructure already implemented:
https://github.com/ccxt/ccxt
https://github.com/ccxt/ccxt
This is dangerous haha!
It seems the price coming back from cryptonator doesn't update, TUSD is a stable coin but surely the bitcoin rate against it should be updating every 30s ? https://api.cryptonator.com/api/ticker/btc-tusd
https://www.cryptonator.com/api/ > Prices are updated every 30 seconds
It seems the price coming back from cryptonator doesn't update, TUSD is a stable coin but surely the bitcoin rate against it should be updating every 30s ? https://api.cryptonator.com/api/ticker/btc-tusd
https://www.cryptonator.com/api/ > Prices are updated every 30 seconds
It also seems exceptionally risky to grab a price off some random website and then trade blindly on another. Shouldn't you be getting the prices from the market you are actually trading in?
You are correct, the code is more of a tutorial, it should have error prevention.
I made it pull from a third party global average because an individual market can shift a lot depending on how volatile it is.
I made it pull from a third party global average because an individual market can shift a lot depending on how volatile it is.
Wouldn’t you want your trading bot to have access to that volatility?!
my point is the API above doesn't seem to update much at all. Maybe every 30 minutes?
https://apiv2.bitcoinaverage.com/#price-data this seems like a better free API, wondering what others use? is there anything with tickdata not just 15s?
https://apiv2.bitcoinaverage.com/#price-data this seems like a better free API, wondering what others use? is there anything with tickdata not just 15s?
One redeeming quality of cryptocurrencies existing is that all the exchanges have REST compliant APIs and real time data available for free
The finance sector’s biggest meme for the last decade and a half was that PhD quants were required to program trading algorithms when really a single exchange cost $12,000 a month for real time data, it came with horrible documentation, support, antiquated protocols, FIX, and then you had to figure out how to co-locate your own server to connect to the exchange fast enough to prevent front running
The gatekeeping is what kept this specialized for the most part
The finance sector’s biggest meme for the last decade and a half was that PhD quants were required to program trading algorithms when really a single exchange cost $12,000 a month for real time data, it came with horrible documentation, support, antiquated protocols, FIX, and then you had to figure out how to co-locate your own server to connect to the exchange fast enough to prevent front running
The gatekeeping is what kept this specialized for the most part
> FIX
Those that do not understand FIX are doomed to reinvent it badly. I only got an appreciation for the protocol when I implemented a FIX server from scratch. Even the wierd text+field delimiter encoding is not bad (except for performance of course).
Session management, order entry and recovery in FIX is well designed and well documented (although it takes a few readings of the spec to put the pieces together). The biggest issue is the plethora of underdocumented extensions that have been bolted on to satisfy niche needs. If you are lucky you can ignore them most of the time.
Another big issue is a lot of exchanges treat FIX as more of a guideline than an hard spec and you have to workaround the lossage.
The best exchange binary protocols I have worked with (at least here in EU, borrow heavily from FIX.
Those that do not understand FIX are doomed to reinvent it badly. I only got an appreciation for the protocol when I implemented a FIX server from scratch. Even the wierd text+field delimiter encoding is not bad (except for performance of course).
Session management, order entry and recovery in FIX is well designed and well documented (although it takes a few readings of the spec to put the pieces together). The biggest issue is the plethora of underdocumented extensions that have been bolted on to satisfy niche needs. If you are lucky you can ignore them most of the time.
Another big issue is a lot of exchanges treat FIX as more of a guideline than an hard spec and you have to workaround the lossage.
The best exchange binary protocols I have worked with (at least here in EU, borrow heavily from FIX.
It's also one of the reasons why I've built https://tardis.dev (historical tick-level cryptocurrency market data replay API).
Hey , just check your product , its looks great!! good job.
Wondering from how long you have been storing this data and have you using the Public API from each exchange to fetch all this data?
Hi, thanks! It took me over six months to finish it and as always with my ideas seems like market for it is too small, at least currently.
Data is available since 2019-03-30 for majority of the supported exchanges. Yes it's all from public APIs.
Data is available since 2019-03-30 for majority of the supported exchanges. Yes it's all from public APIs.
I can imagine , as a tech stack , you must be using ccxt , nodejs and some Timeseries database ?
Also what all distinct "channels" you are saving ?
No it's all custom stuff written from scratch, some in .NET Core, some in JavaScript (Cloudflare workers, Node.js client libs).
I'm saving to flat files on Google Cloud Storage and Wasabi (S3 compatible alternative), given TBs of data it's much cheaper. Filtering is being done on demand.
Regarding channels that heavily depends on exchange, you can use https://tardis.dev/api/v1/exchanges/:exchange API to get the list (eg. https://tardis.dev/api/v1/exchanges/bitmex)
This reminds me of the movie "The Hummingbird Project" - https://www.imdb.com/title/tt6866224/ , where speed matters in all Financial transactions and even a 1ms matters in the game
you are so right
> The gatekeeping is what kept this specialized for the most part.
It certainly kept the vast majority of retail out, but I disagree that this was the main cause of specialization. I think that the number of lone geniuses who could've beat the market using algo trading is a number approaching zero.
It's true that you don't need a PhD, or even higher education, as the math you need can be entirely self taught. However, you need plenty of specialized knowledge to avoid the accidental discharge of the vast number of very expensive footguns that litter the landscape. This knowledge can be gained from literature, but without having someone streetwise as a guide, it's going to be very difficult to discern genuine insight from wishful thinking.
It certainly kept the vast majority of retail out, but I disagree that this was the main cause of specialization. I think that the number of lone geniuses who could've beat the market using algo trading is a number approaching zero.
It's true that you don't need a PhD, or even higher education, as the math you need can be entirely self taught. However, you need plenty of specialized knowledge to avoid the accidental discharge of the vast number of very expensive footguns that litter the landscape. This knowledge can be gained from literature, but without having someone streetwise as a guide, it's going to be very difficult to discern genuine insight from wishful thinking.
Honestly, it’s pretty easy to create a rough approximation of many popular hedge fund strategies. Like on Quantopian, it’s pretty easy to whip up a simple factor strategy in a couple hours. The returns aren’t bad, either.
The part that is hard are trading costs and slippage. But assuming that retail investor’s capital is relatively small and no trading costs (Alpaca provides this), an individual can do pretty well.
The parts that are hard for hedge funds is slippage, illiquidity of assets, trading costs, and client demands. As an individual you really don’t have to worry about those things and consequentially things become much easier.
The only downside is that it’s harder to get as much cheap leverage as the big guys. Also some (HFT) strategies are off limits to individuals, but that’s fine, because HFT really isn’t a good way to make money in the first place.
The part that is hard are trading costs and slippage. But assuming that retail investor’s capital is relatively small and no trading costs (Alpaca provides this), an individual can do pretty well.
The parts that are hard for hedge funds is slippage, illiquidity of assets, trading costs, and client demands. As an individual you really don’t have to worry about those things and consequentially things become much easier.
The only downside is that it’s harder to get as much cheap leverage as the big guys. Also some (HFT) strategies are off limits to individuals, but that’s fine, because HFT really isn’t a good way to make money in the first place.
After a cursory glance, is it me or is floating point being used? I thought this was something you wanted to avoid because of precision errors.
ie 0.2 * 0.1
ie 0.2 * 0.1
This will work!!
parseFloat((0.2 * 0.1).toFixed(10))
parseFloat((0.2 * 0.1).toFixed(10))
The issue is more obvious when rounding something like 0.5, which, due to lack of precision can become either 0 or 1.
Shameless plug: use https://www.npmjs.com/package/bigint-money
Shameless plug: use https://www.npmjs.com/package/bigint-money
> no KYC required
Just believing really hard that the law doesn't apply to you doesn't make it true.
Just believing really hard that the law doesn't apply to you doesn't make it true.
> Just believing really hard that the law doesn't apply to you doesn't make it true.
If you just want to trade crypto/crypto, many exchanges don't require a KYC process. If you want to deposit/withdraw actual dollars, that's a different story.
In any event, KYC is something that exchanges are responsible for, not its customers.
If you just want to trade crypto/crypto, many exchanges don't require a KYC process. If you want to deposit/withdraw actual dollars, that's a different story.
In any event, KYC is something that exchanges are responsible for, not its customers.
I'm scratching my head wondering what the hell that actually that's supposed to mean. Their terms when you sign up do mention a KYC process:
> Blockbid is registered with the Australian Transaction Reports and Analysis Centre (AUSTRAC) which is the regulatory body that deals with the Anti-Money Laundering and Counter-Terrorism Financing Act 2006 (AML/CTF Act).
> [... truncated ...]
> Blockbid operates a Know Your Customer (KYC) and AML/CTF Program that meets the requirements of the AML/CTF Act (and associated laws).
> Blockbid is registered with the Australian Transaction Reports and Analysis Centre (AUSTRAC) which is the regulatory body that deals with the Anti-Money Laundering and Counter-Terrorism Financing Act 2006 (AML/CTF Act).
> [... truncated ...]
> Blockbid operates a Know Your Customer (KYC) and AML/CTF Program that meets the requirements of the AML/CTF Act (and associated laws).
Thanks for finding this guys. Recently we moved to be a crypto to crypto exchange. Without fiat, we no longer are obliged to enable the KYC process for our users.
We will most likely bring it back in as our fiat support matures.
We will most likely bring it back in as our fiat support matures.
Are you still regulated by and report to AUSTRAC? If so, I do not see how you would avoid KYC.
Hell, I don't even know why you'd want to explicitly avoid KYC unless your going out of your way to create a money laundering service.
Also, there's some typos in those terms, ex: no space before email in "contact us [email protected]".
Hell, I don't even know why you'd want to explicitly avoid KYC unless your going out of your way to create a money laundering service.
Also, there's some typos in those terms, ex: no space before email in "contact us [email protected]".
How exactly is exchanging cryptos with other cryptos conducive to money laundering?
I can tell you why you want to avoid KYC: It's really annoying for the users and it's labor-intensive for the company.
I can tell you why you want to avoid KYC: It's really annoying for the users and it's labor-intensive for the company.
> How exactly is exchanging cryptos with other cryptos conducive to money laundering?
One of the fundamental rules of money laundering is that anything that can be used for money laundering will be used for money laundering. Allowing anonymous users to shift from one crypto to another could be part of a money laundering routine. It does not have to include any fiat currency in the flow.
> I can tell you why you want to avoid KYC: It's really annoying for the users and it's labor-intensive for the company.
Ha! It's annoying, labor intensive, and a cost center for non-crypto companies as well. I can assure you that the entire non-crypto financial services industry would avoid it if they could as well.
One of the fundamental rules of money laundering is that anything that can be used for money laundering will be used for money laundering. Allowing anonymous users to shift from one crypto to another could be part of a money laundering routine. It does not have to include any fiat currency in the flow.
> I can tell you why you want to avoid KYC: It's really annoying for the users and it's labor-intensive for the company.
Ha! It's annoying, labor intensive, and a cost center for non-crypto companies as well. I can assure you that the entire non-crypto financial services industry would avoid it if they could as well.
> Allowing anonymous users to shift from one crypto to another could be part of a money laundering routine. It does not have to include any fiat currency in the flow.
What's the point? That's like exchanging cash for cash. I guess you could use it to "launder" crypto in the sense that you can make transfers off-chain. The exchange still has those records though, they don't need KYC to provide them, if necessary.
What's the point? That's like exchanging cash for cash. I guess you could use it to "launder" crypto in the sense that you can make transfers off-chain. The exchange still has those records though, they don't need KYC to provide them, if necessary.
Ya, that probably wont last long if the exchange is centralized. Would have to re implement on bisq/kyber/uniswap or another decentralized exchange for this to last without KYC. Then of course you have to deal with massively increased latency/network speed to settle trades.
If any wants to try their hand at quant trading, Quantopian is a great place to get started. Great data sources and you can get allocated capital and make a little bit of money. Also the competitions are fun and you get paid $50/day for having (essentially) the best Sharpe ratio.
If you want to trade your own money, Alpaca is really awesome. Pretty easy to import an algorithm from Quantopian and use it on Alpaca.
I’ve been managing about 15% of my portfolio through Alpaca and it’s been great. On my account, I can get 4x intraday leverage, which has been absolutely awesome. Also there’s no trading costs, so I don’t have any problem with daily rebalancing (I actually rebalance to 4x leverage in the morning and deleverage at the end of the day).
Highly recommended! There’s a lot of alpha in the market for retail investors (with leverage, no transaction costs, and the ability to short).
If you want to trade your own money, Alpaca is really awesome. Pretty easy to import an algorithm from Quantopian and use it on Alpaca.
I’ve been managing about 15% of my portfolio through Alpaca and it’s been great. On my account, I can get 4x intraday leverage, which has been absolutely awesome. Also there’s no trading costs, so I don’t have any problem with daily rebalancing (I actually rebalance to 4x leverage in the morning and deleverage at the end of the day).
Highly recommended! There’s a lot of alpha in the market for retail investors (with leverage, no transaction costs, and the ability to short).
Quantopian looks amazing, I hadn't seen that. How much historical data and fundamentals do they have for equities?
They have minute data and a bunch of free fundamentals for US markets going back to 2002. If you want to trade equities, they should have everything you need. And if they don't, you can buy proprietary datasets for a reasonable price, or you can upload your own.
Advertisement for their API? Trading is gambling unless you have some insider info to increase your odds. Only rich or stupid people gamble.
Actually, it's not irrational for non-rich smart people to gamble ie poker is a game where skill almost always wins the day, and betting baseball parlays can, with a good deal of study, land in a positive expectation.
Furthermore, the sheer number of quite profitable high speed trading platforms seems to totally rebuke your assertion.
Furthermore, the sheer number of quite profitable high speed trading platforms seems to totally rebuke your assertion.
Cool, until the tax declaration comes up.
Luckily, there are now tools such as Accointing or CryptoTax that take care of the 30,000 transactions.
Luckily, there are now tools such as Accointing or CryptoTax that take care of the 30,000 transactions.
won't tax declaration only apply to executed orders? those will be much lower.
Yes, with 20% you probably have good chances not do get anything taxable at all
should there be successful executed orders in the magnitude of 30k. I'd be more than happy to handle the tax declation some way or the other :-)))
[deleted]
I understand that back-testing is a foolish way to work out whether a bot works, but it'd be interesting to see how this performs on arbitrary lengths of past history.
Like, is this enough to beat a lazy index fund calculated over the same time, with interest reinvested?
Why is backtesting foolish? As long as you haven't used the data you're using to backtest in any way, it seems like a reasonable way to test your strategy.
I think I was mainly trying to avoid risking that anyone reads this and thinks past performance is an indication of future performance.
Your point is perfectly valid and is a useful addition to my prior post.
Your point is perfectly valid and is a useful addition to my prior post.
I think because the data you are backtesting doesn't react to the trades you make.
Unless you have order book data, then you can simulate market impact of your trades as well.
Depends on your scenario, but in a general sense, no, the extra degree of confidence from having that extra info is limited.
Just being visible in the OB will cause other people to act differently from if you weren't. This of course has a profound effect on strategies relying on passive orders.
Just being visible in the OB will cause other people to act differently from if you weren't. This of course has a profound effect on strategies relying on passive orders.
Or course you're right, there is a lot of nuance into that.
Btw, would you care to review historical tick crypto data API (https://tardis.dev) I've built? Seen your 'about' hence question.
At first glance looks useful. But data quality is really hard to judge without a deep dive.
That does sound like a fun experiment.
I will give a shout to these guys if anyone is interested in back testing -> https://www.coinapi.io/
I will give a shout to these guys if anyone is interested in back testing -> https://www.coinapi.io/
[deleted]
Great to see people are interested in the tutorial.
I'd be happy to answer any questions to the best of my ability.
I'd be happy to answer any questions to the best of my ability.
I can see that there is no variable of "fees" which usually applies even on a LIMIT order.
For example Coinbase Pro has Maker fees of 0.15%. Ideally this fees should be added to BUY order and subtracted from SELL order, as that will actually cost you.
Cool stuff. 20% Deviation from the market price is quite high though. I suppose orders will never get a chance to execute. What would be a more realistic percentage? Could it be based on the current volatility for example? Or am I missing something.
Yep, you are 100% correct.
This is a very naive bot, it will rarely ever execute at that margin but you will never lose either. You are just betting on the fact that hopefully someone buys out the orderbook by accident or because they really want that asset.
A more realistic margin is 0.0000001% but when you start doing that, you will start losing money so you will want a much better algorithmic trading strategy.
I will probably write a hedging tutorial next.
This is a very naive bot, it will rarely ever execute at that margin but you will never lose either. You are just betting on the fact that hopefully someone buys out the orderbook by accident or because they really want that asset.
A more realistic margin is 0.0000001% but when you start doing that, you will start losing money so you will want a much better algorithmic trading strategy.
I will probably write a hedging tutorial next.
the current implementation will do great on flash crashes :-)
>trading programmatically
>trading money
>in javascript
Am I alone in thinking this is insane?
>trading money
>in javascript
Am I alone in thinking this is insane?
No, it is indeed fairly insane. It's a good proof-of-concept of API-based trading, but I am far too timid to trust this sort of thing to a bot, especially considering that other people are running better bots.
tbf, you ought to be getting alarm bells from "Australian crypto exchange, no KYC required" long before we start looking at whether the coding approaches are remotely sane.
Considering the well established likelihood of a flash crash in the markets you're trading in, the exchange's own code being stuck together with duct tape or your/their assets being frozen, you probably shouldn't be trading anything you're not willing to lose to buggy javascript anyway.
Considering the well established likelihood of a flash crash in the markets you're trading in, the exchange's own code being stuck together with duct tape or your/their assets being frozen, you probably shouldn't be trading anything you're not willing to lose to buggy javascript anyway.
40 lines, yeah...I can do it in 2 lines even, if I use external libraries. body and boom, I'm done.
When did Blockbid exchange launch? Looks like just yesterday. The btc/usd order book is just 5 levels deep.
It seems to be for a specific exchange that has $240 24 hour volume...
[deleted]
40 lines of JS definitely scare me.