This series of articles outlines my attempts at improving the performance of my ethereum bots. I hope these will be useful for anybody doing automated defi trading where it is critical for your transaction to be in the next block (or even the first transaction included in the next block).
- Optimizing defi trading, part 1: Ethereum nodes
- Optimizing defi trading, part 2: Gas prices
- Optimizing defi trading, part 3: Pending transactions
Use a local node
When I started out, I used services like Infura and Alchemy for connecting to the ethereum blockchain. These are great for starting out because they have generous free tiers, however once you get your scripts working, you'll eventually want to move to your own private local ethereum node for a few important reasons.
No rate limits
Whenever a new block is mined, you can imagine a timer starts counting down to the next block. You want to get as much done as quickly as possible and send out your transactions early enough to be in the next block. The later you send out your transactions, the more likely it won't be included in the upcoming block.
This means being able to make hundreds or thousands of ethereum calls in a burst at the start of each new block. Unfortunately third party services have a rate limit to prevent users from making too many calls in a short time.
Your own local node will not have a fixed rate limit, but the performance will depend on the hardware of the server running the node.
No call limits
Third party nodes set a limit to how many calls you can make each day or month in total. For some types of bots, the limit is high enough that you won't need to worry about hitting it. However if you are hitting the limit, your options are to upgrade your subscription and pay more for a higher limit, or to run your own local node where you won't have a call limit.
If you have the expertise or you can learn to run your own node, it becomes far more cost-efficient to do so, instead of paying more money to the third party node provider.
Low latency
Connecting to a remote node means you suffer from network latency. This is a delay in receiving the response to any call.
Even a relatively low latency of 20ms means you are limited to 50 calls in a second (if the calls are done synchronously).
Connecting to your local node over ipc means every call gets a response very quickly at less than 1ms for each call. This lets my bot check thousands of prices and contract values in a couple of seconds, something that it couldn't do when using third party nodes even if there were no rate limit or call limit.
Downsides
There are different costs related to running and maintaining the node (server costs, time costs). This section is specifically about the downsides of using your own node.
When you use a third party service for running a node, the provider runs a global network of nodes that are peered to each other, and as a whole has many more peers to external nodes and miners than your own node would. Having many peers means blocks and transactions get propagated faster, and peering with miner nodes directly is the best chance at getting transactions included in the next block.
However your single local node doesn't have this kind of network and peering. This means blocks may be delayed when receiving, to the point that you may even skip a block on rare occasions (it really depends on your peers and the origin of each mined block). Similarly, your outgoing transactions may not propagate in time to the miner that mines the next block.
In this screenshot, you can see block #11660805 and #11660807 were not received individually.
The following sections suggest some ways to get around these issues.
Broadcasting to multiple nodes
Your local node is great for reading lots of data from the ethereum blockchain, but sending transactions could be improved.
One way to do this is to broadcast your transactions through your local node and also through third party nodes at the same time. The free tiers of third party nodes should be more than enough if you only broadcast through them, while making all your eth calls on your local node instead.
Transactions with the same nonce won't be included multiple times, so you can feel free broadcasting the same transaction everywhere you can. In the end, it will only be included in the blockchain once.
Block routing service
I recently discovered a service called BloxRoute. This is a service that promises to deliver blocks as fast as possible to your local node, and to propagate transactions quickly. You can do this by setting up their gateway which acts as a fake peer for your ethereum node. It will deliver block data to your node just like any other peer, and your node can still peer with regular nodes as usual.
BloxRoute is not the only service of this kind. There's a great article here comparing BloxRoute with other similar services which I recommend checking out.
