This is the last article in the series where I describe the small improvements I made and publish the link to the github repository.

If you missed the previous article and you want to read them:

• Building an Arbitrage Bot on Starknet: Part 0 - Understanding the Fundamentals
• Building an Arbitrage Bot on Starknet: Part 1 - The basics
• Building an Arbitrage Bot on Starknet: Part 2 - The bot
• Building an Arbitrage Bot on Starknet: Part 3 - Conclusions

Improvements made

I ended my previous article by describing what improvements could be made. In this short time (there were holidays) I worked on

• Made a more robust loop. Now if a coroutine fails, the bot crashes.
• Removed ccxt. Implemented the Binance class using the Binance APIs.
• Fixed some bugs and refactored a bit AVNU

In the future, I am still interested in removing AVNU and creating a more robust infrastructure.

StarkShift

You can find the repository of StarkShift here. Development will continue on github and I will only write articles when there are interesting developments or features.

I just wanted to describe how an arbitrage bot works and show how easy it is to create a new one.

This article is part of a series on building an arbitrage bot between centralised (CEX) and decentralised (DEX) exchanges on Starknet.

In this article I will finally cover the bot implementation, explaining the interfaces, the communication with the exchanges and the Arbitrage class.

• Building an Arbitrage Bot on Starknet: Part 0 - Understanding the Fundamentals
• Building an Arbitrage Bot on Starknet: Part 1 - The basics
• Building an Arbitrage Bot on Starknet: Part 2 - The bot
• Building an Arbitrage Bot on Starknet: Part 3 - Conclusions

Project overview

The bot is written in Python 3.12 using asyncio. It makes extensive use of coroutines and some of the features offered by Asyncio. If you not know how asyncio works, you can start here.

As I mentioned in my previous article, I decided to use poetry as a dependency manager. One of its features is automatically creating the project folder when poetry new is executed. The folder looks like this:

.
├── Dockerfile
├── README.md
├── poetry.lock
├── pyproject.toml
├── starknet_arbitrage
│   ├── __main__.py
│   ├── arbitrage.py
│   ├── core
│   │   └── types.py
│   ├── exchange
│   │   ├── base.py
│   │   ├── cex
│   │   │   └── binance.py
│   │   └── dex
│   │       └── avnu.py
│   └── starknet.py
└── tests
    └── __init__.py

• __main__.py loads the setting and starts the bot
• arbitrage.py contains the logic of the bot
• core defines common types
• exchange handles the communication between centralised, decentralised exchanges and the bot
• starknet.py handles the communication between Starknet and the bot

Core

The module defines types and logic useful for the whole library. Currently, I have only defined common types in core/types.py

@dataclass
class Token:
    name: str
    address: str = ""
    decimals: int = 18

    def __str__(self) -> str:
        return f"{type(self).__name__}(name={self.name})"


@dataclass
class Symbol:
    base: Token
    quote: Token

@dataclass
class Ticker:
    raw: dict
    bid: Decimal
    bid_amount: Decimal
    ask: Decimal
    ask_amount: Decimal


@dataclass
class Wallet:
    raw: dict
    token: Token
    amount: Decimal

    def __str__(self) -> str:
        return f"{type(self).__name__}(token={self.token}, amount={self.amount})"

    @staticmethod
    def empty(token: Token) -> Wallet:
        return Wallet({}, token, Decimal(0))


@dataclass
class Order:
    raw: dict
    symbol: Symbol
    amount: Decimal
    price: Decimal
    side: str

The names of the classes are rather self-explanatory. raw fields contain the original message sent by the exchange. They can be used for debugging or extracting exchange-dependent information, as required by AVNU.

Decimal objects provide support for fast, correctly-rounded decimal floating-point arithmetic. Although they are slower than float, they are a good compromise for avoiding the floating- point errors that float suffers from.

Exchange comunication

Although centralised and decentralised exchanges are completely different services, with different features and different APIs, I want a unified interface so the bot can work easily. We are only interested in:

• subscribe to a ticker
• send market orders

Based on this, the Exchange interface is defined as:

class Exchange:
    @abc.abstractmethod
    async def subscribe_ticker(self, symbol: Symbol, **_):
        """Subscribe to the ticker channel."""

    @abc.abstractmethod
    async def buy_market_order(self, symbol: Symbol, amount: Decimal, *args, **kwargs):
        """Insert a buy market order."""

    @abc.abstractmethod
    def receiver_queue(self) -> asyncio.Queue:
        """Return the queue containing the messages from the Exchange."""

    @abc.abstractmethod
    async def sell_market_order(self, symbol: Symbol, amount: Decimal, *args, **kwargs):
        """Insert a sell market order."""

The receiver_queue method returns a Queue containing the messages received from the exchange. Remember that the bot connects to the exchanges using websockets. There will be a continuous flow of messages, which will be parsed by the class and added to the receiver_queue.

I decided to implement only two exchanges: Binance and AVNU. The latter is not a real DEX but an aggregator. However, it is useful because it offers an API that provides prices and quotes for all the DEXes implemented.

Binance

No need to introduce Binance. It is the centralised exchange with the highest volume and market share. To handle the websocket, ccxt is used. The Binance class defined in exchange/cex/binance.py is a wrapper around the ccxt methods.

class Binance(Exchange):
    """Binance exchange."""

    def __init__(self, api_key: str, secret_key: str) -> None:
        self._exchange_handle = ccxt.pro.binance(
            {"apiKey": api_key, "secret": secret_key}
        )
        self._receiver_queue = asyncio.Queue()
        
    ...

As you can see, __init__ needs api_key and secret_key to authenticate in the authenticated channel and to send orders. The class is very simple. The bot calls subscribe_ticker, then ccxt sends the request to subscribe to the ticker channel and _handle_ticker handles the messages.

    async def _handle_ticker(self, symbol: str):
        """Handle ticker messages and connection."""
        while True:
            msg = await self._exchange_handle.watch_ticker(symbol)

            ticker = Ticker(
                msg,
                Decimal(msg["info"]["b"]),
                Decimal(msg["info"]["B"]),
                Decimal(msg["info"]["a"]),
                Decimal(msg["info"]["A"]),
            )

            await self._receiver_queue.put(ticker)

    async def subscribe_ticker(self, symbol: Symbol, **_):
        """Subscribe to the ticker."""
        exchange_symbol = f"{symbol.base.name}{symbol.quote.name}"
        asyncio.create_task(self._handle_ticker(exchange_symbol))

The bot will also need to subscribe to the authenticated channel in the same way as _fetch_ticker, so it can receive wallet updates.

The Binance websocket has one problem: it does not send initial snapshots of both, the balances and the ticker. For the ticker, this isn’t a big deal for active pairs, as an update is sent immediately. However, for the balances, it is an issue because an update is only sent when the balances change. This means an initial request for the balances is required.

    async def subscribe_balance(self, symbol: Symbol, **_):
        """Subscribe to the authenticated endpoint for balances.
        
        NOTE: Balances are filtered by `Symbol`.
        
        """
        exchange_symbol = f"{symbol.base.name}{symbol.quote.name}"
        await self._receiver_queue(
            await self._exchange_handle.fetchBalance(exchange_symbol)
        )
        
        asyncio.create_task(self._handle_balance(exchange_symbol))

The method of sending an order is as follows:

async def buy_market_order(
        self, symbol: Symbol, amount: Decimal, *_, **__
    ):
        """Insert a new buy market order."""
        exchange_symbol = f"{symbol.base.name}{symbol.quote.name}"

        order = await self._exchange_handle.create_order_ws(
            exchange_symbol, "market", "buy", float(amount), params={"test": True}
        )

        await self._receiver_queue(Order(
            order, symbol, Decimal(str(order.price)), Decimal(str(order.amount)), "buy"
        ))

AVNU

As aforementioned, AVNU is not a real DEX, it is an aggregator. I use this protocol because its API provides prices and quotes from all supported exchanges. At the expense of higher fees, the bot can easily swap with 10 different DEXes.

There is only one problem with the API: it does not offer a websocket connection, so the class needs to simulate it. Remember that I want to abstract this problem to Arbitrage. The bot expects to receive a stream of messages, not to make requests.The main idea is to create coroutines that make requests every $$N$$ seconds. But first let’s define some useful constants for the class:

ETH = Token(
    "ETH", "0x049d36570d4e46f48e99674bd3fcc84644ddd6b96f7c741b1562b82f9e004dc7", 18
)


URLS = {
    "base": "https://starknet.api.avnu.fi",
    "quotes": "swap/v2/quotes",
    "prices": "swap/v2/prices",
    "sources": "swap/v2/sources",
    "build": "swap/v2/build",
}

ETH is a Token object representing the Ethereum token on Starknet, while URLS is a dictionary containing the AVNU API url and endpoints. AVNU is defined as:

class AVNU(Exchange):
    """AVNU exchange."""

    def __init__(self, account: Account, balance: Symbol) -> None:
        self._account = account
        # Fetch available dexes
        response = requests.get(f"{URLS['base']}/{URLS['sources']}")
        available_dexes = [
            dex["name"] for dex in response.json() if dex["type"] == "DEX"
        ]
        available_dexes.append("lastPrice")

        self._last_prices = {dex: Decimal("0") for dex in available_dexes}
        self._receiver_queue = asyncio.Queue()

        asyncio.create_task(self._fetch_balance(balance))

As you can see, the class starts by fetching the exchanges supported by AVNU and initialising self._last_prices. The request is synchronous because:

• __init__ cannot be asynchronous. This is a limitation of Python, the class initialiser cannot be marked as async
• even if it is a blocking request, it is fine to make this call while initialising

Finally, it creates the coroutine for _fetch_balance which fetches the initial account balances. The method is defined as:

    async def _fetch_balance(self, symbol: typing.Optional[Symbol] = None):
        tokens = [ETH] if symbol is None else [symbol.base, symbol.quote]

        for token in tokens:
            logger.debug(f"Fetching {token.address} balance")
            amount = await self._account.get_balance(token.address)
            await self._receiver_queue.put(
                Wallet({"amount": amount}, token, Decimal(amount) / 10**token.decimals)
            )

Let’s see how subscribe_ticker is implemented:

    async def subscribe_ticker(self, symbol: Symbol, amount: decimal.Decimal, **_):
        asyncio.create_task(self._handle_quotes(symbol, amount))
        
    async def _handle_quotes(
        self,
        symbol: Symbol,
        amount: decimal.Decimal,
    ):
        url = f"{URLS['base']}/{URLS['quotes']}"
        params_sell = {
            "sellAmount": hex(amount * 10 ** int(symbol.base.decimals)),
            "sellTokenAddress": symbol.base.address,
            "buyTokenAddress": symbol.quote.address,
        }
        params_buy = {
            "sellAmount": hex(amount * 10 ** int(symbol.base.decimals)),
            "sellTokenAddress": symbol.base.address,
            "buyTokenAddress": symbol.quote.address,
        }

        async with aiohttp.ClientSession() as session:
            while True:
                logger.debug("Fetching quotes")
                # We need to make two requests because a call to `/quotes` also
                # sets the swap order. Therefore, if we want to buy `base` from
                # `quote` we need the opposite ordeer (and a new `quoteId`)
                resp_sell, resp_buy = await asyncio.gather(
                    session.get(url, params=params_sell),
                    session.get(url, params=params_buy),
                )
                entries_sell, entries_buy = await asyncio.gather(
                    resp_sell.json(), resp_buy.json()
                )

                # `entries_sell` is a list with one element
                entry = entries_sell[0]

                quote_amount = decimal.Decimal(int(entry["buyAmount"], 16))
                quote_amount = quote_amount / (10**symbol.quote.decimals)
                price = quote_amount / amount

                if self._last_prices["lastPrice"] != price:
                    self._last_prices["lastPrice"] = price

                    entry["sellId"] = entry["quoteId"]
                    entry["buyId"] = entries_buy[0]["quoteId"]
                    ticker = Ticker(entry, price, amount, price, amount)

                    await self._receiver_queue.put(ticker)

                await asyncio.sleep(1)

The methods are simple. The first creates a coroutine to _handle_quotes, while the second does the actual work . Rather than using the prices endpoint, quotes is used. Not only to get the best price, but to ask to AVNU to prepare the swap for us. The endpoint builds a route to get the best swap and assigns a unique quoteId to each request. The id can be used later when the class sends the request to build the swap transaction. Two requests are made because it is not possible to change the order of the buy/sell token addresses while the transaction is being built.

The method continues with parsing the outputs, calculating the price, storing the quoteId and adding the Ticker to the receiver_queue.

The last methods are for sending the orders:

    async def buy_market_order(
        self,
        symbol: Symbol,
        amount: Decimal,
        ticker: Ticker,
        slippage: Decimal = Decimal("0.01"),
    ):
        """Insert a buy market order."""
        url = f"{URLS['base']}/{URLS['build']}"

        payload = {
            "slippage": str(slippage),
            "takerAddress": hex(self._account.address),
            "quoteId": ticker.raw["quoteId"],
            "includeApprove": True,
        }

        # FIXME: Share session with `handle_prices_quotes`
        async with aiohttp.ClientSession() as session:
            response = await session.post(url, json=payload)
            resp_calls = await response.json()

            # `resp_calls` is a dict containing `chainId` and `calls` keys. A
            # call is a dict containing `contractAddress, entrypoint` and
            # `calldata`. Values are hexed or string (the selector)
            calls = [
                Call(
                    int(call["contractAddress"], 16),
                    get_selector_from_name(call["entrypoint"]),
                    [int(call_value, 16) for call_value in call["calldata"]],
                )
                for call in resp_calls["calls"]
            ]

            logger.info("sending order")
            transaction_hash = await self._account.execute_v3(calls, auto_estimate=True)
            await self._wait_txn(
                transaction_hash.transaction_hash, symbol, amount, ticker, "buy"
            )

At this stage build endpoint is used to build the swap transaction. In the future I would like to remove AVNU and swap directly on the DEXes.

The quoteId is used to request AVNU to build the swap and return the transaction calldata. The response is prepared for Starknetjs and not for the python version. contractAddress and calldata need to be converted to int while entrypoint is a human readable string and needs to be converted into the selector (i.e. from approve to 0x04270219d365d6b017231b52e92b3fb5d7c8378b05e9abc97724537a80e93b0f).

The final steps are to broadcast the transaction and wait for the sequencer to validate it. Once validated, the Order message is added to the receiver_queue and the new balances are fetched:

    async def _wait_txn(
        self,
        transaction_hash: int,
        symbol: Symbol,
        amount: Decimal,
        ticker: Ticker,
        side: str,
    ):
        """Wait until transaction is confirmed. After create the order and update the balance."""
        await self._account.client.wait_for_tx(transaction_hash)

        order = Order(
            {"transaction_hash": transaction_hash},
            symbol,
            amount,
            # NOTE: `ask` and `bid` are the same
            ticker.ask,
            side,
        )
        await self._receiver_queue.put(order)

        asyncio.create_task(self._fetch_balance(symbol))

Arbitrage class

In the previous sections, I defined the Exchange interface and described how I implemented Binance and AVNU. In this section, I will describe the last component of the bot: the Arbitrage class, the most important one. It is responsible for determining if there is a profit and attempting to take it.

class Arbitrage:
    def __init__(
        self,
        exchanges: list[Exchange],
        symbol: Symbol,
        spread_threshold: Decimal,
        trade_amount: Decimal,
        min_trade_amount: Decimal,
    ):
        self._exchanges = exchanges
        self._symbol = symbol
        self._threshold = spread_threshold
        self._queue: asyncio.Queue[tuple[Any, Exchange]] = asyncio.Queue()
        self._trade_amount = trade_amount
        self._min_trade_amount = min_trade_amount

It requires a list of Exchange, the Symbol on which to trade, spread_threshold which indicates the profit threshold, and trade_amount and min_trade_amount which define the trading range.

The more observant will have noticed that Arbitrage requires a list of Exchange rather than a single CEX and DEX. The class is capable of iterate over multiple exchanges and it does not care whether the exchanges are CEXes or DEXes.

Arbitrage defines two utility functions, _initialize and _merge_queues

    async def _initialize(self):
        for exchange in self._exchanges:
            await exchange.subscribe_ticker(
                self._symbol, amount=int(self._trade_amount)
            )
            queue = exchange.receiver_queue()
            asyncio.create_task(self._merge_queues(queue, exchange))

As the name suggests, _initialize is called to subscribe to the tickers and to merge the queues into a single one by calling _merge_queues:

    async def _merge_queues(self, queue: asyncio.Queue, ex: Exchange):
        while True:
            val = await queue.get()
            await self._queue.put((val, ex))

The functions for calculating the spread and the amount to be swapped are quite simple:

    def _calculate_spread(self, ask: Decimal, bid: Decimal) -> Decimal:
        numerator = bid - ask
        return numerator / bid
        
    def _calculate_amount(
        self,
        ask: Ticker,
        bid: Ticker,
        wallet_ask: Decimal,
        wallet_bid: Decimal,
    ) -> Decimal:
        """Return the amount we can trade."""
        amount = min(self._trade_amount, ask.ask_amount, bid.bid_amount)
        # Check wallets have enough balance
        # NOTE: `wallet_ask` is the quote token balance. Convert in base
        amount = min(amount, wallet_bid, wallet_ask * ask.ask)
        return amount

As you can see, _calculate_amount, follows the formula described in Step 3: Sending orders. Remember that wallet_ask is the total amount of available quote tokens. A base conversion is needed to compare them.

Finally, the last and the most interesting method of the class. The method waits for messages from the exchanges and it decides to send the order if there is a profit.

    async def run(self):
        # Subscribe to the tickers and merge the queues
        await self._initialize()

        best_bid = Ticker({}, Decimal("-INFINITY"), Decimal(0), Decimal(0), Decimal(0))
        best_ask = Ticker({}, Decimal(0), Decimal(0), Decimal("INFINITY"), Decimal(0))
        exchange_ask = exchange_bid = self._exchanges[0]

        wallets = {
            exchange: {
                self._symbol.base.name: Wallet.empty(self._symbol.base),
                self._symbol.quote.name: Wallet.empty(self._symbol.quote),
            }
            for exchange in self._exchanges
        }

        while True:
            msg, exchange = await self._queue.get()

            match msg:
                case Wallet():
                    wallets[exchange][msg.token.name] = msg
                case Order():
                    logger.info(
                        f"{exchange} order executed. {msg.side}: {msg.amount} at {msg.price}"
                    )
                case Ticker():
                    # We want to buy at the lowest price
                    if msg.ask <= best_ask.ask:
                        best_ask = msg
                        exchange_ask = exchange

                    # We want to sell at the highest price
                    if msg.bid >= best_bid.bid:
                        best_bid = msg
                        exchange_bid = exchange

                    # Same exchange, skip
                    if exchange_bid == exchange_ask:
                        continue

                    spread = self._calculate_spread(
                        best_ask.ask,
                        best_bid.bid,
                    )
                    ba = dataclasses.replace(best_ask)
                    ba.raw = exchange_ask
                    bb = dataclasses.replace(best_bid)
                    bb.raw = exchange_bid

                    logger.debug(
                        f"spread: {spread} best ask: {ba.ask} best bid: {bb.bid}"
                    )
                    if spread >= self._threshold:
                        logger.info(f"{spread} above the threshdold")
                        amount = self._calculate_amount(
                            best_ask,
                            best_bid,
                            wallets[exchange_ask][self._symbol.base.name].amount,
                            wallets[exchange_bid][self._symbol.quote.name].amount,
                        )

                        if amount < self._min_trade_amount:
                            continue

                        logger.debug(
                            f"{exchange_ask}: buy: {amount} price: {best_ask.ask}"
                        )
                        logger.debug(
                            f"{exchange_bid}: sell: {amount} price: {best_bid.bid}"
                        )

                        await asyncio.gather(
                            exchange_ask.buy_market_order(
                                self._symbol, amount, best_ask
                            ),
                            exchange_bid.sell_market_order(
                                self._symbol, amount, best_bid
                            ),
                        )

Almost ready

The bot is almost ready. What is missing is loading the config and the main function. The config is in an .env file and is read using the dotenv library. Config is responsible for loading and validating the configuration.

class Config:
    REQUIRED_KEYS = {
        "BASE_ADDR",
        "BASE_DECIMALS",
        "BASE",
        "QUOTE_ADDR",
        "QUOTE_DECIMALS",
        "QUOTE",
        "API_KEY",
        "SECRET_KEY",
        "SIGNER_KEY",
        "NODE_URL",
        "SPREAD_THRESHOLD",
        "MAX_AMOUNT_TRADE",
    }

    def __init__(self, config: dict) -> None:
        for key in self.REQUIRED_KEYS:
            if key not in config:
                raise ValidationError(f"Missing `{key}`")

        self.base = Token(
            config["BASE"], config["BASE_ADDR"], int(config["BASE_DECIMALS"])
        )
        self.quote = Token(
            config["QUOTE"], config["QUOTE_ADDR"], int(config["QUOTE_DECIMALS"])
        )
        self.symbol = Symbol(self.base, self.quote)

        self.api_key = config["API_KEY"]
        self.secret_key = config["SECRET_KEY"]

        self.node_url = config["NODE_URL"]
        self.account_address = config["ACCOUNT_ADDRESS"]
        self.signer_key = config["SIGNER_KEY"]

        self.spread_threshold = Decimal(config["SPREAD_THRESHOLD"])
        self.max_amount_trade = Decimal(config["MAX_AMOUNT_TRADE"])
        self.min_amount_trade = Decimal(config["MIN_AMOUNT_TRADE"])

Finally, the main function:

async def main():
    # Bot settings
    logger = logging.getLogger("bot")
    logger.setLevel(logging.DEBUG)
    logger.addHandler(RichHandler())
    # Load config
    config = {**dotenv_values(".env"), **os.environ}
    config = Config(config)

    # Exchange initialisation
    logger.debug("Connecting to starknet...")
    chain = Starknet(config.node_url)
    account = chain.get_account(config.account_address, config.signer_key)
    logger.debug("Connecting to AVNU...")
    avnu = AVNU(account, config.symbol)

    logger.debug("Connecting to binance...")
    binance = Binance(config.api_key, config.secret_key)

    # Run bot
    logger.debug("All setup, running bot...")
    bot = Arbitrage(
        [binance, avnu],
        config.symbol,
        config.spread_threshold,
        config.max_amount_trade,
        config.min_amount_trade,
    )
    await bot.run()


asyncio.run(main())

Future improvements

The bot explained is very basic and there is room for many improvements. Here are a few:

• Clean up the code and build more robust components. For example, if a coroutine crashes, the bot continues to work without that coroutine. This is a big problem if the crashed coroutine is the function that parses the tickers.
• Remove ccxt.While the library is great for prototyping, it introduces a lot of unnecessary requests and is slow. It can be replaced by calling the Binance API.
• Replace AVNU. Although is great for its API, the bot swaps at the best rates while it should swap between the best and worst prices.
• Avoid using auto_estimate when signing the transaction.

As I mentioned in my previous articles of the series there is a group to discuss MEV on Starknet. If you are interested in the argument, want to discuss or have some interesting information, you can find me there. Feel free to join the group.

This article is part of a series on building an arbitrage bot between centralised (CEX) and decentralised (DEX) exchanges on Starknet.

In this article I will cover the functionalities of the bot, explaining its functionality, how it works, the library used, and the underlying mathematics.

• Building an Arbitrage Bot on Starknet: Part 0 - Understanding the Fundamentals
• Building an Arbitrage Bot on Starknet: Part 1 - The basics
• Building an Arbitrage Bot on Starknet: Part 2 - The bot
• Building an Arbitrage Bot on Starknet: Part 3 - Conclusions

What is an arbitrage bot?

There are many resources describing what an arbitrageur is and what arbitrage bots do. However, I will try to give my own perspective on the subject.

In a market where many different entities offer services for the exchange of assets, these assets may have different prices, even if they are the same. This is because each service operates separately and users determine the price through their behaviour.

Popular services attract more users, which means more liquidity and a more stable price. On the other hand, services with less liquidity may have a more volatile price. Fewer trades also update the price more slowly. These differences are known as market inefficiencies. An arbitrageur is an entity that tries to make a profit from these inefficiencies by buying the asset at a lower price and selling it at a higher price.

Arbitrageurs do not exist only in the crypto space but are also a fundamental part of traditional finance.

CEX vs DEX

In the blockchain space, we have two main types of exchanges: centralised and decentralised. A centralised exchange (CEX) is basically a trading platform run by one company that lets people buy and sell crypto assets. They usually have an order book where users can enter their orders and the engine will execute them.

A decentralised exchange (DEX) lives on a blockchain in the form of smart contracts, sometimes regulated by a DAO. They are a backbone of decentralised finance (DeFi). Automated Market Makers (AMMs) are a type of DEX that use a mathematical formula to determine the price. Unlike CEXes, AMMs consist of liquidity pools, typically with two or three crypto assets. The most popular liquidity pools are Uniswap V2 that follows the formula $$x * y = k$$ or Uniswap V3 with concentrated liquidity.

DeFi products are not just AMMs. The most famous order book product is probably DyDx.

Risks

An arbitrage bot has two big risks:

• Token exposure or inventory risk
• Failed swaps

Token exposure

The risk is not related to the bot, but with the tokens themselves. An arbitrage bot is not interested in the value of the token it is trading. The problem lies in the value of the tokens, if any go down in value, the profits will also go down, reducing the profitability of the bot. The only solution is not to trade very risky pairs or to sell the profits immediately.

Failed swaps

Trades are not atomic, so one of them may be executed at a difference price or fail. In addition, centralised exchanges may also have hidden orders or may not execute (unless the market type is not selected).

For DEXes it is even more complicated:

• Bots must constantly monitor the mempool to see if there are trades that change the price pool, but with private or encrypted mempool this is no longer possible.
• If there are many trades before ours, it may fail because the swapped price is lower than minPrice (or the slippage is higher than the one set).

The solution is to build an atomic arbitrage bot that takes the opportunity opportunity in a single transaction and if it fails, the swaps are not executed. Fortunately, as I wrote in the previous article, Starknet does not have these problems.

Terminology

Before describing how the bot works, let’s look at some definitions. If you are already familiar with them skip this section.

• Token: a representation of an asset that has been tokenised, i.e. USDC or WBTC
• Pair or trading pair: describes two tokens that are traded on an exchange, i.e. BTC and USD
• Base: the first token that appears in a pair
• Quote: the second token that appears in a pair. Also known as counter currency
• Symbol: a unique combination of letters and numbers that represents a pair Usually, consists of the base and quote in the pair, e.g BTC/USD
• Spread: a difference or gap between two prices
• Bid: the highest price a buyer will pay to buy a token
• Ask: the lowest price a seller will pay to sell a token

How the bot works

The bot consists of several steps:

• it fetches the tickers from both exchanges
• it calculates the spread between the two tickers
• if there is a profit, it tries to capture it by placing a buy/sell market order on the exchanges.

Here is a sequence diagram showing an interaction of the bot

Step 1: Getting data

The bot needs to get the token prices from the exchange all the time. With centralised exchanges, there is an endpoint called ticker that returns a message like this:

{
  "symbol": "BTCUSDT",
  "priceChange": "-1067.94000000",
  "priceChangePercent": "-1.619",
  "weightedAvgPrice": "66057.25597271",
  "prevClosePrice": "65957.82000000",
  "lastPrice": "64889.88000000",
  "lastQty": "0.00109000",
  "bidPrice": "64889.88000000",
  "bidQty": "3.24493000",
  "askPrice": "64889.89000000",
  "askQty": "0.16658000",
  "openPrice": "65957.82000000",
  "highPrice": "66849.24000000",
  "lowPrice": "64632.00000000",
  "volume": "20494.74928000",
  "quoteVolume": "1353826899.28544630",
  "openTime": 1722370827899,
  "closeTime": 1722457227899,
  "firstId": 3709477216,
  "lastId": 3711014970,
  "count": 1537755
}

We are only interested in the price and quantity of the bid and ask (bidPrice, bidQty, askPrice, askQty from the example).

AMMs work in a different way and they do not produce a ticker, so we need to simulate it. There are two ways we can get the price from a pool:

• Ask the price directly from the pool (i.e. Uniswap V3 pools) or calculate it from the liquidity (i.e. Uniswap V2 pools).
• Simulate the trade with a specific amount and then calculate the price from the amounts.

The latter is better because it is more precise. In an AMM the amount you are swapping affects the price. The higher the quantity, the more you unbalance the pool (and the less you get), and the more the price changes. Thus, we can ask to the pool to simulate a trade, calculate the price and then simulate the ticker:

  initial_amount = 1 * 10**18 # amount that we are interested to swap
  amount_out = pool.swap(initial_amount) # the pool returns the swapped amount
  price = amount_out / initial_amount
  # build a fake ticker message
  ticker = {
    "bidPrice": price,
    "bidAmont": initial_amount,
    "askPrice": price,
    "askAmont": initial_amoun,
  }

A fundamental concept to know is that prices are ALWAYS IN BASE. For this reason, in the snippet, bid and ask are the same.

Once the tickers are fetched, we need to find the best bid and ask among them. An important thing to understand is that we are trading against these values. This mean that we will sell at the best bid and buy at the best ask.

If we only trade on two exchanges all we need to do is compare the two tickers and get the best amounts. However, with multiple exchanges, the formula is more general. Let $$N$$ be the number of exchanges we use, $$bid_i,\ ask_i$$ $$i \in N$$ the bid and ask of exchange $$i$$. The $$bestBid$$ and $$bestAsk$$ are defined as

Why not asking the order book?

Instead of the ticker, the order book gives us much more information about how much liquidity there is in the pair. This information can be useful in later steps when the bot has to decide on the size of the orders.

The decision to use the ticker is because the logic around order sizing is simple: given a predefined amount, the bot tries to create an order with that amount. In addition, when a trade is made, the ticker is updated and, if the opportunity is still there, we can continue with our swaps.

Step 2: Calculate the spread

Once the bot has fetched best bid and ask, it can proceed to calculate the spreads. As I wrote in terminology, the spread is the difference between the bid/ask of one exchange and the ask/bid of the other. Let’s say $$A$$ and $$B$$ are two different exchanges, the spread is:

Two spreads are calculated, one for each bid/ask side. If one of the spreads is $$> 0$$ or greater than a threshold, the bot moves to the next step: sending the orders.

Step 3: Sending orders

At this point we know there is an opportunity with and there is a profit to be made. We smell 💵 💵 money 💵 💵. The bot needs to send the two orders to the exchanges.

For the centralised exchange, we will send a MARKET. For the DEX, we will build a transaction with the swap to do. As I mentioned in my previous article, in Starknet transactions are executed immediately so we do not need to wait for the new block to be mined.

But what is the order size? The strategy is very simple and consists of to trade a pre-defined amount. However, the trade may not be successful for two reasons:

• Bid or ask amounts may be lower
• Accounts or wallet may have less tokens

We can solve the problem with the following formula. Let $$k$$ be the amount we want to swap, $$ask$$ the price of ask, $$amount_{ask}$$ the maximum amount we can trade at $$ask$$ price and $$wallet_{ask}$$ are the funds available on the exchange. $$bid$$, $$amount_{bid}$$ and $$wallet_{bid}$$ are defined in the same way. Therefore, the maximum value $$A$$ we can trade is:

Once the value to swap has been determined, the bot sends the orders, waits for the response and starts again from the first step.

Project setup

I chose Python to develop the bot. For this type of bot, a faster language like Rust is not needed because the bottleneck is the latency of the exchanges. Because Python is a high-level language, we can develop much faster, and you do not have to worry about things like floating-point notation or big numbers.

As a dependency manager I use poetry

$ poetry new stark-arbitrage

And install the dependencies

$ poetry add python-dotenv rich ccxt starknet-py

Note that starknet-py requires additional external packages. See Installation for more information. The library is required to connect to Starknet and retrieve account balances, and to sing and broadcast the transaction.

The centralised exchange that we will use is Binance through ccxt. For the decentralised exchange we will use AVNU to get the prices and to swap.

Ccxt

Ccxt (CryptoCurrency eXchange Trading Library) is a cryptocurrency trading library with support for many exchange markets, including Binance. The library is multi-language (JS, Python, PHP and C#) and provides a unified API across exchanges.

We are interested in the pro version because it offers support for websocket.

AVNU

AVNU is a decentralised exchange protocol (a dex aggregator) designed to provide the best execution. The service has a nice API that returns the price and a simulated price (in this case it provides the best route for our trade) for all supported exchanges.

We will use AVNU to get the pool prices and ask it to create the transaction, which we will then sign and broadcast.

Conclusions

This article ends the long initial explanation on how Starknet and an arbitrage bot works. In the next article I will start to explain how to develop the steps described in the previous sections.

As I mentioned in my first article of the series there is a group to discuss MEV on Starknet. If you are interested in the argument, want to discuss or have some interesting information, you can find me there. Feel free to join the group.

This article is part of a series on building an arbitrage bot between centralised (CEX) and decentralised (DEX) exchanges on Starknet.

We’ll explore Starknet’s architecture, focusing on elements crucial for arbitrage: the sequencer system and transaction lifecycle.

• Building an Arbitrage Bot on Starknet: Part 0 - Understanding the Fundamentals
• Building an Arbitrage Bot on Starknet: Part 1 - The basics
• Building an Arbitrage Bot on Starknet: Part 2 - The bot
• Building an Arbitrage Bot on Starknet: Part 3 - Conclusions

What is Starknet

Starknet is a permissionless Layer 2 solution for Ethereum that uses zero-knowledge rollups (zk-rollups) to enhance scalability and security. Unlike optimistic rollups, Starknet’s approach allows for faster transaction finality and enhanced security. Key features include:

• Cryptographic proofs for each block.
• Account abstraction.
• STARK technology for proof generation.

Here’s how Starknet works:

1. The chain produces a cryptographic proof for each block.
2. These proofs are then aggregated and published on Ethereum.
3. Anyone can take the proof and verify it independently.

This process ensures transparency and security while leveraging the scalability benefits of Layer 2 technology. Starknet consists of two specialised actors: sequencers and provers. In this article, we will focus on the sequencers.

The sequencer

Sequencers are the backbone of the Starknet network, similar to Ethereum’s validators. While validators ensure network security, sequencers in Starknet provide transaction capacity. They serve as the access point for users to interact with the chain, necessitating high reliability and availability.

Sequencers play a crucial role in Starknet’s operation, following a systematic approach to transaction processing:

Sequencers follow this systematic method of processing transactions:

1. Sequencing: collecting transactions from users and ordering them.
2. Executing: processing the transactions
3. Batching: executed transactions are grouped into batches
4. Block production: batches are grouped into blocks

Once sequencers complete these steps, provers take over to generate cryptographic proofs of these blocks, which are then submitted to Ethereum for final verification.

Although there are plans to decentralise Starknet (here and here), the current centralised sequencer model provides us with two interesting features: efficient transaction ordering and predictable transaction execution.

First-Come-First-Serve (FCFS) model

The sequencer operates on a First-Come-First-Serve (FCFS) basis, processing transactions as they are received. This eliminates the need for mempool monitoring and gas price competition, as the first transaction sent is the first to be processed. In Starknet, it is all about being first.

Understanding the sequencer’s role and the FCFS model is crucial for arbitrage bot developers. The FCFS model shifts the focus from gas price optimization to being the first to submit a transaction, fundamentally changing arbitrage strategies on Starknet compared to other blockchain networks.

Block production limits

Starknet implements block production limits to ensure network stability and efficiency. Blocks are typically created every ~6 minutes or when specific thresholds are met. For the most current information on these limits, refer to the official Starknet Current Limits guide.

Transaction lifecycle

To better understand the execution of transactions within the sequencer, let’s examine the transaction lifecycle. There are three different types of transactions:

• INVOKE used for calling an existing function in a contract
• DECLARE used for declaring new contract classes and activating new contract instances
• DEPLOY_ACCOUNT used for deploying new account contracts in smart wallets.

At a high level, the Starknet transaction flow is as follows:

1. Transaction submission
   • User signs the transaction and sends it to a node.
   • The node broadcast the transaction
   • The transaction is marked as RECEIVED and added to the mempool
2. Mempool validation
   • Mempool performs a validation on the transaction (similar to the Ethereum’s signature)
   • __validate__ function is called (or __validate_declare__, __validate_deploy__ for respective transaction type) to ensure that the transaction is valid
   • Invalid transaction are discarded
3. Sequencer validation
   • The sequencer performs a validation on the transaction before the execution to ensure the transaction is still valid.
   • Invalid transaction are discarded
4. Execution:
   • Valid transactions are executed
   • Transactions are inserted into the pending block and state is updated
   • Transaction is marked as ACCEPTED ON L2. If a transaction fails during execution, the status is set to REVERTED. Otherwise, the status is SUCCESSED.
5. Proof generation and verification: The prover computes the proof and sends it to the L1 verifier.

Combining the two: sequencer and transaction lifecycle

Why are these two aspects important to our arbitrage bot? While I was writing this article explaining their importance, Starknet made a tweet that answer the question perfetcly.

The answer is in “transaction finality extremely fast”. Although the tweet uses the wrong term, in Starknet finality occurs when the prover generates the block proof and publishes it on Ethereum. Validity, on the other hand, is when the sequencer builds the block. The ~2 seconds can be seen as fast validity.

But how is this possible? How can we have “finality” in ~2 seconds when a block is generated every ~6 minutes? The answer lies in stage 4 of the transaction lifecycle or point 2 of the sequencer. After execution, the sequencer updates the state of the chain. Because the sequencer is centralised and operate on a FCFS basis, it can ensure that the transaction order is preserved, allowing the chain state to be updated immediately. Block creation is a formality and only useful for the provers.

This is very important for us because it means we can completely ignore the mempool. We do not need to check it for pending swaps since the chain state has already been updated. But Matteo, are you 100% sure? Let’s follow the motto “Don’t trust, verify” and check if what I wrote is true.

I will illustrate this with two examples: first, the creation of an account, and second, a simple swap on Ekubo.

Account creation

Once we have created the signer and generated the account we need to fund it for deployment. The address of the account is 0x067902aa3c1c65b35995d5bfe04f1fe949795f3510da71ba86bbf0e134402645.

> starkli balance 0x067902aa3c1c65b35995d5bfe04f1fe949795f3510da71ba86bbf0e134402645 --network sepolia
0.000000000000000000

After the faucet

> starkli balance 0x067902aa3c1c65b35995d5bfe04f1fe949795f3510da71ba86bbf0e134402645 --network sepolia
0.025000000000000000

And deploy

> starkli account deploy account3.json --keystore keystore3.json
The estimated account deployment fee is 0.000001863236340482 ETH. However, to avoid failure, fund at least:
    0.000002794854510723 ETH
to the following address:
    0x067902aa3c1c65b35995d5bfe04f1fe949795f3510da71ba86bbf0e134402645
Press [ENTER] once you've funded the address.
Account deployment transaction: 0x06e1e2e16e28bdfe7d7faa6d49ebe080865e7b0b30e579322bf9aaf8656df4c3
Waiting for transaction 0x06e1e2e16e28bdfe7d7faa6d49ebe080865e7b0b30e579322bf9aaf8656df4c3 to confirm. If this process is interrupted, you will need to run `starkli account fetch` to update the account file.
Transaction not confirmed yet...
Transaction not confirmed yet...
Transaction 0x06e1e2e16e28bdfe7d7faa6d49ebe080865e7b0b30e579322bf9aaf8656df4c3 confirmed

>starkli balance 0x067902aa3c1c65b35995d5bfe04f1fe949795f3510da71ba86bbf0e134402645 --network sepolia
0.024998136763659518

From Voyager you can see that the transactions have been executed, the chain status has been updated and they are in the pending block.

Swap

For this example we swap STRK for ETH using Ekubo. As you can see before the swap we have 20.0 STRK tokens.

Swap

Finally, the balance update

I found this group to discuss MEV on Starknet. If you are interested in the argument, want to discuss or have some interesting information, you can find me there. Feel free to join the group.

References

• ZK-STARK: https://starkware.co/stark/ (July 2024)
• Sequencers: https://docs.starknet.io/architecture-and-concepts/network-architecture/starknet-architecture-overview/#sequencers (July 2024)
• Transaction lifecycle: https://docs.starknet.io/architecture-and-concepts/network-architecture/transaction-life-cycle/ (July 2024)

I am a computer scientist currently working in the blockchain space and interested in blockchain, AI and zero knowledge proofs.