Cryptocurrency exchange market data feed handler

starsStars 847
forksForks 274
watchersWatchers 847
current-versionCurrent version 2019.1.2
total-releasesTotal releases 2
open_issues_countOpen issues 13
dateFirst release 2017-07-25
dateLatest release 2019-01-27
updateLast update 2020-12-23

BitcoinExchangeFH - Cryptocurrency exchange market data feed handler

BitcoinExchangeFH is a slim application to record the price depth and trades in various exchanges. You can set it up quickly and record the all the exchange data in a few minutes!

Users can

  1. Streaming market data to a target application (via ZeroMQ)
  2. Recording market data for backtesting and analysis.
  3. Recording market data to a in-memory database and other applications can quickly access to it.
  4. Customize the project for trading use.



Supported exchanges

All exchanges supported by ccxt. Currently more than 130 exchanges are supported.

Websocket feeds of the following exchanges are supported by cryptofeed

  • Bitfinex

  • Coinbase

  • Poloniex

  • Gemini

  • HitBTC

  • Bitstamp

  • BitMEX

  • Kraken

  • Binance

  • EXX

  • Huobi

  • OKCoin

  • OKEx

If the exchange is not supported with websocket API feed, it will automatically fall into using its REST API feed.

Supported database/channel

  • RDMBS (e.g. sqlite, MySQL, PostgreSQL)

  • ZeroMQ

  • Kdb+ (Coming soon)

Getting started

pip install bitcoinexchangefh
bitcoinexchangefh --configuration example/configuration.yaml


The configuration follows YAML syntax and contains two sections

  • subscriptions

  • handlers


Subscription section specifies the exchange and instruments to subscribe.

The first key is the exchange name and then follows the exchange details,

  • instruments

  • number of depth (default is 5 if not specified)

For example,

            - XRP/BTC
            - BCH/BTC
            - ETH/BTC
        depth: 10


After receiving the order book or trade update, each handler is updated. For example, for SQL database handler, it is updated with the corresponding SQl statements.

For example,

        connection: "sqlite://"

SQL handler

The following settings can be customized

Parameter Description
connection Database connection string required by SQLAlchemy
is_rotate Boolean indicating whether to rotate to record the table.
rotate_frequency String in format same as strftime and strptime

ZeroMQ handler

The feed handler acts as a subscriber.

The following settings can be customized

Parameter Description
connection Connection


You can first create a directory .data and run the command

$ bitcoinexchangefh --configuration example/configuration.yaml 
2020-08-07 23:33:32,110 INFO Loading runner
2020-08-07 23:33:32,110 INFO Creating handler sql
2020-08-07 23:33:32,205 INFO Loading handler SqlHandler
2020-08-07 23:33:32,710 INFO Loading exchange Binance
2020-08-07 23:33:34,062 INFO Loading exchange Bitmex
2020-08-07 23:33:40,203 INFO Start running the feed handler
2020-08-07 23:33:40,203 INFO Running handler sql
2020-08-07 23:33:40,210 INFO Running exchange Binance
2020-08-07 23:33:40,215 INFO Running SqlHandler
2020-08-07 23:33:40,219 INFO Running exchange Bitmex
2020-08-07 23:33:40,231 INFO Joining all the processes
2020-08-07 23:33:40,232 INFO Creating table binance_ethbtc_order
2020-08-07 23:33:40,263 INFO Created table binance_ethbtc_order
2020-08-07 23:33:40,265 INFO Creating table binance_xrpbtc_order
2020-08-07 23:33:40,294 INFO Created table binance_xrpbtc_order
2020-08-07 23:33:40,297 INFO Creating table bitmex_ethusd_order
2020-08-07 23:33:40,322 INFO Created table bitmex_ethusd_order

After a while, the data is streamed into the file .data/order_book.db. You can terminate the process and the database contains the following tables.

$ python -c "import sqlite3; print(sqlite3.connect('.data/order_book.db').execute('SELECT name from sqlite_master where type= \"table\"').fetchall())"
[('binance_ethbtc_order',), ('binance_xrpbtc_order',), ('bitmex_ethusd_order',)]


You can first look up to the page FAQ. For more inquiries, you can either leave it in issues or drop me an email. I will get you back as soon as possible.


The application is compatible with version higher or equal to python 3.0.


Always welcome for any contribution. Please fork the project, make the changes, and submit the merge request. :)

For any questions and comment, please feel free to contact me through email (gavincyi at gmail)

Your comment will be a huge contribution to the project!


If you are not satisified with python performance, you can contact me to discuss migrating the project into other languages, e.g. C++.