QuantRocket logo

Disclaimer

Data Collection - Indexes

Though most of our features will come from the Sharadar price and fundamental data, we also wish to add some additional features that reflect the broad market. Specifically, we will include features relating to the S&P 500, the VIX, and the NYSE TRIN (aka Arms Index, a breadth measure).

This data will come from Interactive Brokers.

First, start IB Gateway:

In [1]:
from quantrocket.ibg import start_gateways
start_gateways(wait=True)
Out[1]:
{'ibg1': {'status': 'running'}}

Collect listings

Next we collect the listings from IBKR. (For the S&P 500, we collect the SPY ETF because IBKR provides deeper history for SPY@ARCA than SPX@CBOE.)

In [2]:
from quantrocket.master import collect_ibkr_listings
collect_ibkr_listings(exchanges="ARCA", symbols="SPY", sec_types="ETF")
collect_ibkr_listings(exchanges="CBOE", symbols="VIX", sec_types="IND")
collect_ibkr_listings(exchanges="NYSE", symbols="TRIN-NYSE", sec_types="IND")
Out[2]:
{'status': 'the IBKR listing details will be collected asynchronously'}

Monitor flightlog for the completion messages:

quantrocket.master: INFO Collecting ARCA ETF listings from IBKR website (SPY only)
quantrocket.master: INFO Requesting details for 1 ARCA listings found on IBKR website
quantrocket.master: INFO Saved 1 ARCA listings to securities master database
quantrocket.master: INFO Collecting CBOE IND listings from IBKR website (VIX only)
quantrocket.master: INFO Requesting details for 1 CBOE listings found on IBKR website
quantrocket.master: INFO Saved 1 CBOE listings to securities master database
quantrocket.master: INFO Collecting NYSE IND listings from IBKR website (TRIN-NYSE only)
quantrocket.master: INFO Requesting details for 1 NYSE listings found on IBKR website
quantrocket.master: INFO Saved 1 NYSE listings to securities master database

Lookup Sids

Look up the Sids for the various instruments:

In [3]:
from quantrocket.master import download_master_file
download_master_file("indices.csv", exchanges=["CBOE", "NYSE", "ARCA"], symbols=["SPY", "VIX", "TRIN-NYSE"], sec_types=["IND","ETF"], vendors="ibkr")
In [5]:
import pandas as pd
indices = pd.read_csv("indices.csv")

indices[["Sid", "Symbol", "Name", "Exchange"]]
Out[5]:
SidSymbolNameExchange
0FIBBG000BDTBL9SPYSPDR S&P 500 ETF TRUSTARCX
1IB13455763VIXCBOE Volatility IndexXCBO
2IB26718743TRIN-NYSENYSE TRIN (OR ARMS) INDEXXNYS

Collect historical data

Next, we create a database for collecting 1-day bars for the indexes:

In [6]:
from quantrocket.history import create_ibkr_db
create_ibkr_db("market-1d", 
              sids=[
                  "FIBBG000BDTBL9",
                  "IB13455763", 
                  "IB26718743",
              ], 
              bar_size="1 day")
Out[6]:
{'status': 'successfully created quantrocket.v2.history.market-1d.sqlite'}

Then collect the data:

In [7]:
from quantrocket.history import collect_history
collect_history("market-1d")
Out[7]:
{'status': 'the historical data will be collected asynchronously'}

Monitor flightlog for completion:

quantrocket.history: INFO [market-1d] Collecting history from IBKR for 3 securities in market-1d
quantrocket.history: INFO [market-1d] Saved 13302 total records for 3 total securities to quantrocket.v2.history.market-1d.sqlite

Next Up

Part 3: Moonshot Strategy Code