This is an archived documentation site for release 2.3. For the latest documentation or to access any other site features, please return to www.quantrocket.com

Software Update

Attention version 1 users: Version 2 is a major product upgrade from version 1. Please see the upgrade guide.

How to update

To update to the latest version of QuantRocket, download the latest Docker Compose file.

$ cd /path/to/quantrocket/folder
$ curl 'https://www.quantrocket.com/composefiles/latest/local/docker-compose.yml' -o docker-compose.yml

(You can also download the Compose file from the downloads page.)

Then deploy the changes using Docker Compose.

$ docker-compose -p quantrocket up -d --remove-orphans

You can reclaim disk space by removing old images that are no longer needed:

$ docker image prune --all

To update to the latest version of QuantRocket, download the latest Docker Compose file.

$ cd /path/to/quantrocket/folder
$ curl 'https://www.quantrocket.com/composefiles/latest/cloud/docker-compose.yml' -o docker-compose.yml

(You can also download the Compose file from the downloads page.)

Activate your cloud environment using Docker Machine, then deploy the changes using Docker Compose:

$ docker-machine env quantrocket
$ # then type the command docker-machine displays, for example on Linux/Mac:
$ eval $(docker-machine env quantrocket)
$ docker-compose -p quantrocket up -d --remove-orphans

You can reclaim disk space by removing old images that are no longer needed:

$ docker image prune --all

Check your version

The QuantRocket version number is indicated in the JupyterLab menu bar. You can also check the version from a terminal:

$ quantrocket version
2.3.0

Versioning scheme

The format of QuantRocket version numbers is MAJOR.MINOR.PATCH (for example 2.3.0). As new features are added, typically the minor version is incremented, with the major version occasionally being incremented for large redesigns or backwards-incompatible changes. Patch versions are incremented for bug fixes and small updates in between successive minor version releases.

Due to QuantRocket's microservice architecture, each service/container has its own version number, for example: quantrocket/master:2.3.0. These version numbers will share the same major and minor version as the Compose file's x-quantrocket-version, but the patch versions can differ.

Some services/containers have version numbers with more than three numeric components to reflect the version number of an important piece of underlying software. For example, quantrocket/ibg version numbers have the format 2.3.972.0, where 972 reflects the version of IB Gateway embedded in the container.


Release Notes

Release 2.3.0

2020-10-22

Release 2.3.0 includes improvements for teams and financial advisors, and other enhancements and bug fixes.

Highlights

  • improvements for teams, including a dedicated documentation section, support for linked license keys, and support for read-only S3 credentials. See the teams documentation
  • new dedicated documentation section for financial advisors, and support for placing FA Group/FA Profile orders through Interactive Brokers. See the documentation.
  • new endpoint: quantrocket flightlog wait/quantrocket.flightlog.wait_for_message. This function allows you to search for a log message and block until the message appears. This can be useful when you need your code to wait until a background process such as data collection finishes. See the usage guide.
  • add continuous_future function to Zipline research environment. See the API Reference.

Documentation

Complete 2.3.0 release notes

quantrocket/blotter:2.3.0

  • support for placing FA Group/FA Profile orders through Interactive Brokers. See the usage guide.
  • fix an issue where only a subset of order IDs might be returned when running Alpaca live and paper trading simultaneously and querying order status with the --open/open_orders=True parameter

quantrocket/db:2.3.0

  • add a --region/region parameter to quantrocket db s3config/quantrocket.db.set_s3_config to support pushing to and pulling from S3 buckets in regions other than us-east-1. See the API Reference.
  • read-only S3 credentials are now supported and can be used for pulling databases from S3 (without allowing pushing to S3). Intended for teams.

quantrocket/flightlog:2.3.0

  • new endpoint: quantrocket flightlog wait/quantrocket.flightlog.wait_for_message. This function allows you to search for a log message and block until the message appears. See the usage guide.

quantrocket/history:2.3.0

  • fix an issue reported in a forum post where IBKR data collection failed because ibkr_ConIds were not treated as integers.
  • fix an issue where multiple corporate actions occurring for the same security on the same day resulted in the second corporate action being incorrectly adjusted by the amount of the first adjustment factor, in EDI history databases

quantrocket/license-service:2.3.0

  • license keys are now encrypted at rest and are obfuscated in display output.
  • support linked license keys for teams.

quantrocket/realtime:2.3.0

  • fix an issue reported in a forum post where IBKR data collection failed because ibkr_ConIds were not treated as integers.
  • automatically re-connect when PostgreSQL connection is disconnected after being idle

quantrocket/zipline:2.3.0

  • add continuous_future function to research environment. See the API Reference.
  • fix futures contracts being out of order in data.current_chain(). See related forum post.
  • modify continuous futures "calendar" roll to roll on RolloverDate, not LastTradeDate. See related forum post.
  • fix an issue reported in a forum post where Zipline backtests could fail with KeyError due to stale trading calendars in the calendar cache.
  • make REPORTPERIOD and CALENDARDATE Sharadar fundamentals fields available in Pipeline API

Release 2.2.0

2020-09-03

Release 2.2.0 contains Zipline enhancements, performance improvements, usability improvements, and bug fixes.

Highlights

  • live trading of end-of-day strategies that use daily data. Previously live trading required the use of minute data. See the usage guide.
  • add Zipline pipeline integration for Alpaca easy-to-borrow and IBKR shortable shares datasets. See the usage guide.
  • data quality enhancements for US Stock dataset. See Data Corrections below.

API Changes

  • The preferred syntax for constructing Zipline pipelines with Sharadar or Reuters data has changed. The new syntax utilizes slicing to provide a cleaner syntax that better aligns with the conventions used on quantopian.com. For example, the syntax SharadarQuarterlyFundamentals.REVENUE is now deprecated in favor of sharadar.Fundamentals.slice(dimension="ARQ", period_offset=0).REVENUE. However, the deprecated conventions will still work. See the usage guide.

Documentation

Data Corrections

Corrections have been applied to the US Stock dataset to link together certain price series that were previously split up due to corporate restructurings. For example, in 2015 Google restructured and formed a new parent company, Alphabet. The US Stock dataset previously provided Google's price history under two separate sids: one for the pre-2015 Google and one for the post-2015 Alphabet. These are now linked together under one sid. This also applies to a number of other securities. For additional background, see related forum post.

To update your local copy of the dataset with these corrections, follow these instructions:

US Stock end-of-day dataset

Please drop and re-collect the dataset.

US Stock intraday bundle

Since version 2.1.0, the US Stock Zipline bundle can ingest data either sid by sid or day by day. The sid by sid method is used for initial collection or if there are more than 5 days' worth of updates to ingest. The day by day method is used if there are fewer than 5 days' worth of updates to ingest, as it is much faster in these cases.

The data corrections will be automatically ingested the next time the sid by sid method is used. To force this to happen, you can use the force parameter. This parameter is not included in the client library, so use curl as shown below:

$ BUNDLE_NAME=usstock-1min
$ curl -X POST "http://houston/zipline/ingestions/$BUNDLE_NAME?force=true"

Complete 2.2.0 release notes

quantrocket/blotter:2.2.0

  • fix an issue where the blotter would continue to request the order status of orders already rejected by Alpaca

quantrocket/codeload:2.2.0

  • add a GIT_KEEP_METADATA environment variable which can be used to automatically load a Git repository when your deployment first launches. (This is an advanced use case.) See the usage guide.

quantrocket/fundamental:2.2.0

  • gracefully handle a new situation where the IBKR API may indicate that no Reuters estimates data is available by returning an invalid XML file instead of the usual API error code. See support forum post for background.

quantrocket/history:2.2.0

  • improve error handling when you request historical data from IBKR and have not yet collected the security listings from IBKR

quantrocket/jupyter:2.2.0

  • Add the QuantRocket version number to the JupyterLab menu bar, with an indicator showing when new updates are available.
  • fix a performance issue with quantrocket.fundamental.get_sharadar_sp500_reindexed_like. See forum post for background.

quantrocket/postgres:2.2.0

  • update to TimescaleDB 1.7.3

quantrocket/realtime:2.2.0

  • improve error handling when you request real-time data from IBKR and have not yet collected the security listings from IBKR

quantrocket/zipline:2.2.0

  • live trading of end-of-day strategies that use daily data. Previously live trading required the use of minute data. See the usage guide.
  • add Zipline pipeline integration for Alpaca easy-to-borrow and IBKR shortable shares datasets. See the usage guide.
  • improve query performance of quantrocket.zipline.download_minute_file
  • add Order.open attribute as a shortcut for checking if an order is open. See API reference.
  • The preferred syntax for constructing Zipline pipelines with Sharadar or Reuters data has changed. The new syntax utilizes slicing to provide a cleaner syntax that better aligns with the conventions used on quantopian.com. For example, the syntax SharadarQuarterlyFundamentals.REVENUE is now deprecated in favor of sharadar.Fundamentals.slice(dimension="ARQ", period_offset=0).REVENUE. However, the deprecated conventions will still work. See the usage guide.
  • fix an error when trying to load a context file containing a ContinuousFuture in live trading
  • fix a bug that could cause the incremental ingestion of daily usstock updates to terminate early if encountering sids with no data for a particular date
  • update pandas version from 0.20.1 to 0.22.0 on the zipline service. This matches the pandas version in the "Zipline environment" kernel in JupyterLab.

Release 2.1.1

2020-07-27

This is a patch release that extends Zipline pipeline support to include futures and non-US equities.

quantrocket/jupyter:2.1.1

  • update Zipline environment with the changes described under quantrocket/zipline:2.1.1

quantrocket/zipline:2.1.1

  • add pipeline support for futures
  • fix an issue preventing the use of pipelines with non-US equities. See related forum post.
  • automatically derive the pipeline domain from the trading calendar so that a domain need not be specified

Release 2.1.0

2020-07-23

Release 2.1.0 contains a number of enhancements and usability improvements for Zipline strategy development, as well as bug fixes.

Highlights

  • This release introduces a new Zipline Research API which greatly improves usability by allowing you to perform much of your Zipline strategy development within the interactive environment of a research notebook. It includes support for running pipelines and accessing the data objects used in Zipline strategies. Along with this change, a Zipline kernel has been added to JupyterLab. See the usage guide.
  • A progress meter has been added to Zipline backtests that provides progress and performance statistics during long-running backtests. See the usage guide.
  • The new Zipline Intro tutorial in the Code Library walks you through the improved Zipline workflow.
  • The runtime when ingesting daily updates of the US Stock 1-minute dataset has been signficantly improved. See the fuller note in the API Changes section.

API Changes

  • The runtime for collecting daily updates of the US Stock 1-minute bundle has significantly improved. Previously, collecting an update took several hours but should now take only 10-15 minutes per day if updated daily (longer if collecting multiple days' updates). To take advantage of this speed improvement, you must drop and re-ingest the entire bundle. We regret this inconvenience, but previously ingested bundles did not store enough metadata to take advantage of the newly redesigned ingestion strategy. (Background: Previously, data was ingested by syncing one security at a time from the cloud to your local deployment. On updates, only the diff needed to be synced but determining the diff still took time. Now, if the Zipline service determines it will be faster, it will utilize an alternate strategy of loading individual daily update files, that is, ingesting data day by day instead of sid by sid. This is signficantly faster when you only need to ingest a day or a few day's worth of updates.)
  • In Zipline strategies, commissions and slippage are now disabled by default. See the usage guide for reasoning and examples. The examples are based on the previous default behavior and can be used by users who wish to preserve the previous default behavior. (Note for futures traders: the dictionary of exchange fees by root symbol in the linked example only includes a subset of root symbols for simplicity. The full list of exchange fees by root symbol that was previously used for default futures commissions is available in GitHub.)
  • In Zipline strategies, storing asset objects to context in the initialize() function is no longer supported and will throw an error. Instead, if you need to store asset objects to context, please do so in before_trading_start (or another function called after initialize). Storing asset objects to context in initialize() causes the asset objects to become stale in live trading because the asset objects are always loaded from the stored context instead of being re-instantiated each day. Daily re-instantiation of asset objects is necessary to ensure that the asset's metadata (particularly as concerns the asset's end date) stays up-to-date. Otherwise, assets will not be tradeable in live trading because Zipline will (incorrectly) think they are stale. We are choosing to immediately enforce this change instead of printing a deprecation warning due to the potential adverse effect on live trading.

The following is no longer allowed:

def initialize(context):

    context.aapl = algo.sid("FIBBG000B9XRY4")
    ...

Replace it with this:

def before_trading_start(context, data):

    context.aapl = algo.sid("FIBBG000B9XRY4")
    ...

Documentation

Complete 2.1.0 release notes

quantrocket/account:2.1.0

  • fix a bug where checking your portfolio before collecting securities master listings for your broker caused an error

quantrocket/blotter:2.1.0

  • improve reliability of Alpaca execution monitoring by always requesting an overlapping buffer of executions in case execution records arrive out of order. See related forum post.

quantrocket/history:2.1.0

  • fix an issue caused by a change in the IBKR API behavior which resulted in missing dates when collecting 1-day bars from IBKR. See related forum post.
  • query results will now respect the use of a datetime string (YYYY-MM-DD HH:MM:SS) for the start_date or end_date parameter when querying an intraday history database. Previously, only the date part was extracted and used in the query, even if a datetime string was passed.

quantrocket/ibg:2.1.972.0

  • improve error handling during rollback when setting IBKR credentials fails. See related forum post.

quantrocket/jupyter:2.1.0

  • add Zipline Research API, with support for running pipelines and accessing data objects in interactive research. See the usage guide.
  • add a "Zipline environment" kernel to support using the Research API for Zipline. See the usage guide.
  • add example templates to JupyterLab Launcher for Zipline strategies, Moonshot and Zipline allocation files, countdown crontabs, rollover rules configuration files, and IB Gateway permission files. Previously Launcher templates were only available for Moonshot and MoonshotML strategies.

quantrocket/postgres:2.1.0

  • update to TimescaleDB 1.7.2

quantrocket/realtime:2.1.0

  • fix an unhandled exception that occurred when you try to drop ticks from a real-time database that has no data

quantrocket/zipline:2.1.0

  • add progress meter to Zipline backtests. See the usage guide.
  • add endpoint to check bundle config. See API reference.
  • use multithreading during initial ingestion of US Stock 1-minute data bundle, reducing runtime from 15-18 hours previously to 12-15 hours currently.
  • improve runtime when ingesting daily updates of the US Stock 1-minute dataset. See the fuller note in the API Changes section above.
  • ingest data one security at a time when ingesting from a daily history database. Previously the entire database was ingested in one go which failed on large databases. See related forum post.
  • default to applying no commissions or slippage on backtests. See the usage guide for reasoning and examples. The examples are based on the previous default behavior and can be used by users who wish to preserve the previous default behavior.
  • fix an issue where the presence of illiquid securities with no price data caused an error when querying the US Stock 1-minute data bundle without specifying sids or universes. See related forum post.
  • fix an issue where ingestion failed if a security had a NULL symbol. See related forum post.
  • improve error message when you query a Zipline bundle using a date range that is not present in the trading calendar
  • fix an issue where ingesting a history database failed if you hadn't previously collected listings from Interactive Brokers
  • provide a helpful error message if user tries to ingest two distinct futures chains (for example that trade on different exchanges) that use the same root symbol

Release 2.0.0

2020-06-09

Highlights

API changes

QuantRocket version 2 is a major product upgrade from version 1. Due to the scale of changes, there are many breaking API changes that version 1 users should be aware of. See the migration guide.

Complete 2.0.0 release notes

quantrocket/account:2.0.0

  • support for querying account balances and portfolios from Alpaca
  • store all available exchange rates from ECB
  • derive GBX exchange rate from GBP

quantrocket/blotter:2.0.0

  • support for Alpaca. See usage guide
  • fix a SQL error that happens when you query summary pnl with a date range only and no other parameters

quantrocket/db:2.0.0

  • encrypt S3 credentials at rest

quantrocket/fundamental:2.0.0

quantrocket/history:2.0.0

quantrocket/ibg:2.0.972.0

  • encrypt IBKR credentials at rest

quantrocket/ibgrouter:2.0.0

  • renamed service from launchpad to ibgrouter

quantrocket/jupyter:2.0.0

quantrocket/license-service:2.0.0

quantrocket/master:2.0.0

quantrocket/postgres:2.0.0

  • update to PostgreSQL 12 and TimescaleDB 1.7.1

quantrocket/realtime:2.0.0

  • new real-time data provider: Polygon.io
  • add ability to drop ticks from tick databases. See usage guide
  • automatic compression of real-time databases

quantrocket/satellite:2.0.0

quantrocket/theia:2.0.0

  • update to latest version of Eclipse Theia

quantrocket/zipline:2.0.0

  • Zipline live trading. See the usage guide
  • redesign data collection API to mirror the API of the history service. See usage guide
  • support for incremental ingestion of history databases. See usage guide
  • Add pipeline support for Sharadar fundamentals, institutions, S&P 500 constituents, Reuters financials and estimates, and securities master. See the usage guide