QuantRocket 1.7.1 is now available. This is a bug fix release. Users who collect equities data from IB are encouraged to update. See how to update.
quantrocket/history:1.7.1
- fix an issue with historical data collection where earlier price data may be lost if split or dividend adjustments are detected for a security and data collection is interrupted before the security's entire history can be re-collected. See this community post for background.
This fixes the problem going forward but you also need to force re-collection for affected securities in existing databases.
To see how many securities may be affected in a given database, you can look for securities whose min date is after the date you created the database (note that this approach casts a wide net and might catch false positives like IPOs with genuinely limited data):
$ DB_CREATION_DATE=2019-01-01 # approximate date db was created
$ DB_CODE=my-db
$ sqlite3 /var/lib/quantrocket/quantrocket.history.$DB_CODE.sqlite "SELECT COUNT(*) FROM (SELECT ConId FROM MarketData GROUP BY ConId HAVING MIN(Date) >= '$DB_CREATION_DATE')"
Then delete the history and penultimate max date for those securities:
$ sqlite3 /var/lib/quantrocket/quantrocket.history.$DB_CODE.sqlite "DELETE FROM PenultimateMaxDate WHERE ConId IN (SELECT ConId FROM MarketData GROUP BY ConId HAVING MIN(Date) >= '$DB_CREATION_DATE')"
$ sqlite3 /var/lib/quantrocket/quantrocket.history.$DB_CODE.sqlite "DELETE FROM MarketData WHERE ConId IN (SELECT ConId FROM MarketData GROUP BY ConId HAVING MIN(Date) >= '$DB_CREATION_DATE')"
Then collect data as normal.