Missing day while getting 1day historical data for VIX

Issues
#1

We found the following issue while we were getting VIX (ConId=13455763) historical 1 day data with Interactive Brokers:

We are executing a code similar to the following:

create_db(db_name, conids=[13455763], bar_size="1 day")
collect_history(db_name)
wait_for_collections(db_name)
download_history_file(db_name, filepath_or_buffer=outfilepath)

But 2016-07-15 date is missing from the result data. The final raw data, just after download_history_file, looks like:

13455763,2016-07-12,12.93,13.93,12.75,13.55,0,0.0,1143
13455763,2016-07-13,13.32,13.79,12.92,13.04,0,0.0,1105
13455763,2016-07-14,12.5,13.37,12.14,12.82,0,0.0,1246
13455763,2016-07-18,12.75,13.12,12.33,12.44,0,0.0,914
13455763,2016-07-19,12.53,12.83,11.94,11.97,0,0.0,863

As you can see 2016-07-15 is missing from the result. This execution was made at 2020-07-14 04:30 and again at 10:00 with the same result (US/Eastern).
According to the trading calendar does not look a non-trading day.
We also downloaded the same data from IB directly and this date is not missing there. Another providers like YahooFinance contain this date too.

We are downloading this data daily (4:00 US/Eastern) and we found that yesterday (2020-07-13) file contains this date.

Any explanation for this?

#2

We repeated this today (2020-07-15) and we found the following:

  • 2016-07-15 is (again) in today data (but is not in yesterday's data)
  • 2012-07-16 is missing in today data (but yesterday data contains it)

IB raw data contains both dates for today and for yesterday.

#3

We found a similar behavior but with different missing dates with the SPX Index

#4

Thank you for reporting this issue.

It appears that IBKR has (silently) changed the behavior of their API when requesting chunks of data (unfortunately not the first time they’ve made a silent change). QuantRocket requests daily bars in 4-year chunks, and the issue is occurring at the seams of the requests. A request for 4 years of data ending 2016-07-16 (for example) used to include 2016-07-16 but no longer does.

There will be a fix for this in the upcoming 2.1 release, but you can also work around it by adding something like the following to docker-compose.override.yml to force QuantRocket to request all of the data in one-go, thus avoiding the seams:

version: '2.4'
services:
history:
  environment:
    IB_DURATION_1_DAY: '40 Y'

We’ve also added a test case that validates some collected data against a trading calendar to spot this kind of issue in the future.

1 Like
QuantRocket 2.1.0 Release Notes
Missing dates in IBKR FX data
#5

Thanks a lot for the response and for the quick fix

#6

QuantRocket 2.1.0 is now available and includes a fix for this issue.

#7

This topic was automatically closed after 4 days. New replies are no longer allowed.