Missing dates in IBKR FX data

[Bug reported by user via different channel]

Pulled historical FX data from IBKR. Excerpt from history logs:

Issuing to ibg1 historical data request for 1 W of 5 mins BID for GBP.USD CASH (sid FXGBPUSD) ending 20210511 03:59:59 GMT
Issuing to ibg1 historical data request for 1 W of 5 mins BID for GBP.USD CASH (sid FXGBPUSD) ending 20210518 03:59:59 GMT
Issuing to ibg1 historical data request for 1 W of 5 mins BID for GBP.USD CASH (sid FXGBPUSD) ending 20210525 03:59:59 GMT

Data is missing for every period end day. E.g. missing data for 20210511, 20210518, 20210525.

The IBKR API has (silently) changed how it interprets duration strings of "1 W". Previously a full week of data was returned but now the IBKR API only returns data through the 5:15 PM start of the FX trading session that occurred 7 days ago. For example, for a request for “1 W” of bars ending on 2021-05-17 at 23:59:00:

First bar Last bar
Previous IBKR API behavior 2021-05-11 00:00:00 2021-05-17 23:59:00
New IBKR API behavior 2021-05-11 17:15:00 2021-05-17 23:59:00

Which means part of 2021-05-11 is missing. Durations of "1 M" (1 month) are also affected. A similar issue occurred last year with daily bars and was addressed at that time. (Silent API changes are not uncommon with IBKR.)

The solution is to tell the IBKR API you want “7 D” of bars instead of “1 W”, as it correctly interprets “7 D” without the truncation behavior.

This will be fixed in an upcoming patch release, but you can force the behavior now by adding one or more of the following environment variables (depending on your desired bar size) to the history service in your docker-compose.yml:

history:
  environment:
    IB_DURATION_1_MIN: '7 D'
    IB_DURATION_2_MINS: '7 D'
    IB_DURATION_3_MINS: '7 D'
    IB_DURATION_5_MINS: '7 D'
    IB_DURATION_10_MINS: '7 D'
    IB_DURATION_15_MINS: '20 D'
    IB_DURATION_20_MINS: '20 D'
    IB_DURATION_30_MINS: '20 D'
    IB_DURATION_1_HOUR: '20 D'
    IB_DURATION_2_HOURS: '20 D'
    IB_DURATION_3_HOURS: '20 D'
    IB_DURATION_4_HOURS: '20 D'
    IB_DURATION_8_HOURS: '20 D'

We will also add a calendar test case for intraday FX to spot missing bars going forward. (We run calendar-based missing bar tests for a selection of IBKR asset classes and bar sizes, but not for every combination due to the large number of combinations.)

This issue is fixed in QuantRocket 2.6.1.

This topic was automatically closed after 5 days. New replies are no longer allowed.