IBKR Historical Data Collection - Cal Spread Demo

Technical Help
#1

Hello,
IB data collection for CL seems to be stuck for me and would appreciate any suggestions to help troubleshoot. I'm not currently an active subscriber at this time as I want to confirm setup, data collection, etc.

Setup/Installation:

  • Running on cloud server
  • IB gateway running against live account (only one account setup)
  • Subscribe to NYMEX data in IB

Steps completed so far:

  • Completed steps in the notebook to define universe of CL futures and create database
  • Limited started date to Jan 1, 2023 (to help reduce the data collection)
  • Kicked off collection process

The detailed log repeats versions of the following:

  • Retrying historical data request for 7D to 1min of BID_ASK for CLXX FUT ending YYYYMMDD after no response received from IBKR after XX seconds
  • ibg1 client XXXX got IBKR message code 366: No historical data query found for ticker id: XX
  • Historical Market Data Service error message: API historical data query cancelled

Then after several versions of the above in the log, it shows "Exception in thread historical_data_request_worker" and then "TimeoutError: waiting for next gateway for more than 5 minutes, this is probably a bug"

Then seems to restart and go through the same cycle as above.

I'm assuming there is a step that I'm missing or something in the config/setup that I need to correct. Appreciate any guidance.

#2

First, can you try canceling the collection and stopping IB Gateway:

from quantrocket.history import cancel_collections
cancel_collections('cl-1min-bbo')

from quantrocket.ibg import stop_gateways
stop_gateways(wait=True)

Then start IB Gateway and start the collection again and see if the same thing happens.

The TimeoutError could mean there's too much memory pressure. Can you also run

docker stats

And see how the memory usage looks. More information on memory problems here.

#3

Thank you, Brian for the quick response and guidance.

I did cancel the collection process, stopped the IBG gateway and restarted. Initially that did not work, was getting the same errors.

After shutting down and letting it sit for half a day, I then restarted and seemed to work. I have to assume there is something related to IBG historical data feed, but still not 100% clear on why (and how to troubleshoot if happens again).

Also, I do have 16GB on cloud instance. Appreciate the guidance on watching docker stats - I did launch on server and ibg1 container is ~1GB of memory, didn't see any spikes or excessive memory issues when I received the data collection errors in the logs.

All seems to be working now, so thanks for the guidance. Still not sure why, so will keep troubleshooting if occurs again (at least know what to check for now).

Thanks again.