IBG / Zipline Error

@Brian a strategy failed to complete this morning due to an IBG or Zipline error. IBG was running fine and then when the CRON service initiated a strategy it failed to complete. It appears IBG shut down before the orders were finished. Logs are below. Any insights would be great. Thanks.

2024-12-19 07:35:02 quantrocket.history: INFO [sharadar-us-stk-1d] Collecting Sharadar US STK prices
2024-12-19 07:36:48 quantrocket.history: INFO [sharadar-us-stk-1d] Collecting updated Sharadar US securities listings
2024-12-19 07:36:51 quantrocket.master: INFO Building consolidated securities master from active vendors: ibkr, sharadar, usstock
2024-12-19 07:36:53 quantrocket.master: INFO Completed building consolidated securities master with 46534 records
2024-12-19 07:36:53 quantrocket.history: INFO [sharadar-us-stk-1d] Finished collecting Sharadar US STK prices
2024-12-19 07:40:03 quantrocket.zipline: INFO [sharadar-stk-1d] Ingesting daily bars for sharadar-stk-1d bundle
2024-12-19 07:40:45 quantrocket.zipline: INFO [sharadar-stk-1d] Ingesting adjustments for sharadar-stk-1d bundle
2024-12-19 07:40:47 quantrocket.zipline: INFO [sharadar-stk-1d] Ingesting assets for sharadar-stk-1d bundle
2024-12-19 07:40:48 quantrocket.zipline: INFO [sharadar-stk-1d] Completed ingesting data for sharadar-stk-1d bundle
2024-12-19 07:45:04 quantrocket.zipline: INFO [usstock-1min] Ingesting minute bars for 10859 securities in usstock-1min bundle
2024-12-19 07:49:40 quantrocket.zipline: INFO [usstock-1min] Ingesting daily bars for usstock-1min bundle
2024-12-19 07:50:02 quantrocket.fundamental: INFO Collecting Sharadar US fundamentals
2024-12-19 07:50:19 quantrocket.zipline: INFO [usstock-1min] Ingesting adjustments for usstock-1min bundle
2024-12-19 07:50:22 quantrocket.zipline: INFO [usstock-1min] Ingesting assets for usstock-1min bundle
2024-12-19 07:50:25 quantrocket.master: INFO Building consolidated securities master from active vendors: ibkr, sharadar, usstock
2024-12-19 07:50:26 quantrocket.master: INFO Completed building consolidated securities master with 46550 records
2024-12-19 07:50:26 quantrocket.zipline: INFO [usstock-1min] Completed ingesting data for 10859 securities in usstock-1min bundle
2024-12-19 07:51:23 quantrocket.fundamental: INFO Collecting updated Sharadar US securities listings
2024-12-19 07:51:23 quantrocket.fundamental: INFO Finished collecting Sharadar US fundamentals
2024-12-19 07:55:02 quantrocket.fundamental: INFO Collecting Sharadar US insider holdings data
2024-12-19 07:55:10 quantrocket.fundamental: INFO Collecting updated Sharadar US securities listings
2024-12-19 07:55:10 quantrocket.fundamental: INFO Finished collecting Sharadar US insider holdings data
2024-12-19 08:00:02 quantrocket.fundamental: INFO Collecting Sharadar US institutional investor data
2024-12-19 08:00:03 quantrocket.fundamental: INFO Collecting updated Sharadar US securities listings
2024-12-19 08:00:03 quantrocket.fundamental: INFO Finished collecting Sharadar US institutional investor data
2024-12-19 08:05:01 quantrocket.fundamental: INFO Collecting Sharadar US S&P 500 index constituents
2024-12-19 08:05:03 quantrocket.fundamental: INFO No data collected because already up-to-date
2024-12-19 08:22:03 quantrocket.zipline: INFO [WilliamsPercentR_Mean_Reversion_Strategy_LIVE] Trading WilliamsPercentR_Mean_Reversion_Strategy_LIVE in account DU9745156
2024-12-19 08:22:14 quantrocket.zipline: ERROR Traceback (most recent call last):
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "sym://qrocket_qrzipline_trade_trade_py", line 55, in mule_trade_algo
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "sym://qrocket_qrzipline_trade_trade_py", line 186, in trade_algo
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/algorithm.py", line 694, in run
2024-12-19 08:22:14 quantrocket.zipline: ERROR     for perf in self.get_generator():
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/gens/tradesimulation.py", line 182, in transform
2024-12-19 08:22:14 quantrocket.zipline: ERROR     for capital_change_packet in every_bar(dt):
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/gens/tradesimulation.py", line 117, in every_bar
2024-12-19 08:22:14 quantrocket.zipline: ERROR     handle_data(algo, current_data, dt_to_use)
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/utils/events.py", line 216, in handle_data
2024-12-19 08:22:14 quantrocket.zipline: ERROR     event.handle_data(
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/utils/events.py", line 238, in handle_data
2024-12-19 08:22:14 quantrocket.zipline: ERROR     self.callback(context, data)
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/codeload/zipline/WilliamsPercentR_Mean_Reversion_Strategy_LIVE.py", line 140, in daily_rebalance
2024-12-19 08:22:14 quantrocket.zipline: ERROR     algo.order_target_percent(asset, 0, style=MarketOnOpenOrder(exchange="SMART"))
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/utils/api_support.py", line 56, in wrapped
2024-12-19 08:22:14 quantrocket.zipline: ERROR     return getattr(algo_instance, f.__name__)(*args, **kwargs)
2024-12-19 08:22:14 quantrocket.zipline: ERROR            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/utils/api_support.py", line 125, in wrapped_method
2024-12-19 08:22:14 quantrocket.zipline: ERROR     return method(self, *args, **kwargs)
2024-12-19 08:22:14 quantrocket.zipline: ERROR            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/algorithm.py", line 1957, in order_target_percent
2024-12-19 08:22:14 quantrocket.zipline: ERROR     return self.order(asset, amount,
2024-12-19 08:22:14 quantrocket.zipline: ERROR            ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/utils/api_support.py", line 125, in wrapped_method
2024-12-19 08:22:14 quantrocket.zipline: ERROR     return method(self, *args, **kwargs)
2024-12-19 08:22:14 quantrocket.zipline: ERROR            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/zipline/algorithm.py", line 1314, in order
2024-12-19 08:22:14 quantrocket.zipline: ERROR     return self.blotter.order(asset, amount, style)
2024-12-19 08:22:14 quantrocket.zipline: ERROR            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "sym://qrocket_qrzipline_trade_blotter_py", line 98, in order
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "sym://qrocket_qrzipline_trade_order_py", line 84, in place
2024-12-19 08:22:14 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/quantrocket/blotter.py", line 157, in place_orders
2024-12-19 08:22:15 quantrocket.zipline: ERROR     houston.raise_for_status_with_json(response)
2024-12-19 08:22:15 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/quantrocket/houston.py", line 225, in raise_for_status_with_json
2024-12-19 08:22:15 quantrocket.zipline: ERROR     raise e
2024-12-19 08:22:15 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/quantrocket/houston.py", line 217, in raise_for_status_with_json
2024-12-19 08:22:15 quantrocket.zipline: ERROR     response.raise_for_status()
2024-12-19 08:22:15 quantrocket.zipline: ERROR   File "/opt/conda/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
2024-12-19 08:22:15 quantrocket.zipline: ERROR     raise HTTPError(http_error_msg, response=self)
2024-12-19 08:22:15 quantrocket.zipline: ERROR requests.exceptions.HTTPError: ('400 Client Error: BAD REQUEST for url: http://houston/blotter/orders', {'status': 'error', 'msg': "Can't connect to IB Gateway because no ibg services are running, please start service(s) and try again"})
2024-12-19 08:22:15 quantrocket.zipline: ERROR 

I would look at the ibg logs for clues to when and why ibg stopped. You could also adjust your crontab to make sure ibg is running at the time you need it to be. For example:

30 9 * * mon-fri quantrocket ibg start --wait && quantrocket zipline trade 'my-strategy'
1 Like