Exception ingesting usstock-1min

Technical Help
#1

I get the following exception from flightlog when ingesting 1 minute data via quantrocket zipline ingest 'usstock-1min':

quantrocket-zipline-1|ingesting 390 2025-04-04 minute prices for sid FIBBG000FD28T3 (2262 of 11081)
  quantrocket-zipline-1|Exception in thread zipline_minute_ingester_1:
  quantrocket-zipline-1|Traceback (most recent call last):
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
  quantrocket-zipline-1|    self.run()
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/threading.py", line 975, in run
  quantrocket-zipline-1|    self._target(*self._args, **self._kwargs)
  quantrocket-zipline-1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 503, in wrapper
  quantrocket-zipline-1|exiting due to exception in another worker
  quantrocket-zipline-1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 812, in _ingest_minute_prices_for_sid
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/site-packages/zipline/data/minute_bars.py", line 691, in write_sid
  quantrocket-zipline-1|    self._write_cols(sid, dts, cols, invalid_data_behavior)
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/site-packages/zipline/data/minute_bars.py", line 767, in _write_cols
  quantrocket-zipline-1|    last_recorded_minute = all_minutes[num_rec_mins - 1]
  quantrocket-zipline-1|                           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 5365, in __getitem__
  quantrocket-zipline-1|    return getitem(key)
  quantrocket-zipline-1|           ^^^^^^^^^^^^
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 376, in __getitem__
  quantrocket-zipline-1|    result = cast("Union[Self, DTScalarOrNaT]", super().__getitem__(key))
  quantrocket-zipline-1|                                                ^^^^^^^^^^^^^^^^^^^^^^^^
  quantrocket-zipline-1|  File "/opt/conda/lib/python3.11/site-packages/pandas/core/arrays/_mixins.py", line 276, in __getitem__
  quantrocket-zipline-1|    result = self._ndarray[key]
  quantrocket-zipline-1|             ~~~~~~~~~~~~~^^^^^
  quantrocket-zipline-1|IndexError: index 1792049 is out of bounds for axis 0 with size 1791660
  quantrocket-zipline-1|exiting due to exception in another worker

This is repeatable, even after container restart. It's a 100% blocker.

#2

I'm not able to reproduce this error when ingesting yesterday's prices. Please force a full re-sync like this:

curl -X POST "http://houston/zipline/ingestions/usstock-1min?force=true"

This will take longer to run than a normal update but will resolve any discrepancies between your local copy of the data and the remote copy.

The index 1792049 in your error message relates to the number of records in your local data for that sid, and it seems to be too high. It should be 1791660 for data updated through yesterday. I don't know why yours is different, but the sync should fix it. After the sync, you can run this code in a notebook:

from zipline.data.minute_bars import BcolzMinuteBarWriter
self = BcolzMinuteBarWriter.open("/var/lib/quantrocket/zipline/data/usstock-1min/2020-01-01T00;00;00/minute_equities.bcolz")
table = self._ensure_ctable(15104)
assert table.size == 1791660

If the assertion passes, then it seems like everything is good, and you should not experience this error the next time you update.

#3

Thanks. It's running. I'll update when it's done.

Update: worked.