Error in USSTOCK 1 min data for 2022-07-08

Issues
#1

Hi, Brian.
It seems to be an error in the USSTOCK minute bundle for 2022-07-08. the data from 12:00 up to the close seems to be missing for some / all SIDS.
Example:
Create and ingest bundle for one sid (FIBBG00Y1STTN2) in the terminal:

quantrocket zipline create-usstock-bundle usstock-1min-ysg --sids FIBBG00Y1STTN2
quantrocket zipline ingest usstock-1min-ysg

then load the data:

from quantrocket import get_prices

prices = get_prices('usstock-1min-ysg', start_date='2022-07-08', end_date='2022-07-08', fields=['High', 'Low', 'Close', 'Volume'])
closes = prices.loc['Close']
closes.reset_index(inplace = True)
closes[closes['Time'] > '11:58:00']

Sid index Date Time FIBBG00Y1STTN2
149 149 2022-07-08 11:59:00 1.9
150 150 2022-07-08 12:00:00 NaN
151 151 2022-07-08 12:01:00 NaN
152 152 2022-07-08 12:02:00 NaN
153 153 2022-07-08 12:03:00 NaN
... ... ... ... ...
385 385 2022-07-08 15:55:00 NaN
386 386 2022-07-08 15:56:00 NaN
387 387 2022-07-08 15:57:00 NaN
388 388 2022-07-08 15:58:00 NaN
389 389 2022-07-08 15:59:00 NaN
241 rows × 4 columns

I have not tested in depth but at least it happens for the sids FIBBG000BCMBG4, FIBBG000BPWTW7, FIBBG000BX4QV8. I suspect that for a lot more if not all of them.

I ignore if the problem also happens for other dates.

#2

Thank you for the report. Investigating...

#3

This issue has been resolved. To pull the minutes from 2022-07-08 into an existing bundle, please use the force parameter to force sync the bundle:

$ BUNDLE_NAME=usstock-1min
$ curl -X POST "http://houston/zipline/ingestions/$BUNDLE_NAME?force=true"
#4

Hi, Brian.
Is it possible that the issue affected other days?

#5

The issue only affected that one day. A quick spot check is to pick a liquid security like SPY and query a particular time of day over a longer date range:


prices = get_prices("usstock-1min", sids="FIBBG000BDTBL9", start_date="2022-01-01", times="12:05:00", fields="Close")
print(prices.to_string())

Sid                        FIBBG000BDTBL9
Field Date       Time                    
Close 2022-01-03 12:05:00         471.641
      2022-01-04 12:05:00         473.706
      2022-01-05 12:05:00         473.310
      2022-01-06 12:05:00         465.517
…
#6

Thanks, Brian.
Very helpful!

#7

This topic was automatically closed after 3 days. New replies are no longer allowed.