Data ingestion US-1min bundle

[glenn]

Hi Brian,

Seems like we are experiencing similar data ingestion issues like a couple weeks ago.

Best,

Glenn

[glenn]

21st seems to be not available yet.

>>> from quantrocket import get_prices
>>> get_prices("usstock-1min", data_frequency="daily", fields="Close", start_date="2022-06-16")
Sid               FIBBG000C2V3D6  FIBBG00B3T3HD3  FIBBG00X5FSP48  FIBBG00LPXX872  FIBBG00YYCX683  ...  FIBBG0186GMYC7  FIBBG017779P96  FIBBG0185KW746  FIBBG01819S3P6  FIBBG0186KPCD2
Field Date                                                                                        ...                                                                                
Close 2022-06-16          114.96           49.68           24.46          18.395            9.85  ...             NaN             NaN             NaN             NaN             NaN
      2022-06-17          112.71           49.28           24.45          18.250            9.84  ...            0.05           9.799           0.095            24.4           37.08
      2022-06-21             NaN             NaN             NaN             NaN             NaN  ...             NaN             NaN             NaN             NaN             NaN
      2022-06-22             NaN             NaN             NaN             NaN             NaN  ...             NaN             NaN             NaN             NaN             NaN

[4 rows x 11605 columns]

[Brian]

The bundle was not available until around 6 AM New York time due to issues affecting a subset of securities. Have you tried ingesting since then?

[glenn]

Hi Brian,

Yes we did.

We also tried forced reingest for e.g. QQQ. I t says it is successful
, but we cannot see the data for 21st.

Best,

GLenn

[Brian]

Investigating...

[Brian]

The history database is now available. The Zipline bundle is processing but will take approx. 1.5 hours to be available for ingestion.

[glenn]

Thanks Brian.

As a suggestion, might be good to have a daily check or so after ingestion to see if symbols have updated as expected. I think you have one for overall, but maybe not ticker specific.

[Brian]

The usstock Zipline bundle is now available for ingestion.

I apologize for the delay. We received automatic alerts of the issue and we do actually perform automated spot-checking of individual securities. Unfortunately QQQ was one of the affected securities which is why you couldn’t force ingest it.

The root cause was related to the Juneteenth holiday because it is a new holiday. We will address the root cause.