NoDataForSid: No minute data for sid 6703

[mattc.fleming]

Hi folks,

I have ingested the usstock-1min bundle. When I try to get_prices, without specifying an SID, I get the following error:

code:

from quantrocket.zipline import create_usstock_bundle
create_usstock_bundle("usstock-1min")

from quantrocket.zipline import ingest_bundle
ingest_bundle("usstock-1min")

from quantrocket import get_prices
prices = get_prices("usstock-1min", start_date="2017-01-01", end_date="2017-01-03", fields=["Open","High","Low","Close", "Volume"])

Error:

('500 Server Error: Internal Server Error for url: http://houston/zipline/bundles/data/usstock-1min.csv?start_date=2017-01-01&end_date=2017-01-03&fields=Open&fields=Low&fields=Close&fields=High&fields=Volume', {'status': 'error', 'msg': 'an unhandled exception occurred, please check flightlog for the traceback and notify support if the problem persists as this might be a bug', 'error': 'No minute data for sid 6703.'})

Flightlog:
2020-07-15 13:09:05 quantrocket.zipline: ERROR reraise(exc_type, exc_value, tb)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/_compat.py", line 32, in reraise
2020-07-15 13:09:05 quantrocket.zipline: ERROR raise value.with_traceback(tb)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
2020-07-15 13:09:05 quantrocket.zipline: ERROR rv = self.dispatch_request()
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
2020-07-15 13:09:05 quantrocket.zipline: ERROR return self.view_functionsrule.endpoint
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask_restful/init.py", line 477, in wrapper
2020-07-15 13:09:05 quantrocket.zipline: ERROR resp = resource(*args, **kwargs)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/views.py", line 84, in view
2020-07-15 13:09:05 quantrocket.zipline: ERROR return self.dispatch_request(*args, **kwargs)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask_restful/init.py", line 587, in dispatch_request
2020-07-15 13:09:05 quantrocket.zipline: ERROR resp = meth(*args, **kwargs)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/webargs/core.py", line 441, in wrapper
2020-07-15 13:09:05 quantrocket.zipline: ERROR return func(*args, **kwargs)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "sym://qrocket_app_py", line 495, in get
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "sym://qrocket_qrzipline_bundles_query_py", line 192, in get_history_to_file
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/history_loader.py", line 549, in history
2020-07-15 13:09:05 quantrocket.zipline: ERROR is_perspective_after)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/history_loader.py", line 431, in _ensure_sliding_windows
2020-07-15 13:09:05 quantrocket.zipline: ERROR array = self._array(prefetch_dts, needed_assets, field)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/history_loader.py", line 595, in _array
2020-07-15 13:09:05 quantrocket.zipline: ERROR assets,
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/minute_bars.py", line 1273, in load_raw_arrays
2020-07-15 13:09:05 quantrocket.zipline: ERROR carray = self._open_minute_file(field, sid)
2020-07-15 13:09:05 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/minute_bars.py", line 1081, in _open_minute_file
2020-07-15 13:09:05 quantrocket.zipline: ERROR raise NoDataForSid('No minute data for sid {}.'.format(sid))
2020-07-15 13:09:05 quantrocket.zipline: ERROR zipline.data.bar_reader.NoDataForSid: No minute data for sid 6703.
2020-07-15 13:09:05 quantrocket.zipline: ERROR
2020-07-15 13:46:58 quantrocket.zipline: ERROR Traceback (most recent call last):
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/minute_bars.py", line 1073, in _open_minute_file
2020-07-15 13:46:58 quantrocket.zipline: ERROR carray = self._carrays[field][sid]
2020-07-15 13:46:58 quantrocket.zipline: ERROR KeyError: 6703
2020-07-15 13:46:58 quantrocket.zipline: ERROR
2020-07-15 13:46:58 quantrocket.zipline: ERROR During handling of the above exception, another exception occurred:
2020-07-15 13:46:58 quantrocket.zipline: ERROR
2020-07-15 13:46:58 quantrocket.zipline: ERROR Traceback (most recent call last):
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/data/minute_bars.py", line 1078, in _open_minute_file
2020-07-15 13:46:58 quantrocket.zipline: ERROR mode='r',
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "bcolz/carray_ext.pyx", line 1027, in bcolz.carray_ext.carray.cinit (bcolz/carray_ext.c:13769)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "bcolz/carray_ext.pyx", line 1225, in bcolz.carray_ext.carray._open_carray (bcolz/carray_ext.c:16163)
2020-07-15 13:46:58 quantrocket.zipline: ERROR OSError: data directory does not exist
2020-07-15 13:46:58 quantrocket.zipline: ERROR
2020-07-15 13:46:58 quantrocket.zipline: ERROR During handling of the above exception, another exception occurred:
2020-07-15 13:46:58 quantrocket.zipline: ERROR
2020-07-15 13:46:58 quantrocket.zipline: ERROR Traceback (most recent call last):
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "sym://qrocket_wsgi_py", line 30, in app
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1994, in call
2020-07-15 13:46:58 quantrocket.zipline: ERROR return self.wsgi_app(environ, start_response)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1985, in wsgi_app
2020-07-15 13:46:58 quantrocket.zipline: ERROR response = self.handle_exception(e)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask_restful/init.py", line 271, in error_router
2020-07-15 13:46:58 quantrocket.zipline: ERROR return original_handler(e)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1540, in handle_exception
2020-07-15 13:46:58 quantrocket.zipline: ERROR reraise(exc_type, exc_value, tb)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/_compat.py", line 32, in reraise
2020-07-15 13:46:58 quantrocket.zipline: ERROR raise value.with_traceback(tb)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
2020-07-15 13:46:58 quantrocket.zipline: ERROR response = self.full_dispatch_request()
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
2020-07-15 13:46:58 quantrocket.zipline: ERROR rv = self.handle_user_exception(e)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask_restful/init.py", line 271, in error_router
2020-07-15 13:46:58 quantrocket.zipline: ERROR return original_handler(e)
2020-07-15 13:46:58 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
Any help would be appreciated.

Thanks!
-Matt

[Brian]

There will be a fix for this in the upcoming 2.1 release but for now you'll need to specify sids or universes (which is generally recommended anyway due to the volume of data). The issue is that there are a handful of extremely illiquid securities which actually have no trades at all and the software is not handling that.

[mattc.fleming]

Got it, thanks Brian. All good.

[Brian]

QuantRocket 2.1.0 is now available and includes a fix for this issue.

QuantRocket 2.1.0 Release Notes

QuantRocket 2.1.0 is now available. This release contains a number of enhancements and usability improvements for Zipline strategy development, as well as bug fixes. All version 2 users are encouraged to update to the latest version. See how to update. Highlights This release introduces a new Zipline Research API which greatly improves usability by allowing you to perform much of your Zipline strategy development within the interactive environment of a research notebook. It includes support f…