Usstock-1min ingestion cuts around 6k

Technical Help
#1

from quantrocket.zipline import create_usstock_bundle, ingest_bundle
create_usstock_bundle("usstock-1min")
ingest_bundle("usstock-1min")

    quantrocket_zipline_1|Ingesting minute bars for FIBBG000DBNN59 (6450 of 20009)
    quantrocket_zipline_1|Ingesting minute bars for FIBBG000DBR2Q8 (6451 of 20009)
    quantrocket_zipline_1|Ingesting minute bars for FIBBG000DBR6Y0 (6452 of 20009)
    quantrocket_zipline_1|Exception in thread zipline_minute_sync_1:
    quantrocket_zipline_1|Traceback (most recent call last):
    quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    quantrocket_zipline_1|    self.run()
    quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/threading.py", line 864, in run
    quantrocket_zipline_1|    self._target(*self._args, **self._kwargs)
    quantrocket_zipline_1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 519, in wrapper
    quantrocket_zipline_1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 618, in _sync_minute_bars_for_sid
    quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
    quantrocket_zipline_1|    raise CalledProcessError(retcode, cmd)
    quantrocket_zipline_1|subprocess.CalledProcessError: Command '['aws', 's3', 'sync', '--quiet', '--exact-timestamps', '--delete', 's3://data.quantrocket.com/usstock/zipline/usstock-1min/minute_equities.bcolz/01/28/012805.bcolz', '/var/lib/quantrocket/zipline/data/usstock-1min/2020-01-01T00;00;00/minute_equities.bcolz/01/28/012805.bcolz']' returned non-zero exit status 1.
    quantrocket_zipline_1|

Second run

quantrocket_zipline_1|Ingesting minute bars for FIBBG000D0BQK2 (6251 of 20009)
quantrocket_zipline_1|Ingesting minute bars for FIBBG000D0D7X8 (6252 of 20009)
            quantrocket_houston_1|172.18.0.7 - - [10/Apr/2021:19:13:05 +0000] "POST /ibgrouter/registration/ibg3 HTTP/1.1" 200 5 "-" "python-requests/2.24.0"
            quantrocket_houston_1|172.18.0.6 - - [10/Apr/2021:19:13:05 +0000] "POST /ibgrouter/registration/ibg1 HTTP/1.1" 200 5 "-" "python-requests/2.24.0"
            quantrocket_houston_1|172.18.0.20 - - [10/Apr/2021:19:13:05 +0000] "POST /ibgrouter/registration/ibg2 HTTP/1.1" 200 5 "-" "python-requests/2.24.0"
            quantrocket_zipline_1|Exception in thread zipline_minute_sync_2:
            quantrocket_zipline_1|Traceback (most recent call last):
            quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
            quantrocket_zipline_1|    self.run()
            quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/threading.py", line 864, in run
            quantrocket_zipline_1|    self._target(*self._args, **self._kwargs)
            quantrocket_zipline_1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 519, in wrapper
            quantrocket_zipline_1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 618, in _sync_minute_bars_for_sid
            quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
            quantrocket_zipline_1|    raise CalledProcessError(retcode, cmd)
            quantrocket_zipline_1|subprocess.CalledProcessError: Command '['aws', 's3', 'sync', '--quiet', '--exact-timestamps', '--delete', 's3://data.quantrocket.com/usstock/zipline/usstock-1min/minute_equities.bcolz/00/84/008435.bcolz', '/var/lib/quantrocket/zipline/data/usstock-1min/2020-01-01T00;00;00/minute_equities.bcolz/00/84/008435.bcolz']' returned non-zero exit status 1.

!df -h still shows 6.8GB of space available.

#2

Since it’s two different sids, I think those are probably transient errors and should disappear by retrying.

Usstock-1min get_prices notify support bug
#3

Third time, same thing. I was able to fully collect on my cloud deployment with a t3.large. What could be causing the cloud vs local issue? On local I have allocated 12GB to docker.

 quantrocket_zipline_1|Ingesting minute bars for FIBBG000F6VTZ3 (6993 of 20009)
        quantrocket_zipline_1|Ingesting minute bars for FIBBG000F6YCQ7 (6994 of 20009)
        quantrocket_zipline_1|Ingesting minute bars for FIBBG000F6YK07 (6995 of 20009)
        quantrocket_zipline_1|Exception in thread zipline_minute_sync_2:
        quantrocket_zipline_1|Traceback (most recent call last):
        quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
        quantrocket_zipline_1|    self.run()
        quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/threading.py", line 864, in run
        quantrocket_zipline_1|    self._target(*self._args, **self._kwargs)
        quantrocket_zipline_1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 519, in wrapper
        quantrocket_zipline_1|  File "sym://qrocket_qrzipline_bundles_usstock_usstock_py", line 618, in _sync_minute_bars_for_sid
        quantrocket_zipline_1|  File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
        quantrocket_zipline_1|    raise CalledProcessError(retcode, cmd)
        quantrocket_zipline_1|subprocess.CalledProcessError: Command '['aws', 's3', 'sync', '--quiet', '--exact-timestamps', '--delete', 's3://data.quantrocket.com/usstock/zipline/usstock-1min/minute_equities.bcolz/01/35/013548.bcolz', '/var/lib/quantrocket/zipline/data/usstock-1min/2020-01-01T00;00;00/minute_equities.bcolz/01/35/013548.bcolz']' returned non-zero exit status 1.