Issue with stopping and starting gateways

Technical Help
#1

I been trying to fetch 1 minute interval TRADES data for all stocks on NASDAQ from 1st Jan 2018 to present time. My plan was to pull the data year by year to limit the time it takes to gather the data and the sizes of the databases.

I believe I found a bug or two in relation to multiple gateways.

  1. I have noticed that my data collection appear to be starting from scratch again and again. This is in relation to stopping a gateway while fetching data. After the gateway is stopped it looks like the fetch command is issued again, and starts at the very beginning again.

  2. Starting a new gateway while a fetch is running. The new gateway never appear to be used even it's started and got the right permissions to collect the data.

container versions:
history:latest
ibg:972.0 and 973.1

#2

These weren't so much bugs, just design choices or known limitations. You should find the behavior more in line with your expectations in quantrocket/history:1.3.0. New gateways will be picked up when started mid-collection, and stopping gateways won't cause collection to start over.