Collecting IBKR listings Error Traceback

Hi there,

What can I do with an error after trying to collect listings

collect_ibkr_listings(exchanges=[‘SGX’], sec_types=[‘STK’])

from IBKR? It happens from time to time on different exchanges, so I can collect securities sometimes, and see an error sometimes:

2020-09-22 09:26:18 quantrocket.master: INFO Collecting SGX STK listings from IBKR website
2020-09-22 09:26:29 quantrocket.master: INFO Requesting details for 740 SGX listings found on IBKR website
2020-09-22 09:26:34 quantrocket.master: INFO Expected remaining runtime to collect listings: 0:00:52
2020-09-22 09:31:42 quantrocket.master: ERROR Traceback (most recent call last):
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/botocore/vendored/requests/packages/urllib3/response.py", line 243, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     data = self._fp.read(amt)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/http/client.py", line 457, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     n = self.readinto(b)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/http/client.py", line 501, in readinto
2020-09-22 09:31:42 quantrocket.master: ERROR     n = self.fp.readinto(b)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/socket.py", line 589, in readinto
2020-09-22 09:31:42 quantrocket.master: ERROR     return self._sock.recv_into(b)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/ssl.py", line 1071, in recv_into
2020-09-22 09:31:42 quantrocket.master: ERROR     return self.read(nbytes, buffer)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/ssl.py", line 929, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     return self._sslobj.read(len, buffer)
2020-09-22 09:31:42 quantrocket.master: ERROR socket.timeout: The read operation timed out
2020-09-22 09:31:42 quantrocket.master: ERROR 
2020-09-22 09:31:42 quantrocket.master: ERROR During handling of the above exception, another exception occurred:
2020-09-22 09:31:42 quantrocket.master: ERROR 
2020-09-22 09:31:42 quantrocket.master: ERROR Traceback (most recent call last):
2020-09-22 09:31:42 quantrocket.master: ERROR   File "sym://qrocket_log_py", line 34, in wrapped
2020-09-22 09:31:42 quantrocket.master: ERROR   File "sym://qrocket_mule_py", line 75, in mule_collect_ibkr_listings
2020-09-22 09:31:42 quantrocket.master: ERROR   File "sym://qrocket_master_ibkr_collect_website_py", line 220, in collect
2020-09-22 09:31:42 quantrocket.master: ERROR   File "sym://qrocket_master_ibkr_collect_base_py", line 496, in write_to_db
2020-09-22 09:31:42 quantrocket.master: ERROR   File "sym://qrocket_master_ibkr_collect_sid_py", line 51, in load_ibkr_sids
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py", line 676, in parser_f
2020-09-22 09:31:42 quantrocket.master: ERROR     return _read(filepath_or_buffer, kwds)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py", line 454, in _read
2020-09-22 09:31:42 quantrocket.master: ERROR     data = parser.read(nrows)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py", line 1133, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     ret = self._engine.read(nrows)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/pandas/io/parsers.py", line 2037, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     data = self._reader.read(nrows)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "pandas/_libs/parsers.pyx", line 860, in pandas._libs.parsers.TextReader.read
2020-09-22 09:31:42 quantrocket.master: ERROR   File "pandas/_libs/parsers.pyx", line 875, in pandas._libs.parsers.TextReader._read_low_memory
2020-09-22 09:31:42 quantrocket.master: ERROR   File "pandas/_libs/parsers.pyx", line 929, in pandas._libs.parsers.TextReader._read_rows
2020-09-22 09:31:42 quantrocket.master: ERROR   File "pandas/_libs/parsers.pyx", line 916, in pandas._libs.parsers.TextReader._tokenize_rows
2020-09-22 09:31:42 quantrocket.master: ERROR   File "pandas/_libs/parsers.pyx", line 2063, in pandas._libs.parsers.raise_parser_error
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/_compression.py", line 68, in readinto
2020-09-22 09:31:42 quantrocket.master: ERROR     data = self.read(len(byte_view))
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/gzip.py", line 469, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     buf = self._fp.read(io.DEFAULT_BUFFER_SIZE)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/gzip.py", line 82, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     return self.file.read(size)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/botocore/response.py", line 76, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     chunk = self._raw_stream.read(amt)
2020-09-22 09:31:42 quantrocket.master: ERROR   File "/opt/conda/lib/python3.7/site-packages/botocore/vendored/requests/packages/urllib3/response.py", line 258, in read
2020-09-22 09:31:42 quantrocket.master: ERROR     raise ReadTimeoutError(self._pool, None, 'Read timed out.')
2020-09-22 09:31:42 quantrocket.master: ERROR botocore.vendored.requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Read timed out.
2020-09-22 09:31:42 quantrocket.master: ERROR

This is a timeout while trying to download a file from S3. I think the default timeout is 60 seconds. The file being downloaded is not very big (less than 1MB) so the likely explanation is some spottiness in the network, which could be anywhere between your machine and AWS.