Getting unclear errors on IBKR paper trading

anatoliz · November 9, 2020, 6:03pm

Getting the following log on a daily strategy run:
Here is the line that gets executed in Cron / and log follows

15 10 * * mon-fri quantrocket master isopen 'XNYS' --in '1h' && quantrocket zipline trade 'a13v_adm' --bundle 'usstock-1d-bundle' --data-frequency 'daily' --account 'XXXXXX'

#Log:
2020-11-09 07:51:10 quantrocket.countdown: INFO Successfully loaded quantrocket.countdown.crontab.sh
2020-11-09 10:15:02 quantrocket.zipline: INFO [a13v_adm] Trading a13v_adm in account DU2761432
2020-11-09 10:15:07 quantrocket.zipline: ERROR Traceback (most recent call last):
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "sym://qrocket_log_py", line 34, in wrapped
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "sym://qrocket_qrzipline_trade_trade_py", line 62, in mule_trade_algo
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "sym://qrocket_qrzipline_trade_trade_py", line 184, in trade_algo
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/algorithm.py", line 669, in run
2020-11-09 10:15:07 quantrocket.zipline: ERROR for perf in self.get_generator():
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 205, in transform
2020-11-09 10:15:07 quantrocket.zipline: ERROR for capital_change_packet in every_bar(dt):
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 133, in every_bar
2020-11-09 10:15:07 quantrocket.zipline: ERROR handle_data(algo, current_data, dt_to_use)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/utils/events.py", line 218, in handle_data
2020-11-09 10:15:07 quantrocket.zipline: ERROR dt,
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/utils/events.py", line 237, in handle_data
2020-11-09 10:15:07 quantrocket.zipline: ERROR self.callback(context, data)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "sym://qrocket_qrzipline_trade_algorithm_py", line 163, in handle_data
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "sym://qrocket_qrzipline_trade_context_py", line 36, in store_context
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 502, in dump
2020-11-09 10:15:07 quantrocket.zipline: ERROR NumpyPickler(f, protocol=protocol).dump(value)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 409, in dump
2020-11-09 10:15:07 quantrocket.zipline: ERROR self.save(obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR f(self, obj) # Call unbound method with explicit self
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict
2020-11-09 10:15:07 quantrocket.zipline: ERROR self._batch_setitems(obj.items())
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(v)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 521, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR self.save_reduce(obj=obj, *rv)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 634, in save_reduce
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(state)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR f(self, obj) # Call unbound method with explicit self
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict
2020-11-09 10:15:07 quantrocket.zipline: ERROR self._batch_setitems(obj.items())
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(v)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 521, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR self.save_reduce(obj=obj, *rv)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 634, in save_reduce
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(state)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR f(self, obj) # Call unbound method with explicit self
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict
2020-11-09 10:15:07 quantrocket.zipline: ERROR self._batch_setitems(obj.items())
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(v)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR f(self, obj) # Call unbound method with explicit self
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict
2020-11-09 10:15:07 quantrocket.zipline: ERROR self._batch_setitems(obj.items())
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(v)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR f(self, obj) # Call unbound method with explicit self
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict
2020-11-09 10:15:07 quantrocket.zipline: ERROR self._batch_setitems(obj.items())
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems
2020-11-09 10:15:07 quantrocket.zipline: ERROR save(v)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR return Pickler.save(self, obj)
2020-11-09 10:15:07 quantrocket.zipline: ERROR File "/opt/conda/lib/python3.6/pickle.py", line 496, in save
2020-11-09 10:15:07 quantrocket.zipline: ERROR rv = reduce(self.proto)
2020-11-09 10:15:07 quantrocket.zipline: ERROR TypeError: can't pickle zipline._protocol.BarData objects
2020-11-09 10:15:07 quantrocket.zipline: ERROR

Brian · November 9, 2020, 9:39pm

It sounds like you’re saving the data object to context. You can’t do that because the context object gets pickled in live trading and Python can’t pickle the data object.

anatoliz · November 10, 2020, 1:46am

thank you, that makes sense - should have figured it out.

anatoliz · November 12, 2020, 5:48pm

Still getting errors after removing all data structures:

I was not sure if this line may still be a problem:
context.hedge_secs = [context.ief, context.tmf, context.tmv]
// but assumed list should be ok.

Here is the full code:

def initialize(context):
"""
Called once at the start of the algorithm / every day in live trading.
"""
# Create our dynamic stock selector.
attach_pipeline(make_pipeline(), 'SP500')
"""
Stocks Mometum trading
"""
context.MLEN1 = 125 ## Use momentum length 1
context.MLEN2 = 125 ## Use momentum length 2
context.NB_STOCKS = 20 ## Number of stocks to hold
context.STD_LEN = 20 ## Standard deviation length
context.VIXF_LEN = 16 ## Vix ema fast length
context.VIXS_LEN = 50 ## Vix ema fast length
context.BRF_LEN = 10 ## Bond ratio fast
context.BRS_LEN = 15 ## Bond ratio slow

def before_trading_start(context, data):
"""
Setup cash, leverage and trading
"""
context.LEVERAGE = 1.60
a13v.API_INIT(context, data, context.LEVERAGE, 0.98, 0.0)
"""
Setting global stocks
"""
context.spy = sid("FIBBG000BDTBL9")
context.lqd = sid("FIBBG000BBV9N3")
context.ief = sid("FIBBG000BHR296")
context.tmf = sid("FIBBG000CX0W11")
context.tmv = sid("FIBBG000DW07R9")
context.hedge_secs = [context.ief, context.tmf, context.tmv]

Errors:

Nov 12 07:15:02 35.202.88.247 quantrocket.zipline INFO [a13v_adm] Trading a13v_adm in account DU2761432

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR Traceback (most recent call last):

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "sym://qrocket_log_py", line 34, in wrapped

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "sym://qrocket_qrzipline_trade_trade_py", line 62, in mule_trade_algo

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "sym://qrocket_qrzipline_trade_trade_py", line 184, in trade_algo

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/algorithm.py", line 669, in run

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR for perf in self.get_generator():

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 205, in transform

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR for capital_change_packet in every_bar(dt):

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 133, in every_bar

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR handle_data(algo, current_data, dt_to_use)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/utils/events.py", line 218, in handle_data

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR dt,

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/zipline/utils/events.py", line 237, in handle_data

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self.callback(context, data)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "sym://qrocket_qrzipline_trade_algorithm_py", line 163, in handle_data

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "sym://qrocket_qrzipline_trade_context_py", line 36, in store_context

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 502, in dump

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR NumpyPickler(f, protocol=protocol).dump(value)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 409, in dump

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self.save(obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR f(self, obj) # Call unbound method with explicit self

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self._batch_setitems(obj.items())

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(v)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 521, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self.save_reduce(obj=obj, *rv)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 634, in save_reduce

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(state)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR f(self, obj) # Call unbound method with explicit self

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self._batch_setitems(obj.items())

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(v)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 521, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self.save_reduce(obj=obj, *rv)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 634, in save_reduce

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(state)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR f(self, obj) # Call unbound method with explicit self

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self._batch_setitems(obj.items())

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(v)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Load missing events

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR f(self, obj) # Call unbound method with explicit self

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self._batch_setitems(obj.items())

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(v)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 476, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR f(self, obj) # Call unbound method with explicit self

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 821, in save_dict

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR self._batch_setitems(obj.items())

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 847, in _batch_setitems

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR save(v)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 292, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR return Pickler.save(self, obj)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR File "/opt/conda/lib/python3.6/pickle.py", line 496, in save

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR rv = reduce(self.proto)

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR TypeError: can't pickle zipline._protocol.BarData objects

Nov 12 07:15:42 35.202.88.247 quantrocket.zipline ERROR

Brian · November 16, 2020, 2:08pm

Same error message, same issue. The best way to troubleshoot is to gradually simplify your code until the error goes away, then you'll know what was causing it.