Zipline backtest worker expires too quickly

Technical Help
#1

Hi there,

I've been trying to backtest a fairly complex algo through zipline and have gotten things to run pretty smoothly. The problem is that I noticed the backtest worker appears to kill itself every 24 hours or so. The algo requires minute-level processing of dozens of stocks, and so far I can only run about 1.5 months of backtest before the worker dies. Any way to extend its life?

    quantrocket_zipline_1|DAMN ! worker 1 (pid: 36) died, killed by signal 9 :( trying respawn ...
    quantrocket_zipline_1|Respawned uWSGI worker 1 (new pid: 87610)
    quantrocket_zipline_1|[deadlock-detector] a process holding a robust mutex died. 
    recovering...

Thanks in advance,
Paul.

#2

Most commonly that message means you are running out of memory. Linux will kill the process using too much memory in that case. I don’t believe there is any hard timeout on the worker, so I think memory is the more likely explanation. 24 hours may just be the point at which it grows too much for your available memory.

You can use docker stats to monitor the memory consumption of each container.