Zipline live orders canceled with error despite GTC param

bjsun · January 8, 2021, 10:29pm

Hi Brian,

I've finally gotten my live algorithm to open positions, but the stop orders I put in place concurrently to protect them are getting canceled at the end of day.

I realize that zipline cancels orders by design at the end of each day (and this is stated in your usage guide). To get around this, I added an order param {'Tif' : 'GTC} when submitted, which according to the IBKR API docs should keep the order alive until cancelled or expired.

This doesn't seem to work, as the blotter appears to intentionally cancel the open orders (and also generates errors when doing so):

quantrocket_houston_1|172.18.0.8 - - [08/Jan/2021:21:00:11 +0000] "DELETE /blotter/orders?order_ids=6001%3A17 HTTP/1.1" 202 83 "-" "python-urllib3/1.26.2"
quantrocket_houston_1|172.18.0.8 - - [08/Jan/2021:21:00:12 +0000] "DELETE /blotter/orders?order_ids=6001%3A19 HTTP/1.1" 202 83 "-" "python-urllib3/1.26.2"
quantrocket_houston_1|172.18.0.8 - - [08/Jan/2021:21:00:12 +0000] "DELETE /blotter/orders?order_ids=6001%3A21 HTTP/1.1" 202 83 "-" "python-urllib3/1.26.2"
quantrocket_houston_1|172.18.0.8 - - [08/Jan/2021:21:00:12 +0000] "DELETE /blotter/orders?order_ids=6001%3A23 HTTP/1.1" 202 83 "-" "python-urllib3/1.26.2"
  quantrocket_flightlog_1|2021-01-08 13:00:13 quantrocket.blotter: WARNING ibg1 client 6001 got IBKR error code 202: Order Canceled - reason:
quantrocket_houston_1|172.18.0.7 - - [08/Jan/2021:21:00:13 +0000] "POST /flightlog/handler HTTP/1.1" 200 5 "-" "-"
  quantrocket_flightlog_1|2021-01-08 13:00:13 quantrocket.blotter: WARNING ibg1 client 6001 got IBKR error code 202: Order Canceled - reason:
quantrocket_houston_1|172.18.0.7 - - [08/Jan/2021:21:00:13 +0000] "POST /flightlog/handler HTTP/1.1" 200 5 "-" "-"
  quantrocket_flightlog_1|2021-01-08 13:00:13 quantrocket.blotter: WARNING ibg1 client 6001 got IBKR error code 202: Order Canceled - reason:
quantrocket_houston_1|172.18.0.7 - - [08/Jan/2021:21:00:13 +0000] "POST /flightlog/handler HTTP/1.1" 200 5 "-" "-"
  quantrocket_flightlog_1|2021-01-08 13:00:13 quantrocket.blotter: WARNING ibg1 client 6001 got IBKR error code 202: Order Canceled - reason:

If this isn't the right way to keep stop orders open overnight, what is?

Thanks,
Paul.

Brian · January 12, 2021, 2:52pm

The blotter doesn’t automatically cancel any orders. It is just reporting back a cancellation message it received from the IBKR API (which uses “error code” 202 for any order cancellation).

bjsun · January 12, 2021, 8:53pm

Ok, good to know the blotter is not intentionally killing the order at EOD. I'll try to debug on IBKR side. Thanks.

bjsun · January 19, 2021, 9:57pm

Coming back to this, it looks like zipline live is intentionally canceling open orders at EOD after all.

To test this, I placed STOP orders (both buy and sell) from within a strategy file, and also placed identical STOP orders in the same account using quantrocket.blotter.place_orders() from a jupyter notebook. All orders had the time in force param {'Tif':'GTC'}, which should keep them open overnight (at least).

Right as the market closed, the orders placed from zipline were cancelled, but the orders placed from jupyter remained open.

Also, I checked the IB Gateway logs, and it confirms that these orders were canceled by user (and not on the IBKR side):

2021-01-19 21:00:00.671 [AV] INFO  [JTS-EServerSocket-10337] - CANCELSOURCE source: SELreason: Cancelled by User
2021-01-19 21:00:00.671 [AV] INFO  [JTS-EServerSocket-10337] - In Compression.composeHeader(): sending encryptionBasedHeader: 8=FIXCOMP^A8098=0^A9=
2021-01-19 21:00:00.716 [AV] INFO  [JTS-EServerSocket-10337] - CANCELSOURCE source: SELreason: Cancelled by User
2021-01-19 21:00:00.716 [AV] INFO  [JTS-EServerSocket-10337] - In Compression.composeHeader(): sending encryptionBasedHeader: 8=FIXCOMP^A8098=0^A9=
2021-01-19 21:00:00.757 [AV] INFO  [JTS-CCPDispatcherS2-35] - Order Canceled - reason:
2021-01-19 21:00:00.758 [AV] INFO  [JTS-EServerSocket-10337] - CANCELSOURCE source: SELreason: Cancelled by User
2021-01-19 21:00:00.758 [AV] INFO  [JTS-EServerSocket-10337] - In Compression.composeHeader(): sending encryptionBasedHeader: 8=FIXCOMP^A8098=0^A9=
2021-01-19 21:00:00.761 [AV] INFO  [JTS-EServerSocket-10337] - CANCELSOURCE source: SELreason: Cancelled by User
2021-01-19 21:00:00.762 [AV] INFO  [JTS-EServerSocket-10337] - In Compression.composeHeader(): sending encryptionBasedHeader: 8=FIXCOMP^A8098=0^A9=
2021-01-19 21:00:00.808 [AV] INFO  [JTS-EServerSocket-10337] - CANCELSOURCE source: SELreason: Cancelled by User
2021-01-19 21:00:00.810 [AV] INFO  [JTS-EServerSocket-10337] - In Compression.composeHeader(): sending encryptionBasedHeader: 8=FIXCOMP^A8098=0^A9=
2021-01-19 21:00:00.848 [AV] INFO  [JTS-EServerSocket-10337] - CANCELSOURCE source: SELreason: Cancelled by User
2021-01-19 21:00:00.850 [AV] INFO  [JTS-EServerSocket-10337] - In Compression.composeHeader(): sending encryptionBasedHeader: 8=FIXCOMP^A8098=0^A9=
2021-01-19 21:00:02.542 [AV] INFO  [JTS-CCPDispatcherS2-35] - Order Canceled - reason:
2021-01-19 21:00:02.544 [AV] INFO  [JTS-CCPDispatcherS2-35] - Order Canceled - reason:
2021-01-19 21:00:02.547 [AV] INFO  [JTS-CCPDispatcherS2-35] - Order Canceled - reason:
2021-01-19 21:00:02.556 [AV] INFO  [JTS-CCPDispatcherS2-35] - Order Canceled - reason:
2021-01-19 21:00:02.560 [AV] INFO  [JTS-CCPDispatcherS2-35] - Order Canceled - reason:

Is there a way to prevent zipline from cancelling these orders? Overnight stops are pretty essential for risk management.

Brian · January 21, 2021, 9:21pm

Try this:

import zipline.api as algo
from zipline.finance.cancel_policy import NeverCancel

def initialize(context):
    algo.blotter.cancel_policy = NeverCancel()

Currently you will need to do this and specify GTC in order_params. In the future we should automatically default to GTC if you’ve set NeverCancel as the policy.

bjsun · January 22, 2021, 9:09pm

Thanks for this, I didn't even know this setting existed! I'm also surprised that CancelEOD would be the default zipline setting.

The code snippet above actually throws an exception - AttributeError: module 'zipline.api' has no attribute 'blotter'

After digging into the zipline docs, this is what worked for me:

from zipline.api import set_cancel_policy, cancel_policy

def initialize(context):
    set_cancel_policy(cancel_policy.NeverCancel())