Illegal instruction error on houston on Apple M1 chip

After following the docker and quantrocket installations instructions on an Apple M1 machine, we’re seeing houston restart indefinitely with an Illegal instruction error in the logs like this:

e[36mhouston_1          |e[0m /opt/quantrocket/bin/houston.sh: line 15:    18 Illegal instruction     /usr/local/openresty/bin/openresty -g "daemon off;"
e[36mhouston_1          |e[0m no BASIC_AUTH_USER found so not creating .htpasswd file
e[36mhouston_1          |e[0m LETSENCRYPT_DOMAIN not set so not installing SSL
e[36mhouston_1          |e[0m qemu: uncaught target signal 4 (Illegal instruction) - core dumped
e[36mhouston_1          |e[0m /opt/quantrocket/bin/houston.sh: line 15:    18 Illegal instruction     /usr/local/openresty/bin/openresty -g "daemon off;"
e[36mhouston_1          |e[0m no BASIC_AUTH_USER found so not creating .htpasswd file
e[36mhouston_1          |e[0m LETSENCRYPT_DOMAIN not set so not installing SSL
e[36mhouston_1          |e[0m qemu: uncaught target signal 4 (Illegal instruction) - core dumped
e[36mhouston_1          |e[0m /opt/quantrocket/bin/houston.sh: line 15:    17 Illegal instruction     /usr/local/openresty/bin/openresty -g "daemon off;"
e[36mhouston_1          |e[0m no BASIC_AUTH_USER found so not creating .htpasswd file
e[36mhouston_1          |e[0m LETSENCRYPT_DOMAIN not set so not installing SSL
e[36mhouston_1          |e[0m qemu: uncaught target signal 4 (Illegal instruction) - core dumped
e[36mhouston_1          |e[0m /opt/quantrocket/bin/houston.sh: line 15:    17 Illegal instruction     /usr/local/openresty/bin/openresty -g "daemon off;"

We’ve tried forcing linux/amd64 as a platform without success.

There’s an issue in docker for mac which shows a possible workaround:

Does it work if you set that environment variable in your docker-compose.override.yml?

version: 2.4
services
  houston:
    environment:
      QEMU_CPU: max

I tried setting houston in the quantrocket provided docker-compose.yml this way:

  houston:
    image: "quantrocket/houston:2.5.0"
    platform: "linux/amd64"
    ports:
      - "1969:80"
    environment:
      QUANTROCKET_VERSION: "2.5.0"
      QEMU_CPU: "max"
    restart: always

But that didn’t help.

(btw, the mac install documentation shows jupyterlab running in port 1969, which confused a couple workmates).

@brian any updates on this?

Docker has had many issues with M1 and only recently went GA. Now that it has, we’ll try to get hold of an M1 and see if QuantRocket needs to do something. I assume you followed the special Docker instructions for M1?

In the meantime the clear workaround is to install on a different workstation or in the cloud.

Can you try restoring your docker-compose.yml to its original state and then replacing

image: 'quantrocket:houston:2.5.0'

with

image: 'quantrocket/houston:multiarch20210503'

and then running

docker-compose up -d houston

This is a multi-architecture build of houston which should support ARM. But it has not been tested because AWS doesn’t yet have M1 instances and Scaleway, the only cloud vendor that does, is sold out.

Even if this fixes houston, I can’t promise there won’t be issues with other containers besides houston, but this would be a good place to start.

Thanks @brian, we tried that out and found this new error in the logs:

nginx: [error] failed to initialize Lua VM in /usr/local/openresty/nginx/conf/nginx.conf:72

Unfortunately then this will have to be on hold until we can gain access to an M1.

sounds good! please keep us posted of the progress…

One other thing to try pending M1 access…

The 2.5.0 houston is on an somewhat old version of openresty/nginx, and it’s possible a newer version would help (houston is basically just an openresty server). With a clean docker-compose.yml, please try replacing

quantrocket/houston:2.5.0

with

quantrocket/houston:latest

which has the latest version of openresty, and see if that will run on M1.

That seems to make it!

Seeing something weird with port 80 not responding but 1969 in mac but it’s probably a different issue. We get jupyter lab on the 1969 port correctly… Will test some API calls for a while and report back if we find issues.

Please be aware that houston:latest with the updated version of OpenResty/Nginx breaks the zipline container’s ability to log to flightlog. You are welcome to keep using it but it will not be ready for general release until that is resolved.

1 Like

Thanks for the pointer! We’ll keep an eye on this.