Access security master from outside docker

Technical Help
#1

so I am making more use of quant rocket to bridge security master from my strategies to IBKR. I have a few different services, I guess you would call them for simulation and trading. sometimes its most convenient to access security master data by hitting the security master sqlite database directly but of course that is not really very convenient from outside the container where the security master lives.

it occurs to me that perhaps it would be cool to have the security master be part of the Postgres database which is already running in the container? you could achieve that with very little code change by using an extension like sqlite_fdw to link the sqlite db from Postgres, which I googled but have not actually used.

or you could have the security master schemas be hosted in Postgres, this would of course make it simple to access this data from anywhere in the system. this might be viewed in some way as breaking encapsulation but I for many years have used SQL for many things and I so I choose not see it that way haha.

maybe this is feedback ?

#2

There are several options for accessing the security master database from outside Docker:

Option 1. Schedule a cron job in QuantRocket to push the security master database to S3. Download the database from S3 onto the other services that need it.

Option 2. If the other services are running on the same host as QuantRocket, you could alternatively download the database with docker cp instead of using S3. See local backup and restore.

Option 3. Another option if your services are running on the same host as QuantRocket is to query the database directly using docker exec. For example:

docker exec quantrocket-master-1 sqlite3 /var/lib/quantrocket/quantrocket.v2.master.main.sqlite 'SELECT COUNT(*) FROM Security'

Option 4. Another option is to create a custom script that accepts a query as a parameter, runs the query and saves the results to a file, and returns the file. See the return_file parameter of execute_command. This would allow you to query over HTTP/S using the quantrocket client.

Option 5. While this would be Python instead of SQL, you could connect to QuantRocket from your other applications and use quantrocket.master.get_securities.

Let me know if any of these work for your use case.

#3

Brian,
thanks for your help. I have been using the docker exec workaround. I like sqlite and use it extensively for private applications. Postgres or other client/server rdbms I like more for data that has value for consumers outside those inward facing contexts. I use security master data in many places in my code.

Thanks,
George Coles