Error fetching data from ib: Filtering securities to permitted exchanges

Technical Help
#1

hi
i am following the instructions given in 'Getting started with equities Part I: Fetching data'
i have issued the following commands but get the error message shown from flightlog
it appears to indicate i requested 6 securies but 0 are permitted
WARNING Filtering securities to permitted exchanges: NYSE (6 securities requested, 0 permitted).
my interactive brokers account is subscribed for nyse market data.
and my quantrocket account has the exchange nyse subscribed
any help would be appreciated
thanks
darryl

------------------------------------commands----------------------------------------------------------------------------------------------

PS C:\Users\Darryl\quantrocket> quantrocket master universe 'demo-stocks' --infile securities2.csv
code: demo-stocks
inserted: 6
provided: 6
total_after_insert: 6
PS C:\Users\Darryl\quantrocket> quantrocket history create-db 'demo-stocks-1d' --universes 'demo-stocks' --bar-size '1 day'
status: successfully created quantrocket.history.demo-stocks-1d.sqlite
PS C:\Users\Darryl\quantrocket> quantrocket history fetch 'demo-stocks-1d'
status: the historical data will be fetched asynchronously

--------error from flightlog------------------------------------------------------------------------------------------
e[0me[33m2018-01-31 19:44:43 quantrocket.master: WARNING Filtering securities to permitted exchanges: NYSE (6 securities requested, 0 permitted)
e[0me[31m2018-01-31 19:44:43 quantrocket.history: ERROR [demo-stocks-1d] Fetching demo-stocks-1d history failed due to error: ('400 Client Error: BAD REQUEST for url: http://houston/master/securities.csv?universes=demo-stocks', {'status': 'error', 'msg': 'No securities match the query parameters'})

#2

The permission message is about quantrocket permissions not IB permissions. It confirms that your allowed exchange is NYSE. Are you sure the stocks in your demo-stocks universe are listed on NYSE (as the primary listing exchange)?

#3

The securities2.csv was obtained by running:
quantrocket master get --exchanges 'NYSE' --sec-type 'STK' --outfile securities2.csv
the six securities in the file are:
conId,Symbol,SecType,Etf,PrimaryExchange
4027, AAP,STK,0,NYSE
4050, ABM,STK,0,NYSE
4065, ABT,STK,0,NYSE
4151, ADC,STK,0,NYSE
4165, ADM,STK,0,NYSE
4187, ADX,STK,0,NYSE
The file states they are all listed on nyse as the primary exchange
however i could not find ADC listed according to Listings Directory for NYSE Stocks
i dont know if that helps.

#4

Puzzling. Nothing you posted looks amiss.

Do you get the filtering warning if you query that universe directly (as the history service is trying to do)?

quantrocket master get -u demo-stocks
#5

looks like the same error, this is with flightlog -d.

[1;36mquantrocket_flightlog_1|2018-01-31 22:27:24 quantrocket.master: WARNING Filtering securities to permitted exchanges: NYSE (6 securities requested, 0 permitted)e[0m
e[1;37m quantrocket_houston_1|172.18.0.10 - - [31/Jan/2018:22:27:24 +0000] "POST /flightlog/handler HTTP/1.1" 200 5 "-" "-"e[0m
e[1;37m quantrocket_houston_1|172.18.0.1 - - [31/Jan/2018:22:27:24 +0000] "GET /master/securities.csv?universes=demo-stocks HTTP/1.1" 400 71 "-" "python-requests/2.18.4"e[0m

#6

Double check what's in that universe directly in the database. You can go into one of the containers:

docker exec -ti quantrocket_db_1 bash

Then run this query:

sqlite3 /var/lib/quantrocket/quantrocket.master.main.sqlite 'select s.ConId, Symbol, SecType, PrimaryExchange from security s inner join universe u on u.ConId = s.ConId where u.Code = "demo-stocks"'
#7

you may be on to something.
i issued the commands:

PS C:\Users\Darryl\quantrocket> docker exec -ti quantrocket_db_1 bash
root@85c4d8f6d543:/# sqlite3 /var/lib/quantrocket/quantrocket.master.main.sqlite 'select s.ConId, Symbol, SecType, PrimaryExchange from security s inner join universe u on u.ConId = s.ConId where u.Code = "demo-stocks"'
root@85c4d8f6d543:/#

nothing is returned from the sql query.
using sqlitestudio i connected to the quantrocket.master.main.sqlite database
and found it consisted of 7 tables:

delisted
etf
Lotsize
rollover
security
timezone
universe

only the timezone table contained any data all the other tables are empty.
i dont know what this database should contain but i suspect this is wrong!
darryl

#8

If you get results from this query there must be data in some securities master database somewhere. Do you have two deployments running by chance? Check docker ps.

#9

I believe I have this same problem:

user@ubuntu:~/quantrocket$ quantrocket flightlog stream

2018-02-20 15:18:48 quantrocket.master: WARNING Filtering securities to permitted exchanges: NASDAQ (29 securities requested, 0 permitted)
2018-02-20 15:18:48 quantrocket.history: ERROR [demo-stocks-1d] Fetching demo-stocks-1d history failed due to error: ('400 Client Error: BAD REQUEST for url: http://houston/master/securities.csv?universes=demo-stocks', {'status': 'error', 'msg': 'No securities match the query parameters'})

user@ubuntu:~/quantrocket$ quantrocket master get --exchanges ‘NASDAQ’ --sec-type 'STK' --outfile securities2.csv
msg: No securities match the query parameters
status: error

So maybe my universe isn't set. So I try:

user@ubuntu:~/quantrocket$ quantrocket master universe 'demo-stocks' --infile securities.csv -r
code: demo-stocks
inserted: 29
provided: 29
total_after_insert: 29
user@ubuntu:~/quantrocket$ quantrocket master get -u demo-stocksmsg: No securities match the query parameters
status: error
user@ubuntu:~/quantrocket$

I was following the demo-stocks-1d tutorial but this is where I am stuck.

#10

Regarding this query, it looks like you have smart quotes around NASDAQ; those will be treated as part of the search query. If you remove the smart quotes does the query return data?

#11

No

user@ubuntu:~/quantrocket$ quantrocket master get --exchanges NASDAQ --sec-type 'STK' --outfile securities2.csv

msg: No securities match the query parameters
status: error
user@ubuntu:~/quantrocket$ quantrocket master get --exchanges 'NASDAQ' --sec-type 'STK' --outfile securities2.csv
msg: No securities match the query parameters
status: error
user@ubuntu:~/quantrocket$

#12

Did you previously fetch the listings from IB (quantrocket master listings -e NASDAQ -t STK) and did flightlog report that some listings were saved to the db?

If so, I would look in the database directly and see if anything is there:

docker exec -ti quantrocket_db_1 bash
$ sqlite3 /var/lib/quantrocket/quantrocket.master.main.sqlite 'select ConId, Symbol, SecType, PrimaryExchange from Security'

If nothing is there, and something was there at one point, I would check if there are somehow two deployments by running docker ps and seeing if there are any containers that don't start with quantrocket_.

#13

Looks like I had an empty database. I reran the commands and now I'm good! Thank you.