Create dbs for "global market profiles"

Hi, I am trying to replicate the result of the "global-market-profiles" codes, which I can clone by running

$ quantrocket codeload clone 'global-market-profiles'

The code presumes the presence of the dbs

countries = (
    # North America
    ("usa-stk-1d", "United States"),
    ("canada-stk-1d", "Canada"),
    ("mexico-stk-1d", "Mexico"),
    # Asia
    ("australia-stk-1d", "Australia"),
    ("japan-stk-1d", "Japan"),
    ("hongkong-stk-1d", "Hong Kong"),
    ("singapore-stk-1d", "Singapore"),
    # Europe
    ("spain-stk-1d", "Spain"),
    ("netherlands-stk-1d", "Netherlands"),
    ("uk-stk-1d", "United Kingdom"),
    ("germany-stk-1d", "Germany"),
    ("france-stk-1d", "France"),
    ("belgium-stk-1d", "Belgium"),
    ("swiss-stk-1d", "Switzerland"),
    ("austria-stk-1d", "Austria"),
    ("sweden-stk-1d", "Sweden"),
    ("italy-stk-1d", "Italy"),

which the prerequisite describes "Data collection is not covered by this tutorial.".

I, however, would really appreciate some direction as to what functions to run to create the global non-us databases.

Thank you.

Global non-US data is available from EDI or IBKR (if you have an IBKR account). EDI data is generally preferable for historical research because (1) the data is survivorship-bias-free and (2) it is much easier to collect.

EDI data collection is straightforward: purchase the data then collect it as covered in the docs. For example, the canada db would be collected like this:

>>> from quantrocket.history import create_edi_db, collect_history
>>> create_edi_db("canada-stk-1d", exchanges="XTSE")
>>> collect_history("canada-stk-1d")

For the short sale notebook, you will need to collect IBKR shortable shares data (which does not require an IBKR account).

The primary vs consolidated notebook can't be recreated with EDI data as all of their data is primary exchange.

For the sector notebook, you would want to look at edi_Industry, instead of looking at ibkr_Sector as shown in the notebook. That notebook also expects universes (named 'canada-stk' for example) which can be made as shown in the docs.

The global-market-profiles repo predates the EDI data integration, so IBKR data was used originally. Due to the amount of effort required to collect that many exchanges from IBKR, data collection steps were omitted from the tutorial as it's not the greatest user experience. Collecting IBKR data requires the following steps:

Collecting IBKR data is entirely doable but the level of patience required increases in proportion to the number of exchanges.

1 Like