Missing EDI data for some countries

Technical Help
#1

I subscribed to EDI data for 5 countries but I found for some of the countries, there are certain dates where data is missing for almost all securities.

Here is closing price data I have for XJPX:

from quantrocket import get_prices 

DB = "japan-1d"
start_date = "2007-01-01"
prices = get_prices(DB, start_date=start_date, fields="Close")
closes = prices.loc["Close"]
closes.count(axis=1).plot() # get total number of non NaN entries in every row

japan_screenshot
japan_screenshot859×326 25.6 KB

There are six dates where only a handful of stocks have data:

missing_dates

Data is missing for other fields as well on these dates (Open, Volume, TradedValue etc.)

Here is Sweden and Netherlands data:

sweden_screenshot
sweden_screenshot865×456 37.1 KB

netherlands_screenshot
netherlands_screenshot855×450 38.8 KB

Checking EDI documentation, I saw this note:

repush_note_1
repush_note_1849×125 15.6 KB

repush_note_2
repush_note_2701×89 12.2 KB

Could the missing data be in a repush file and haven't been added back into the dataset? Or is there another reason for the missing data?

Appreciate your help on this.

#2

Investigating…

#3

Thank you for reporting this issue. The missing data is present in the raw source files, so the explanation you propose is plausible. We have automated checks in place to detect low price counts and reprocess files in case of this issue, but those checks failed in some instances, and we are still investigating why. Thank you for your patience as we continue working to resolve the issue.

#4

Thank you, good to know the data is available. Awaiting a resolution.

#5

We have re-processed the files for the low count days for the affected exchanges. Please drop the XJPX, XAMS, and XSTO databases and re-collect them. We have also addressed the issue that caused the automated low count checks to fail, so this problem should be avoided going forward.

Note that the smaller dips in XSTO shown in your screenshot are not related to this issue. They appear to be related to market holidays on other exchanges that result in no trading activity for certain stocks that trade on XSTO but have their primary listing on another exchange.

#6

Dropping and recollecting the data for those exchanges shows the problem is fixed. Thanks very much!