Hi @brian, there was a large drop in the EDI XSTO data count last month:
Is it possibly related to the same issue I posted previously?
Appreciate the help.
This looks like a similar case of late data from EDI. The data is now available. Please drop and re-collect the XSTO database to obtain the data for Aug 11 and 12.
I apologize for the lack of a more automated solution on the QuantRocket end to this occasional late data publication by EDI. Let me give some thought to whether there is a better solution to get the late files into your database.
Great, the data appeared on my end. Would be grateful if you thought of a better way to add in the occasional late data from EDI. Thanks
The history service doesn't currently have the ability to automatically backfill EDI data for dates you've already collected, so dropping and re-collecting the database will be necessary when EDI adds data to earlier files. That being the case, my recommendation for a more automated solution is for you to schedule a regular cron job on your countdown crontab that drops and re-collects XSTO. That will keep your database updated with any late-arriving data. QuantRocket's backend data server has now been scheduled to look for backfilled EDI files early Sunday morning New York time, so Sunday afternoon NY time would be a good time to schedule dropping and re-collecting XSTO on your end.
This will work. I now scheduled cron jobs to drop and recollect all my EDI databases on Sunday afternoons NY time. Thank you!