Error in MFG data prior to 09-04-2008

Issues
#1

usstock-1day and the minute data have a 1,000x price jump in MFG in 09-04-2008. Looks like the data prior is in error.

Sid,Date,Symbol,Open,High,Low,Close,Volume,Vwap,TotalTrades,UnadjOpen,UnadjHigh,UnadjLow,UnadjClose,UnadjVolume,UnadjVwap
FIBBG000Q85HQ2,2008-09-04,MFG,**0.0042,0.0042,0.004,0.004,**781153000,0.004,1961,8.16,8.16,7.77,7.89,781153,7.9299
FIBBG000Q85HQ2,2008-09-05,MFG,**3.8629,3.9697,3.7967,3.9646,**455113,3.9152,1669,7.59,7.8,7.46,7.79,455113,7.6928
#2

Thank you for reporting this. It is now fixed in both datasets and should be reflected in your local copies if you updated them overnight.

1 Like
#3

More of the same issue:

PPO,FIBBG000BBBQ84,data prior to 2007-07-05
DDD,FIBBG000D42FJ0, data prior to 2010-02-04
SYBX,FIBBG000R33JD4,data prior to 2015-11-12
MTW,FIBBG000D2M6H4,data prior to 2016-03-04
HBB,FIBBG00HJ4P620,data prior to 2017-10-02
IR,FIBBG002R1CW27,data prior to 2020-03-02

#4

Investigating...

#5

All of these but DDD have been fixed at the source and will be reflected the next time you update the db/bundle. Still investigating DDD...

#6

Thanks Brian.

#7

I've adjusted tolerances in our processing pipeline to catch and fix more of these faulty jumps going forward. DDD is a separate root cause and will take more time to address. I will update this ticket once it's resolved.

#8

Great, thanks. No doubt you're aware, but there are sometimes legitimate 10x or more returns. E.g. biotechs on life-support can jump that much on news. The list I sent above was manually filtered for some of those.

#9

Absolutely. There's no blunt hammer being applied based on price percentages. There are sometimes reliable clues in the adjustment factors and their relation to OHLCV. Other times, the data is ambiguous and can only be corrected manually. The examples you highlighted fell into both categories.

1 Like
#10

The issue with DDD has now been resolved. To get the corrected data into your Zipline bundle, you can run this command (assuming your bundle is called usstock-1min):

curl -X POST 'http://houston/zipline/ingestions/usstock-1min?sids=FIBBG000D42FJ0&force=true'

To get the corrected data into the daily history database, please drop and re-collect the database.

#11

This topic was automatically closed after 14 days. New replies are no longer allowed.