Polygon Realtime Aggregates Double Ticks?

Noticed in the Polygon data received window that the total_ticks seem to be doubled when streaming minute aggregates, but this was not the case with Alpaca. Is this intentional or an issue? Am I doing something wrong? See snippet below; notice each minute is coming through twice.

We’ve noticed that before with Polygon minute aggregates (not second aggregates or ticks). QuantRocket is just returning what it’s getting from Polygon. The good news is that the aggregate database (if you use one) will effectively dedupe it.

Ah ok. Yea, I did notice that the aggregate database does deduplicate the minute ticks, so that part is ok. Only challenge is that I use the streaming interface within my code as well, and so as long as the dupes are consistent I can filter them out.

Just wanted to make sure I didn't misconfigure something.

Thanks Brian.

Hey @Brian, after looking further into this I'm thinking there might be an actual issue here. While the prices are aggregated and deduped to produce correctly priced bars, the volume is still being duplicated for realtime data collected from Polygon and aggregated using TimescaleDB. This makes sense when you think about it.

I realized this issue when one of my strategies traded a very low volume ticker, that should have been filtered out by my minimum volume filter of 1.5m volume on the day. After troubleshooting this issue, I realized it was due the duplicate Polygon aggregates being summarized.

Here's my exact setup below for reproducing this issue:

Database Setup

quantrocket realtime create-polygon-tick-db 'usstock-realtime-polygon' --universes 'usstock-active' --fields 'MinuteOpen' 'MinuteHigh' 'MinuteLow' 'MinuteClose' 'MinuteVolume' 'MinuteVwap'

quantrocket realtime create-agg-db 'usstock-realtime-minute-polygon' --tick-db 'usstock-realtime-polygon' --bar-size '1m' --fields 'MinuteOpen:Open' 'MinuteHigh:High' 'MinuteLow:Low' 'MinuteClose:Close' 'MinuteVolume:Sum' 'MinuteVwap:Close'

Realtime Data Collection

quantrocket realtime collect 'usstock-realtime-polygon'

Realtime Data Usage

realtime_prices = get_prices('usstock-realtime-minute-polygon', start_date='2021-07-06', sids=['FIBBG011861C42'])

total_volume_on_july_2_2021 = realtime_prices['FIBBG011861C42'].loc['MinuteVolumeSum'].loc['2021-07-06'].sum()

# Total Volume Output (double what it should be)
1852172.0

This is easily fixed by configuration: just use MinuteVolume:Close instead of MinuteVolume:Sum.

Shit, I didn't even think of that. I guess the only issue is that I wouldn't be able to aggregate into larger than 1min bars when using 1min aggregates from Polygon as source, but 1min bars meet all my needs for now so this works.

Thanks Brian!