[BUG Report]: Calendar returns invalid values

Technical Help
#1

If I query on the NYSE calander at exactly 9:30AM the resutls returned are invalid:

print(shifted_timestamp)
timestamp_ago = pandas.Timestamp.now('America/New_York') - shifted_timestamp
print(list_calendar_statuses(['NYSE'], ago = timestamp_ago)['NYSE'])

this produces the following output:

2019-01-03 09:30:00-05:00
{'status': 'open', 'since': '2019-01-02T09:30:00', 'until': '2019-01-03T16:00:00', 'timezone': 'America/New_York'}

Notice that the "until" field is correct and is the same day as the day I am querying at 4pm. However the "since" field is from 24 hours before the time I am querying. This implies that the exchange remained open and trading for the last 24 hours which is obviously incorrect.

As a side note I have spent about a month with a paid subscription and ran into so many bugs in containers or code that the system is almost unusable for me. This mostly takes the form of me regularly needing to restart containers and even had to hack a few of them and rewrite them (which wasnt easy since the source code isnt available). I wouldnt mind so much if it were open source as I could actually fix these bugs as I go, but as a paid service it feels far from production ready since we rely on @Brian to address these things and can't do them ourselves. I am happy to pay for a service but it needs to work well. Either that means partially opening up the source code or using the income to pay developers to bring this up to snuff honestly.

I would be more than happy to contribute for free (since I use the system) if I could rather than just complaining. But please can we bring this system to a stable point one way or the other. Please let me know if I can help with the code.

#3

This uses trading_calendars under the hood and it looks like you can trick it if you request at the exact second the market opens. A bit of an edge case but we’ll address it.

Regarding your other comments, sorry you’re not happy with the product. You may want to consider developing your own custom solution so that you have complete control. There are plenty of happy QuantRocket customers who are using the software in production, so your issues might be specific to you. I can’t say for sure because you’re speaking in generalities. If you’re having to restart containers all the time, I would make sure your system and Docker have enough memory.

1 Like
#4

@Brian So far I've had to circumvent most parts of QR other than the history part of it.

I wouldnt say its much of an edge case. Anyone processing on minute data in a back test is bound to hit this problem as 9:30:00 will be one of the points they back test. Also it doesnt appear to happen on ust 9:30 but also 9:31, so appears to be at least a minute window this can happen.

I have 64 gigs of memory and dont really approach anywhere near it being full when I operate so far. The issue seems to be in the containers. For example the license server randomly wont recognize licenses (and yes the license info shows im under limit) and when that happened I needed to restart container and then it worked. there are at least a dozen bugs I've come across that seem to be in the source code itself or the containers which I usually find once I go through the on-disc source code in the container. In most cases I was able to fix it by writing my own overlay containers that rewrite the code in some way. For example lets encrypt certificates arent saved to a volumn in containers, they get re-aquired when you restart the environment. This can lead to using up your lets encrypt limit and then your server is pretty much bricked unless you change your domain. In this case simply adding a volumn mapping doesnt solve it because the QR code is written in such a way that it checks if the certificates are there and if it is or not processes secondary logic on bootup. so simply mapping a volume in that case wasnt enough, extra steps had to be taken to make that work.

Other problems seem to be design choices rather than bugs. For example lack of support for docker-swarm, which is forced by the way the containers work (using reverse dns lookup to detect each other). While this doesnt hinder functionality it does cost server costs to be several thousand more than they would need to since I cant just bring up spot instances with high compute power on demand and need to run everything on a single computer.

Writing my own replacement is of course always an option, but I'd much rather focus on developing my strategies and pay someone to do that. I dont mind using your service, and I'd rather help you fix minor bugs (which I could do trivially) than write something from scratch or wait on you. But ultimately it may eventually come to me migrating away, its true. Like i said the system doesnt feel stable considering the number of bugs I keep running into and general performance limitations.

I have to wonder what percentage of subscription revenue is going towards development of the product. I would much rather pay money to see this developed than to develop my own or even to contribute bug fixes (which isnt an option anyway). But it doesnt feel like much of the money is going into development, please correct me if I'm wrong.