Live Tracking Server Sync Disruption

Incident Report for Geopointe

Postmortem

We noticed an issue with our cloud server instance where it was not sending any data. We noticed an issue with real time live tracking partner where they were having issues on their side. We are leaning towards the issue that we had with real-time live tracking partner, cause a corruption in our instance that was unaccounted for and left our instance in a corrupted state. When we tried to restart our app servers, which is a usual solution to issues like this, we were unable to get the app server to update and were in a hanging state waiting for AWS to respond. We have since updated our Cloud server configuration so that this issue does not happen again, as well as updating our alarming system to be updated in situations like this one. No data was lost in this during the incident.

Posted Mar 11, 2024 - 17:26 EDT

Resolved

issue with our real-time live tracking partner, a geopointe live tracking dependency caused an issue with daily summary sync. Reported sync issue from 3/8 -3/11
Posted Mar 08, 2024 - 17:17 EST