Service was fully restored earlier today, and as of 9pm UTC, traffic has been balanced between our two Data Centers.
What to expect next: We are closing this incident. If you have any additional questions, please contact our Customer Care team. https://support.kaltura.com
Posted Aug 06, 2020 - 21:23 UTC
Monitoring
Previous Impact: Newly uploaded media was not immediately visible in KMS or My Media page. Also, users experienced inconsistent results when running search queries.
Current Status: The issue was mitigated as of 4:52 pm UTC by shifting traffic to our West Coast data center and we have verified that normal performance was restored.
The root cause has now been identified and resolved. Root cause has been confirmed as contained to only Elastic Search (no database or DB indexing issue) and specifically to the intervals at which content was pushed into and made available in Elastic Search and KMC/KMS applications.
What to expect next: Our Engineering team has confirmed root cause, resolved the issue and will provide an RCA. Traffic will be re-balanced at 9pm UTC. Please contact Customer Care for additional information.
Posted Aug 06, 2020 - 19:03 UTC
Update
Previous Impact: Newly uploaded media was not immediately visible in KMS or My Media page. Also, users experienced inconsistent results when running search queries.
Current Status: The issue was mitigated as of 4:52 pm UTC by shifting traffic to our West Coast data center and we have verified that normal performance was restored.
What to expect next: Our Engineering team is further investigating root cause in order to share additional details. An additional update will follow within 2 hours.
Posted Aug 06, 2020 - 18:01 UTC
Update
Impact: Newly uploaded media is not visible in KMS or My Media page. Also, users could experience inconsistent results when running search queries.
Current Status: The issue has now been mitigated as of 12:52 pm EDT by shifting traffic to our West Coast data center.
What to expect next: Our Engineering team is currently investigating root cause in order to resolve the issue. Additional updates will follow.
Posted Aug 06, 2020 - 17:02 UTC
Investigating
Our Engineering team is currently investigating uploads not immediately appearing in KMS/KMC or showing as visible after upload completes. Additional updates will follow.