We experienced a limited outage of VoC Feedback survey answering system for certain customers which use custom survey domains in invites for WEB channel between May 15, 18:34:00, 2017 СЕТ and May 17, 12:07:00, 2017 СЕТ. This outage resulted in that during affected period the clients of certain customers which were attempting to open the survey from an email invite were shown a message "This site can't be reached" and "[domain_name].customer_name.tld refused to connect". This outage didn't affect VoC Feedback backend usage (allows to create new and edit existing surveys) and receiving invites.
We're not seeing any evidence of existing questionnaire data loss.
Downtime period: May 15,
Total measured downtime during business hours*: 11 hours 7 minutes
Total measured downtime: 41 hours 33 minutes
Why did it happen?
After restarting one of the services that supports custom survey domains in invites a web server has failed to launch due to an unexpected issue.
What have been done to avoid such issues?
A monitoring module was implemented that will perform required checks and monitor the accessibility of VoC Feedback answering system for custom domains to avoid such outages in the future.
We apologize for the inconveniences this outage has caused.
* Business hours: 9:00 - 17:00 CET