What happened?

During the affected period of time a limited amount of import files were not accepted and processed by our backend server if the import file size was > 1 MB. This includes manual imports and imports initiated via custom ETL process.

This issue did not affect automated SFTP imports.

We are contacting individual affected customers to assist them in resolving this issue.

Affected period of time: 31.07.2019 08:10:06 CET - 01.08.2019 ~14:03:00 CET

Total: 1 day, 4 hours and 53 minutes

Why did it happen?

The issue was caused due to an error in the configuration of backend proxy server which was introduced during routine housekeeping event. Once the issue was identified it was immediately fixed.

The behavior is back to stable now.

What are we doing to avoid such issues?

1. Changes to critical parts of infrastructure will be reviewed by at least two individuals with access to infrastructure until final approval to minimize the effect of human error.

2. Since the process failed partially, we couldn't detect the issue quicker. We will tweak our monitoring scripts to catch isolated issues like the one occurred.