Disk failure the culprit in July 14 trading outage: SGX

An application also failed to detect the problem.

The prolonged disruption in Singapore Exchange (SGX) trading last Thursday, July 14, was due to a disk failure and was prolonged due to challenges in the orders and trade reconciliation process.

According to a statement by SGX, at about 9:38am, SGX detected Input/Output errors on a disk that runs the application to send out clearing confirmation messages to members.

“As the application did not detect the disk failure, which it should have, it did not automatically cutover to SGX’s backup secondary system. SGX initiated a manual cutover from the primary to secondary systems at 1012 hours,” SGX said.

As a result, some clearing confirmation messages were not generated, causing trading to be ceased at 11:38am, SGX said.

Meanwhile, SGX ensured the public that the disk has been replaced and complete health checks have been conducted.

“We are working with our vendor to review the application which sends out clearing confirmation messages, and will implement the necessary changes to ensure detection by the application of specific hardware problems. We will improve our process in data generation, and fine tune the data files to better enable our members in their reconciliation processes,” SGX said.

“We will work with members to review their order and trade reconciliation process, to improve overall recovery and market resumption, in the event of a similar recurrence. We will increase the number of our Business Continuity Planning scenarios which require industry-wide participation for reconciliation and recovery,” it added.