According to Ofcom, the fine follows an incident which took place on June 25 of last year, during which BT experienced “a network fault that affected its ability to connect calls to emergency services between 06:24 and 16:56.” During that time, nearly 14,000 call attempts from 12,392 callers, were reported as being unsuccessful.
The UK regulator describes the incident taking place in three phases. The first of these took place from 06:24 to 07:33, when BT’s emergency call handling system was disrupted by what was later found to be a configuration error in a file on its server.
“This resulted,” said Ofcom, “in call handling agents’ systems restarting as soon as a call was received; agents being logged out of the system; calls being disconnected or dropped upon transfer to the emergency authorities; and calls being put back in the queue.”
BT was initially unable to determine the cause of the issue and attempted to switch to its disaster recovery platform. 'Phase two' of the incident – which took place from 07:33 to 08:50 – witnessed BT’s first attempt at this, something which proved unsuccessful “due to human error.” Ofcom states that this was due to instructions being “poorly documented and the team being unfamiliar with the process.”
The final phase took place from 08:50 to 16:56, which saw the rate of unsuccessful calls decrease once traffic was migrated successfully to the disaster recovery platform. “However, usual service was not fully restored initially as the disaster recovery platform struggled with demand.”
Discussing Ofcom’s findings, a spokesperson for the regulator said: “We found that BT did not have sufficient warning systems in place for when this kind of incident occurs, nor did it have adequate procedures for promptly assessing the severity, impact and likely cause of any such incident or for identifying mitigating actions.
“We also found that BT’s disaster recovery platform had insufficient capacity and functionality to deal with a level of demand that might reasonably be expected.”
The spokesperson continued: “The incident also caused disruption to text relay calls, which meant people with hearing and speech difficulties were unable to make any calls, including to friends, family, businesses and services. This left deaf and speech-impaired users at increased risk of harm.”
There were no confirmed reports by emergency authorities of serious harm to members of the public as a result of the incident. However, the potential degree of harm, said Ofcom, “was extremely significant.”
Ofcom’s director of enforcement, Suzanne Cater, said: “In considering the level of financial penalty, we took into account factors such as the seriousness, duration and degree of harm.
“We also considered steps BT has taken to remedy these issues, including fixing the error that caused the disruption; making improvements to fault monitoring; improving the disaster recovery platform; and documenting a clear process for switching to it.
“And we recognised that BT self-reported the incident, in line with its obligations, and provided regular updates. BT has also cooperated fully with our investigation and has provided Ofcom with information in a timely manner when requested.”
BT notified Ofcom of the issue, as required by law, with the latter’s investigation opening on June 28 2023.
Asked to comment on the fine, a BT Group spokesperson said: “We take great pride in underpinning the national 999 service and recognise the critical importance our infrastructure plays. The level of disruption to the service on Sunday 25 June last year has never been seen before and we are sincerely sorry for the distress caused.
“We accept the specific points raised in Ofcom’s findings, and have put in place comprehensive measures to prevent this series of events reoccurring, and improve end-to-end resilience of the system as a whole.”
The spokesperson continued: “While no technology is 100 per cent resilient, we have built a highly robust network with multiple layers of protection to connect the public to blue light services in their time of need.
“We take our responsibility to the emergency services and the public seriously, and on this occasion, we fell short of our own high standards for the 999 service.”
According to BT, the comprehensive measures mentioned above focus on tech, processes and training. They include a permanent fix/new alarm monitoring in relation to the initial fault, enhanced documentation and training in relation to the disaster recovery system, as well as improvements to the disaster recovery system itself.
Edited to include response from BT.
