Scenario Testing – Leveraging What You Have

Once firms have defined their important business services (IBS) and set up to 3 impact tolerances for each, firms have the unenviable task of testing their ability to remain within them. Whilst the number of IBS will differ from firm to firm and depend on their complexity, size and geographical reach, many firms have defined between 30 and 50 IBS. If firms were to take a purely theoretical approach to testing, they could quickly find themselves attempting to test each of these IBS with multiple severe but plausible scenarios, covering a disruption of people, technology, data, cyber, property and third parties. For firms that typically carry out 1 to 4 traditional crisis management exercises a year, this is an unrealistic task.

Current state assessments

Many organisations would readily admit that they have not yet understood how to fully leverage the full value of the data they hold. A pragmatic approach to scenario testing begins with firms identifying and analysing that data. Many firms will have a wealth of data about past events; incidents, near misses, crisis management exercises, disaster recovery and business continuity plan tests; after all many organisations spent a large period of 2020 in crisis mode in response to the pandemic. This data set together with the IBS resource mapping (people, property, third parties, technology, data), traditional business continuity metrics, an understanding of peak periods and critical foundational infrastructure, will provide a crucial set of inputs to help firms understand the level of confidence they have in their ability to recover within impact tolerance for each of their severe but plausible scenarios. Inevitably there will be gaps in the data and some IBS may not have been sufficiently stressed by a previous exercise or recent incident. These gaps should be the priority for the firm’s first round of scenario testing.

Unsurprisingly, the real issue for firms will be building the evidence base for those scenarios that test critical dependencies on third party suppliers. Historically, financial market infrastructure has been a point of contention between firms and the regulators, with differing opinions on whose responsibility it is to gain assurance over their resilience. The Policy made it clear that the regulators believe that responsibility sits with firms. Industry collaboration groups will likely be key in firms gaining the visibility and overall levels of assurance the regulators are expecting.

Severe but plausible scenarios & the slightly less severe

The term severe but plausible scenarios was very deliberate. It prompts firms to break away from the traditional risk management approach of considering both impact and likelihood when assessing and treating risk. Several large-scale incidents in the industry over the last decade prior to happening were deemed extremely unlikely and as a result, firms were ill prepared to respond. As a result, the regulators are urging firms to focus on the potential impact of scenarios, regardless how unlikely they may seem. Therefore, when it comes to defining the set of scenarios that firms use to test their IBS and impact tolerances, firms should be including their worst-case scenarios. For many this is a large-scale failure of technology or malicious cyber-attack. Many firms have spent the last decade bolstering their cyber defences with a focus on prevention and detection and this has been a deliberate choice. Effective response and recovery solutions come with a heavy price tag and typically require a large-scale transformation program focused on retiring legacy technology, implementing data vaults, or developing isolated skeleton infrastructure to serve as a head start for rebuilding technology if the worst were to happen. For many firms, this level of investment is unpalatable and due to the unlikelihood of such an event occurring the risk was accepted. As a result many firms do not need to run complex scenario tests, exercises or simulations to know that in the event of a large scale technology failure or malicious cyber incident they would far exceed their impact tolerances, This will unlikely be news to the regulators so firms should also look to test a set of slightly less severe but plausible scenarios, covering people (e.g. pandemic), data (e.g. corruption), technology (e.g. misconfiguration), supplier (e.g. Cloud outage) and property (e.g. severe weather event), where they are able to recover within impact tolerance. These test results, alongside remediation roadmaps that seek to close the resilience gaps and bring firms back within tolerance for those worst-case scenarios will help the regulators establish a baseline set of scenarios that they expect all firms to be able to recover from, within impact tolerances.

Following the first round of regulatory interaction Executive’s will be focusing on these three questions:

1. What scenarios are, or will be in the regulator’s wish list; the list of scenarios that overtime, regulators will expect firms to be able to demonstrate they can recover within?
2. What is the timeframe for this wish list of scenarios?
3. How much is it going to cost us?