A frequent objection to quantitative cyber risk assessment is that it’s too hard to get enough data to make an accurate forecast. The fallback position is that professional judgement and experience are somehow more reliable than estimates based on incomplete data. And there’s a certain amount of good feeling that goes with that. It’s easy – somehow it just feels right. But wait a minute, is there a rule about how much data is enough? The fact is that useful, quantitative insights about cyber risk are possible with surprisingly little data. High-quality data on likelihood and cost of an incident is surprisingly east to get, and much of it is free.

“Could we be hit by a cyber attack? If we were, how bad could it be? If these questions are on your mind, a good place to start looking for answers is the IRIS 2022 Report from Cyentia. This report continues their mission of building community knowledge and equipping security programs with data and resources to inform decision-making. It analyzes 77,000 publicly disclosed cyber events experienced over a 10-year period by some 35,000 organizations. Considering the scale and depth of the analysis, it is remarkably readable.

What’s the chance that we could be a victim? The incident frequency analysis estimates the likelihood that an organization would experience a cyber incident in the coming year. The analysis groups incidents by organization size, and then by industry type. For example, Healthcare and Financial sectors had the largest share of incidents on public record, and large organizations were 2.5 times as likely to experience an incident as small and mid-sized ones. Larger organizations were also more likely to report at least two events over the 10 years studied. If a Healthcare provider with a $100 million annual budget wanted to know its chance of being hacked in the next year, a simple look-up would suggest a baseline probability between 2.18% and 12.95% based on size alone. An adjustment factor of 1.03 for Healthcare moves the upper end slightly higher to  13.3%, making it about a once in 7 years event. How serious a threat that represents really depends on individual circumstances, most notably strength of the organization’s cyber defenses and its tolerance for loss. 

If it did happen, how bad would it be? To some degree, the report confirms intuition: every organization is unique. The amount lost can vary widely, and organization size doesn’t seem to matter as much as industry type. A top-level summary shows an expected loss of $211,000 for Healthcare, with a 5% probability it could exceed $13 million. In relative terms, that represents about one-eighth of the annual operating budget for a $100 million organization, and a 1 in 20 chance it could be worse. Not catastrophic, perhaps, but big enough to have significant operational impact.

The report spends some time debunking two common myths, one being  the widespread practice of using fixed cost-per-record rates to estimate the value of compromised data records. The data show that published rates only work within a very narrow range of record counts, with a much higher cost-per-record for small breaches and a much lower one for mega breaches. The good news is that the report set out a a simple model for accurately forecasting the loss for any number of records compromised.

Concerning mega-breaches, cyber incidents have earned a Smaug-like reputation for how much destruction they can bring. The IRIS study shows that this dragon has a very long and heavy tail, Analyzing the top 10% of loss events revealed a relatively low probability of extreme losses ($2 billion or more) and that a loss this big was most likely to happen in a large organization in the Information Technology, Financial or Professional Service sectors. For a mid-sized organization in Health Care there should be some comfort in the notion that they are probably among the many organizations that never experience a serious security incident. However unlikely though, this dragon can breathe fire; an extreme outcome is still a real risk that needs to be managed realistically.

So what should we do about it? That depends on the kind of attack, and the reports takes a deep dive into attack patterns, identifying eight distinct types, ranked by frequency in each industry sector. System Intrusion came first in frequency (50%), financial loss (60%), and the amount of data compromised (39%); Accidental Disclosure ranked second for frequency (23%) but fourth for financial loss (5.2%); Ransomware ranked third overall, accounting for 6.5% of incidents and 7% of the overall financial loss. Fraud and Denial of Service attacks brought up the rear.

If System Intrusion is the greatest risk how do the attackers get in? The report finishes with an analysis of initial access techniques, bringing in information about attack techniques from other sources, including the MITRE ATT&CK website. A tabulation of initial access methods by industry sector identifies which ones pose the greatest threat in each. In Healthcare, for example,  hijacking valid account credentials ranked at the top, followed by Trusted Relationships and Phishing. The simple message is that if a threat actor isn’t posing as an employee or a trusted business partner, chances are they will try to target one through a phishing attack. That doesn’t answer the question of how to keep them from doing harm, but it’s a place to start planning how to keep them out. MITRE provides a list of mitigating controls that can help.

Overall, the IRIS report is a very useful resource. It analyzes a large body of data that can give insights about broad industry trends and help inform decisions about risk and cyber defense. The level transparency about methods and limitations are also helpful, especially when attempting to apply it to individual situations.  The 10-year time scale is acknowledged as a two-edged sword, making for a rich data set but possibly understating the significance of more recent attack types like Ransomware, which was almost unheard of until the last few years of the period examined. Despite the limitations, the IRIS report is an important point-in-time snapshot that sets out a rigorous analysis and presents it in language that is engaging and breezy with just enough statistical detail to be useful readers who are analytically inclined.