The Need for Scientifically-Sound Cyber Risk Assessment

We need sound, rigorous, scalable methods to estimate cybersecurity risks of the products currently on the market, delivering patient care, today.

By Shannon Lantzy, MedCrypt Vice President of Consulting

The current standard practice for postmarket/continuous risk management is based on inconsistent estimation of qualitative risks (e.g., “low, medium, or high impact if device vulnerability is exploited”). This system’s accuracy is difficult to measure, and therefore difficult to systematically evaluate or improve. It is also hard to translate into action. We witness weeks-long debates between teams overrating a pentest finding as “high” or “critical,” because the rating system is open to interpretation. Worse, we see companies performing mathematical operations on ordinal rating scales (e.g., “low x low = 1, high x medium = 6”). This is similar to saying “a banana plus a banana equals two.” (Hat tip to Jason Tugman for that analogy.)

As with all new technology with great promise, connectivity in medical devices comes with new risks. The FDA, other global regulators, and medical device manufacturers use rigorous methods (e.g., randomized, controlled clinical trials) to demonstrate and evaluate clinical effectiveness and patient safety. However, cybersecurity risks cannot be measured in the same way. The industry needs more sound, rigorous, and scalable methods to generate and use evidence of cybersecurity risk. To achieve this, there are massive efforts underway, such as the effort to develop software bills of material (SBOM), implementation of continuous integration/continuous development pipelines, and other approaches to make security part of the automated approach to developing medical technology. However, these solutions are far from ready and available immediately across all products.

‍

‍

The medical device industry urgently needs to try new approaches. We need to protect patients who are using a plethora of devices to receive care today. We need to protect clinical innovation and public health by taking action on cybersecurity risk signals that matter and establishing a tolerance for risks that are below a reasonable threshold (i.e., let’s not waste our time on low risks). We also need to automate postmarket risk surveillance, so that it can scale with ever-increasing numbers of products, software, and vulnerabilities in the wild. We need translation between cybersecurity risks and business risks. And we need an approach that is accessible today, this without moving new mountains. It is time to maximize the use of existing data, guidance, and tools.

There are a variety of quantitative approaches to estimating widely uncertain, often qualitative sources of risk and benefit (see How to Measure Anything in Cybersecurity Risk for a primer). They include Bayesian approaches, estimating credible intervals and running simulations to forecast risk, to structured and systematic ways to elicit risk estimates from experts. These approaches are immediately applicable to medical device cybersecurity, theoretically. They have been demonstrated for cybersecurity in other industries, and have been used for benefit-risk assessment of traditionally qualitative evidence for medical products outside of cybersecurity. They could lay the foundation for improved approaches to medical device security risk assessment.

They could lay the foundation for improved approaches to medical device security risk assessment.

‍