Finding security threats that matter: Two industrial case studies

In the past decade, speed has become an essential trait of software development (e.g., agile, continuous integration, DevOps) and any inefficiency is considered unaffordable time waster. Such a fast pace causes challenges for architectural threat analysis. Leading techniques for threat analysis, like STRIDE, have the advantage of being systematic. However, they are not equipped to discern between important and less critical threats, while the threats are being discovered. Consequently, many threats are discarded at a later time, when their risk value is assessed. An alternative technique, called eSTRIDE, promises to remove these inefficiencies by focusing the analysis on the critical parts of the architecture. Yet, no empirical evidence exists about the actual effect of trading off systematicity, for a more focused attention on high-priority threats. This paper contributes with an empirical study comparing these two approaches in the context of two industrial case studies. We found that the two approaches yield the same number of security threats during a given time frame. However, participants using eSTRIDE found twice as many high-priority threats. The underlying analysis procedures cause similarities and differences in the execution. In addition, security expertise has an effect (albeit small) on the quality of analysis outcomes and execution. •Rigorous comparison of two threat analysis techniques with practitioners.•Evidence of the risk-first approach yielding twice as many high-priority treats.•Security expertise induces a slower-paced but more precise (and detailed) analysis.