November 8, 2022
We began creating our vulnerability administration platform (VMP) at Spotify in Q2, 2020, and now that we’ve carried out it and use the system in our day-to-day work, we wished to take a second to share our journey to assist cut back safety dangers in an environment friendly and scalable method.
Vulnerabilities
Preventing vulnerabilities inside Spotify is the Security Tribe’s precedence, and we focus loads of our efforts on lowering the publicity of vulnerabilities. For instance, in our Golden Path, we embed our safety features out of the field so Spotify builders can construct their apps securely with none further effort. Inevitably, vulnerabilities come up, so it’s important that we usually consider our method to resolving them.
Toward automation
A vulnerability is assessed and reported on a vulnerability detection service, which we name a reactive management (RC). Once it’s reported, we seek for the proprietor of the impacted asset, notify them, and clarify the small print of the vulnerability, offering remediation steps and steering. Once the proprietor is notified, they’re chargeable for fixing the vulnerability. There’s way more to it in actuality, however that’s the sample of operations — and if you see a sample, it’s time to contemplate automation!
Vulnerability administration platform (Kitsune)
At first, we used Spotify’s Comet notification system for automation. We constructed peripheral programs to ingest vulnerability data from reactive controls into Comet. Then Comet would ship electronic mail notifications to asset house owners. When an asset proprietor fastened a vulnerability, they clicked the “Resolve” button within the electronic mail to replace us.
With Comet, we had the fundamental automation sample lined, however we wished to go even additional.
We reevaluated our vulnerability lifecycle by first defining the states that every vulnerability may very well be in, together with the actions required of the accountable crew. It was at this stage we determined so as to add extra states than what Comet supplied.
We wished to visualise the vulnerabilities every crew was assigned on Backstage — from there, groups might prioritize and act on these vulnerabilities. The final piece we included was the relational knowledge construction, which went past the occasion format knowledge Comet offered, making it simpler for us to course of the information for additional evaluation.
Those motivations led us to start out constructing Kitsune, our vulnerability administration platform.
Kitsune began as a backend API service that was to develop into the supply of fact about vulnerabilities inside Spotify, with a Backstage UI plugin for person interactions.
Kitsune now works as our vulnerability lifecycle administration system. Teams are robotically notified when vulnerabilities exist. And when mandatory, Kitsune can escalate to applicable stakeholders. We present vulnerability record pages on Backstage, and groups can go there anytime to assessment the vulnerabilities which are assigned to them, similar to earlier than dash planning, to incorporate vulnerability fixes of their day-to-day work.
Funnels of vulnerability sources
Kitsune started with vulnerabilities reported by two reactive controls, certainly one of which is the safety bug bounty program HackerOne. But we aimed to have the ability to deal with vulnerabilities from different RCs, too. In order to maintain the programs decoupled, we designed a limitation to Kitsune itself, so it handles enterprise logic utilized to vulnerabilities on the whole. We launched the idea of a “mediator” that takes care of RC-specific enterprise logic and converts the information to be suitable with Kitsune. After the launch of the mediator system, we have been capable of combine the VMP with just a few different RCs cleanly, and we’re working so as to add extra reactive controls to increase the protection. Also, it’s widespread {that a} vulnerability is reported by different sources, together with our staff, so we made it doable to add manually reported vulnerabilities to Kitsune with a CSV file.
Metrics
Providing insights primarily based on metrics was one huge objective from the start of this undertaking — driving groups to interact in fixing vulnerabilities, maintaining Spotify safe, and permitting the Security groups to plan methods. We designed the system with these targets in thoughts and have collected mandatory knowledge. With that relational knowledge, we generate metrics that assist Spotify staff perceive the present danger of every unit inside the org and the way they’ve resolved these previous vulnerabilities. We visualize this on Security Hub, our UI plugin on Backstage. In addition, we ship quarterly experiences to higher administration primarily based on the information we’ve collected.
What’s forward
We began growth on our vulnerability administration program a few 12 months and a half in the past, and it’s nonetheless evolving. There are loads of enhancements we are able to make, however our objective is to all the time allow Spotify to take dangers responsibly by participating individuals in an environment friendly and scalable manner.