By Astha Singhal, Lakshmi Sudheer, Julia Knecht
The Application Security groups at Netflix are accountable for securing the software program footprint that we create to run the Netflix product, the Netflix studio, and the enterprise. Our prospects are product and engineering groups at Netflix that construct these software program providers and platforms. The Netflix cultural values of ‘Context not Control’ and ‘Freedom and Responsibility’ strongly affect how we do Security at Netflix. Our aim is to handle safety dangers to Netflix through clear, opinionated safety steerage, and by offering threat context to Netflix engineering groups to make pragmatic threat choices at scale.
A couple of years in the past, we revealed this weblog put up about how we had organized our staff to focus our bandwidth on scalable investments versus simply conventional Appsec features, which weren’t scaling properly in our quickly rising surroundings. We leaned into the thought of strategic safety partnerships and automation investments to create extra leverage for software safety. This grew to become the inspiration for our present org construction with groups centered on Appsec Partnerships and Appsec Engineering. In this working mannequin, we offered essential Appsec operational providers to Netflix — together with bug bounty, pentesting, PSIRT (product safety incident response), safety critiques, and developer safety training — through a shared on-call rotation.
Over the previous few years, this mannequin has allowed us to give attention to investments like Secure by Default for baseline safety controls, Security Self-Service for clear actionable steerage and Vulnerability Scanning at scale for software program provide chain safety. We needed to share an replace on learnings from this mannequin, how our wants have advanced, and the place we anticipate to go from right here.
Among probably the most notable wins, we’ve been capable of make the most of this scale centered strategy to productize software safety for our quickly rising studio engineering ecosystem, standardize safety baseline for all Enterprise apps, and construct paved roads to offer Secure by Default Authentication & Authorization capabilities for central information engineering instruments. Our focus has been on bettering total safety assurance versus simply vulnerability prevention. We are actually increasing this strategy to extra elements of our ecosystem. This mindset has additionally allowed us to take a position our capability for white-glove service in the direction of cheap residual threat and normal steerage so we will cut back the necessity for white-glove engagements in the long run (e.g., funding in an API proxy that gives baseline safety controls without spending a dime versus pentesting all purposes that will finally sit behind that API proxy). This strategy has additionally allowed us to construct robust relationships with central engineering groups at Netflix (Data Platform, Developer Tools, Cloud Infrastructure, IAM Product Engineering) that may proceed to function central factors of leverage for safety in the long run.
However, it has not been all sunshine and rainbows. On the partnership aspect, the bespoke nature of every partnership implies that there isn’t consistency and redundancy constructed into the working mannequin and the associated partnership artifacts (e.g., Security Strategy and Roadmap, Threat Model, Deliverable Tracking, Residual Risk Criteria, and many others). This results in inadequate context sharing and excessive operational churn each time we’ve personnel modifications. The partnership constitution has additionally grown laterally into the infrastructure house as we stack our leverage bets on infrastructure parts (like Service Mesh, Container Platform, and many others). The ability units and area depth in these partnerships has additional diversified the talents on the staff. But it is a tradeoff on our skill to serve generalized Appsec oncall wants like bug bounty triage with excessive consistency. Given that partnerships give attention to long-running strategic initiatives, the wins could be few and much between and that may be tough for staff motivation. We additionally discovered numerous areas by which safety partnership work bleeds into safety product solutioning and it may be tough to determine the suitable handoff factors.
Additionally, because the complexity of our ecosystem grows, the aim of “single PoC into information security” turns into more and more harder to take care of. The staff is now investing in consistency and scalability of partnership artifacts and communication channels, higher redundancy and context sharing on the staff via squad working fashions, crisper engagement standards, and definition of achieved for partnership engagements.
Our Appsec Engineering staff builds merchandise to assist us scale, e.g.: a dynamic Asset Inventory that understands the nuances of our bespoke engineering ecosystem and the way our purposes and information relate to one another. This has advanced their identification to be a software program engineering staff that focuses on safety issues versus a safety engineering staff that writes code/software program. Our hiring has mirrored that shift, and we’ve added extra devoted software program engineers (SWEs) to the staff to assist us construct out software program. With this shift, we’ve integrated engineering greatest practices, and our merchandise have acceptable investments towards reliability and sustainability. As the staff skews in the direction of extra software program engineering centered expertise, ramping as much as help the shared Appsec-focused on-call has been difficult.
While initially constructed to help AppSec use instances round offering steerage to builders in a self-service approach, curiosity within the wealthy information and relationships we’ve in our instruments, particularly our Asset Inventory, has grown. As a consequence, we’ve continued to spend money on making our options scalable and accessible, so safety engineers can get the information they want extra simply to drive safety use instances. We’ve additionally found, via interviews with engineers, that self-service steerage doesn’t stand by itself. Moving ahead, the staff is investing in understanding our buyer use instances higher, and shifting our self-service story towards higher-context, extra opinionated automated steerage to make sure builders have the whole lot they should make really knowledgeable choices concerning the safety of their purposes (much like how they could make resiliency or different product choices).
As the Netflix enterprise and engineering workforce has grown, our software program footprint has additionally grown and turn into extra heterogeneous. At the identical time, partnerships have grown increasingly more strategic, and engineering has grown increasingly more software-focused. As our staff specialised, what emerged was a lack of strategic focus for our AppSec Professional Services constitution. These providers now want extra devoted strategic funding as the amount and help wants have grown. So, we are actually constructing out a devoted functionality centered on these essential providers which might be essential investments to be made and may not be served successfully through a shared Appsec on-call. This can be our “Appsec Reviews and Assessments” operate and we’re hiring for passionate, early profession Appsec engineers to hitch this group.
We will proceed to be taught as we undergo this subsequent section of evolution of our program. We hope to proceed to share these learnings with the broader group inquisitive about scalable product and software safety.