Introduction

This is a book about making developers more productive, embedding security practices into the SDL and ensuring that security risks are accepted and understood.

The focus is on the Dev part of SecDevOps, and on the challenges of creating Security Champions for all DevOps stages.

All content is released under an Creative Commons license (CC BY 3.0) and the GitHub repository Book_SecDevOps_Risk_Workflow contains all text and ideas.

This book is based on successful and unsuccessful real world applications of these ideas. Any feedback, suggestions or comments will be highly appreciated (please open an issue for them)

Book under construction

There are multiple sections of this book that are still on ‘draft’ mode (as you can see by the rest of this introduction, which still needs a serious rewrite)

Here are some ideas that I feel will be good on this intro section.

• aimed at: Developers, Security Professionals, Risk Practitioners, Software architects, Security Champions
• presented Risk Workflow
  • is key make security decisions explicit
  • based on JIRA, but it can also be made to work on GitHub
  • provides a framework to make development/security decisions accountable, visible and understood
  • the ‘Accept Risk’ button changes the dynamics & makes the risks/information real
• big focus on the role of Security Champions and the automation of Security Knowledge and Workflows
• practical examples of SecDevOps and DevOps workflows/tools will be shown (git workflows, docker, travis, Jenkins, others…)
• objective is to make SecDevOps into DevOps of just Dev (with SECurity and OPerationS happening behind the scenes, automatically in the CI pipeline)
• based on real world application of these ideas
  • objective to scale this ideas and ask for feedback
• The JIRA Risk workflow fits as the way to implement SecDevOps
• Lots of good thinking on SecDevOps
• See this issue for more details on this decision Issue 15
• Book is positioned on adding security to DevOps with a focus on Risk acceptance and Code Analysis workflows

Here is good definition of SecDevOps is:

SecDevOps (Securing DevOps) The scenario here is an organisation is embarking on a DevOps and agile ways of working adoption journey. There is concern about security and we are asked to advise on how to embed security into the DevOps style of operation. And this means embedding and ensuring “secure by design” discipline in the software delivery methodology using techniques such as automated security review of code, automated application security testing, educating and empowering developers to use secure design patterns etc.

(previous) Introduction

This book will give you a solution for the following common problems inside AppSec and Development teams:

• “How do I get my manager to take security issues in my app seriously”
• “How do I get time to spend on non-functional requirements and refactoring”
• “We are product-driven development team and don’t have time for anything that is not customer-driven”
• “We know our current development, testing and deployment environment is highly inefficient, but how can we prove that to management”
• “We constantly do hacks and compromises before deadlines, but we can’t measure its real impact, and how they always tend to be a false economy”

I find that collectively developer teams know what needs to be done, but they are not usually listened to or empowered to make important decisions about their development priorities and environment. There are always side effects of rushed code and hacks.

This book contains the materials created while using JIRA to manage RISK and measure those side effects

All information presented is based on real work experience in creating, implementing and deploying multiple JIRA RISK Projects.

There are 2 main target audiences is AppSec and Developers.

Why AppSec

AppSec (Application Security) has massive problem of how modern developers work and therefor how to communicate with developers

I created this workflow in order to solve this problem

Why Developers

Because developers can use this workflow to control their development workflow, and handle the ‘backlog pit of despair’ problem.

Why JIRA

• It’s market penetration (most companies I work for use JIRA)
• JIRA and GitHub are the only Issue tracking system that I have seen that have the kind of features that are required for the kind of complex workflows that are needed
• JIRA can be integrated with various other tools used in Agile/DevOps/CI allowing for further integration. (ie future ready)
• comment that this is very different from the JIRA Risk plugins that exist current in the JIRA marketplace

Why not infoSec

InfoSec tickets don’t tend to work that well with JIRA tickets. InfoSec usually has their tools and dashboards to manage (for example Firewalls, Anti-Virus detections, Active Directory issues)

Could these same techniques be used to manage other types of Risk. Yes but since I don’t have that much experience with it, I will leave it to the reader.

Note that if your ‘InfoSec’ team/person cannot code (and would not be hired by the Dev team), then that is NOT AppSec. InfoSec is very important, but it is key to understand its limitations when there are no coding skills in that department (there are a number of conversions and activities that can only be done by professionals that understand development)

Is this view too restrictive? Can someone be a valuable AppSec specialist without actually being able to code. Would these qualities compensate for lack of coding skills: * understanding modern development, * being able to explain and discuss issues with developers, * being able to explain, configure, and use tools

Why I’m writing this book

• my struggle to communicate with developers
• explain how I arrived at the JIRA Risk project workflow
• brief O2 Platform history, and how I used to communicate with developers in a common language (in O2 it was via O2 scripts and mini-tools)
• O2 Platform REPL and how it changed how I code
  • what IDEs should be doing for developers and AppSec professionals

Change log

Here are the changes made (per version):

• v0.66 (Dec 2016)
  • created or updated:
    • “Don’t blame the developers”
    • “Can’t do Security Analysis when doing Code Review”
    • “Defensible findings”
    • “Disposable IT infrastructure”
    • “Every project starts with 100% code coverage”
    • “Everything is code”
• v0.65 (Nov 2016)
  • created or updated:
    • “Learning resources”
    • “Abusing the concept of RISK”
    • “Make sure your Security Champions are given time”
    • “Using Git as a Backup Strategy”
    • “Threat Model per Feature”
    • “Threat Model Confirms Pen Test”
    • “Can’t do Security Analysis when doing Code Review”
    • “Employ Graduates to Manage JIRA”
    • “Linking source code to Risks”
    • “Why GitHub and JIRA?”
    • “Risk Dashboards and emails”
    • “The Authentication micro-service cache incident”
• v0.64
  • created or updated:
    • “The Pollution Analogy”
    • “Risk Workflow for Software Vendors”
    • “Security Champions Involved In Decisions”
    • “Is The Decision Hyper Linked”
    • “Horizontal Dev Ops”
    • “Good Managers Are Not The Solution”
    • “Feedback Loops”
• v0.63 (Oct 2016)
  • created or updated:
    • “Conference for Security Champions”
    • “Collaboration Technologies”
    • “The smaller the ticket scope the better”
    • “Describe Risks as Features rather than as Wishes”
    • “Capture knowledge when developers look at code”
    • “Using logs to detect risks exploitation”
    • “Understand Every Project’s Risks”
    • “Feedback loops are key”
    • “Cloud Security”
    • “Creating better briefs”
    • “I don’t know the security status of a website”
    • “Relationship with existing standards”
• v0.62
  • created or updated:
    • “Using Artificial Intelligence for proactive defense”
    • “If you have a heartbeat, you qualify!”
    • “What it takes to be a Security Champion”
    • “If you don’t have an Security Champion get a mug”
    • “How to review Applications as a Security Champion”
    • “Risk accepting threat model”
    • “Inaction is a risk”
    • “Create an Technology Advisory Board”
• v0.61
  • created or updated
    • “Run Apps Offline”
    • “5000% code coverage”
    • “Annual Reports should contain a section on InfoSec”
    • “Why SecDevOps?”
    • “In DevOps Everything is Code”
    • “Do security reviews every sprint”
  • added ‘draft notes’ section (after appendix)
  • added 14 audio transcription files
• v0.60
  • created or updated
    • “Security makes you a Better Developer”
    • “When Failed Tests are Good”
    • “Creating Small Tests”
    • “Creating Abuse Cases”
    • “Deliver PenTest reports using JIRA”
    • “Email is not an Official Communication Medium”
    • “Making it expensive to do dangerous actions”
• v0.59
  • added section on:
    • “Getting Assurance and Trust from Application Security Tests”
    • “Hyperlink everything you do”
    • “Developer Teams Need Budgets”
    • “Developers Should be Able to Fire Their Managers”
    • “Every Bug is an Opportunity”
    • “Code Confidence Index”
    • “Chained threat models”
• v0.58
  • Started refactoring content into SecDevOps section
  • New content and multiple content fixes
• v0.56
  • Changed title to ‘SecDevOps Risk Workflow’ (from ‘Jira RISK Workflow’)
  • Added SecDevOps section
• v0.55
  • First contribution via PR
  • added license details (CC BY 4.0)
• v0.54
  • Major section on Security Champions
• v0.52
  • Major content changes including import of multiple chapters that were in the Quality Book
  • bumping to version 0.50 due to the current volume of content
• v0.11
  • Added automation using leanpub-book-site tool
  • Lots of content added, first pictures in chapters
• v0.10 (Oct 2016)
  • Created Git repo and hooked Leanpub
  • Added first set of content files

Contributions

The following individuals helped in the creation of this book with suggestions, comments, and Pull Requests:

• Dave van Stein - https://github.com/davevs
• Ann-Marie Grace - https://github.com/Ambg05
• Stephen Porell - s.porell@comcast.net
• Erlend Oftedal - https://github.com/eoftedal
• Henrik Rossen Jakobsen - https://github.com/henrikrossen
• James Wharton - https://github.com/JamesWharton

Disclaimers

• (unit) Test - For me a test is anything that can be executed with one of these Unit Test Frameworks: https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
• RISK - Abuse the concept, found RISK to be the best one for the wide range of issues covered by AppSec, while being understood by all players
• 100% Code Coverage - not the summit, but base-camp (i.e. not the destination). 100% code is not enough, we really need 500% or more Code Coverage
• AppSec - Non Functional requirements - AppSec is about understanding and controlling the unintended behaviors of applications

License

All original content in this book is released under the Creative Commons Attribution 4.0 International (CC BY 4.0) license (see https://creativecommons.org/licenses/by/4.0/)

For reference this is what it means:

Printed version

If you are reading this on a digital device, here are some reasons why you might want to consider buying the printed book ¹:

• Better way to learn the concepts in this book (see “Physical Books are the best technology for reading”)
• better graphs and diagrams (which are easier to understand and cross reference on paper)
• helps funding the development of this and other books (that I’m working on)

This Book has a Dual Focus

This book has a dual focus. First, it looks at how application security (AppSec) fits within the security, development and operations (SecDevOps) world. Secondly, the book demonstrates how a risk workflow is the key to making the SecDevOps world work.

The first part of the book considers the actions and the activities that AppSec introduces into SecDevOps. These actions include testing, insurance, and using the techniques that are part of AppSec to improve SecDevOps.

The second part of the book details a powerful technique that gives meaning to the work of SecDevOps in an organization. The technique is a risk workflow, based on JIRA or GitHub, that captures and tracks every action and idea, and their implications, that are raised by SecDevOps. These ideas, actions, and implications must be considered, addressed, and accepted by the organization.

This book presents ideas, concepts, and suggestions to make the risk workflow work in the real world. The subject matter is entirely based on real-world experiences, real-world experiments, and real-world projects across small and large organizations.

Why GitHub and JIRA?

My current experience is that only GitHub and JIRA have the workflows and the speed that allow these risk workflows to be used properly in the real world.

I know there are other tools available that try to map this and create some UIs for risk workflows, but I believe that you need something very close to the way developers work. GitHub and JIRA meet this essential requirement, as they are both connected to the source code.

JIRA is more powerful from the point of view of workflows. JIRA allows you to have more complex workflows, which is quite interesting, and JIRA gives you a risk acceptance button, which is very powerful.

GitHub is simpler than JIRA, and can be easier and faster to use, although its reporting capabilities aren’t as built as JIRA’s.

These two tools are the only ones I have seen that can make this workflow perform in the real world.

1. Sec-DevOps

1.1 Concepts

1.1.1 SecDevOps or DevSecOps

• Which one is better
• Which one sends the better message
• DevSecOps seems to be getting a couple more google queries : https://www.google.co.uk/trends/explore?q=secdevops,devsecops
• DevSecOps has the advantage that puts Dev (Development) first (which is good since that is most important part)
• SecDevOps is a good extension of DevOps which is already a known practice
  • I also like the idea that when Security becomes embedded in the SDL and Sec-DevOps just becomes DevOps
• I really like the definitions here
• DevSecOps & SecDevOps - Is there a difference? https://www.linkedin.com/pulse/devsecops-secdevops-difference-kumar-mba-msc-cissp-mbcs-citp

1.1.2 Don’t blame the developers

A very common and dangerous misconception in the security world is that developers are responsible for any security vulnerabilities, or even security mistakes.

The developers wrote the code, but it is essential to understand the ecosystem and the environment that produced that code.

Most developers don’t control their schedule, their learning, or how much time they get to visualize and understand what they do. Therefore, to make them responsible for the code they are writing is a very dangerous and short-sighted analysis of their role.

The key influencers of code are the development and the business managers, and the business owner, who controls what gets done and is responsible for what gets created.

The other challenge is that the current development frameworks and IDEs (Integrated Development Environments) don’t reward the visualization and understanding of the side effects of code.

This means that when you are programming, it is very hard to understand the side effects of changing or adding a feature, or making a particular kind of implementation.

In such circumstances, how can we expect developers to be accountable and responsible for those code changes when they aren’t aware of the bigger picture?

There might be some cases where the developers should know better, but in my experience, those cases are very rare. My experience is that developers want to do the right thing but they don’t get the tools or even the allocation to do so.

Blaming developers creates a negative dynamic between security people and developers in an already adversarial blame culture. What we need to do before blaming the developers, is understand the ecosystem, and look at the reward system and the workflow. This is a much more positive way of looking at things.

1.1.3 Good resources on DevSecOps and SecDevOps

DevSecOps & SecDevOps - Is there a difference? https://www.linkedin.com/pulse/devsecops-secdevops-difference-kumar-mba-msc-cissp-mbcs-citp

SecDevOps

• https://www.reddit.com/r/secdevops/
• https://twitter.com/hashtag/secdevops
• SecDevOps: Embracing the Speed of DevOps and Continuous Delivery in a Secure Environment https://securityintelligence.com/secdevops-embracing-the-speed-of-devops-and-continuous-delivery-in-a-secure-environment/
• SecDevOps: Injecting Security Into DevOps Processes https://blog.newrelic.com/2015/08/27/secdevops-rugged-devops/
• SecDevOps - Source Code Review Consultants (at speed) https://www.seek.com.au/Job/32037282?cid=advlinkedin
• The 12 Days of SecDevOps http://blog.threatstack.com/the-12-days-of-secdevops
• Advanced Cloud Security and applied SecDevOps https://www.blackhat.com/us-16/training/advanced-cloud-security-and-applied-secdevops.html
• Swimming application security upstream with SecDevOps http://www.esg-global.com/blog/swimming-application-security-upstream-with-secdevops
• https://devops.com/tag/secdevops/
  • Murphy’s DevOps: Is Security Causing Things to Go Wrong? https://devops.com/2016/02/29/murphys-devops-is-security-causing-things-to-go-wrong/
  • Flash Mob Inflection: Rugged DevOps Revolution https://devops.com/2016/02/19/flash-mob-inflection-rugged-devops-revolution/
  • Security Breaks DevOps – Here’s How to Fix It https://devops.com/2015/07/08/security-breaks-devops-heres-how-to-fix-it/
  • The devOpsSec Dilemma: Effective Strategies for Social Networking https://devops.com/2015/05/28/devopssec-dilemma-effective-strategies-social-networking/
  • Automated Security Testing in a Continuous Delivery Pipeline https://devops.com/2015/04/06/automated-security-testing-continuous-delivery-pipeline/
  • It’s time security pros shake their DevOps fear, uncertainly, and doubt https://devops.com/2015/05/18/its-time-security-pros-shake-their-devops-fear-uncertainly-and-doubt/
  • ChatOps: Communicating at the speed of DevOps https://devops.com/2014/07/16/chatops-communicating-speed-devops/
• SecDevOps: The New Black of IT http://www.slideshare.net/CloudPassage/sec-devops-webinar-deck

DevSecOps

• https://github.com/devsecops/awesome-devsecops (more links)
• https://twitter.com/hashtag/devsecops
• http://www.devsecops.org/
• http://www.devseccon.com/
• A primer on secure DevOps: Why DevSecOps matters http://techbeacon.com/devsecops-foundations
• Why Did We Need to Invent DevSecOps? http://blog.threatstack.com/why-did-we-need-to-invent-devsecops
• DevOpsSec Securing Software through Continuous Delivery http://www.oreilly.com/webops-perf/free/devopssec.csp
• DevSecOps: The Marriage of SecOps and DevOps www.tripwire.com/state-of-security/security-awareness/devsecops-the-marriage-of-secops-and-devops/
• A Look Back at DevOpsDays Austin 2016 http://blog.threatstack.com/a-look-back-at-devopsdays-austin-2016
• MASTERED DEVOPS? WHAT’S NEXT? DEVSECOPS, THAT’S WHAT! https://www.cloudreach.com/us-en/2014/11/devops-devsecops/?topic=aws
• Gartner: DevOps is good; DevSecOps is better http://searchcio.techtarget.com/tip/Gartner-DevOps-is-good-DevSecOps-is-better

1.1.4 History of Sec-DevOps

1.1.5 Making the Sec part invisble

• concept that a lot of the work done today (2016) in AppSec and SecDevOps is part of a transition into just DevOps
  • when security in embedded into the Dev SDL
  • when security is invisible
• Case-Study: The biggest advantages of Microsoft’s Security push has been the quality and robustness of their products, and big improvement on their SDL Story: IE body count from Renegades of the Empire book)
• add reference (and some content from) Secure coding (and Application Security) must be invisible to developers

1.1.6 Rugged Software

• https://www.ruggedsoftware.org/
• Rugged Manifesto
• add explanation of what it is (and its history)
• why it didn’t really work (as least according to the original expectations)
  • lack of community adoption
  • ‘Security Driven’ vs ‘Developer driver’
• The Docker case study
  • why Docker was so successful (as an idea and adoption)
• lessons learned

1.1.7 Using Artificial Intelligence for proactive defense

We need AI to understand code and applications. Our code complexity is getting to a level that we need to start to use artificial intelligence capabilities to understand it, and to get a grasp of what is going on, so we can create secure applications that have no unintended side effects.

As AI becomes much more commonplace, we should start to use it more to source code analysis and application analysis. Kevin Kelly has some very interesting analysis on the use of AI, where he discusses the idea that one of the next major revolutions will be where we start adding AI to everything, because the cost of AI will become so low that we will be able to add AI to many devices.

When you analyse an app, you should use everything you have. You should use static, dynamic, interactive, human, and increasingly you should use artificial intelligence to optimise your analysis.

When you are doing security analysis, you are dealing with a vast amount of data, displayed on a multi-dimensional graph. What you have is a graph of the relationships, of what is happening. You are looking for the connections, for the paths within the graph, that are made of what is really going on and what is possible.

Artificial intelligence technology can assist the human who will put context on those connections. I think we are a long way from being able to do this kind of analysis automatically, but if we can make the human’s job of reviewing the results easier, or even possible, that is a major step forward.

1.1.8 When Failed Tests are Good

When you make a code change, it is fundamental that every change you make breaks a test, or breaks something. You are testing for that behaviour; you are testing for the particular action that you are changing.

This means you should be happy when you make a change and the test fails, because you can draw confidence from knowing the side effects of the change. If you make a test change in one place and a couple of tests break, and you make another test change in a different place and fifty tests break, you get a far better sense of the impact of the changes that you made.

A more worrying prospect is when you make code changes but you don’t see any test failing, and nothing breaks. This means you don’t understand the side effects of the change that you just made. And tests should be teaching you the side effects of changes.

Broken tests are great when the test that you expect to break is the one that fails. The changes that you were expecting to make are the ones that happen, and that makes sense.

When a test breaks, and you review that code, you understand why the break happened, and you understand the side effect. If the test fix is fast, you have a very quick and effective loop of TDD.

Sometimes, I will write something and it passes, so I know the code is working the way it is supposed to work. But I will have a couple of other cases where the test fails, and this allows me to confirm that it is failing in a predictable and expected way.

Sometimes, I will codify those failures in tests, to have and to give the assurance that a break happened in a particular place, so I should change this place, or this is the fix that happened the way that I expected it to happen.

1.1.9 Why SecDevOps?

I like SecDevOps because it reinforces the idea that is an extension of DevOps. SecDevOps points to the objective that eventually we want the Sec part to disappear and leave us with DevOps.

Ultimately, we want an environment where 99% of the time, DevOps don’t care about security. They program in an environment where they either cannot create security vulnerabilities, or it is hard to create them, or it is easy to detect security issues when they occur.

This doesn’t mean you don’t need security teams, or AppSec experts. It doesn’t mean you don’t need a huge amount of work behind the scenes, and a huge amount of technology to create those environments.

You don’t make security invisible by getting rid of it. You make security invisible by automating the security checks, and by increasing visibility into what is going on.

At the moment, when we look at security activities, we often see security doing things that are the proper responsibility of development, or testing, or infrastructure, or documentation, or even management.

Anybody who works in AppSec for a while always finds themselves asking difficult questions. They interrogate the system rigorously, but the information they seek should already be known and available to them.

AppSec will often create tools to attack an application to visualize what is going on. I have had many experiences of spending time creating technology to understand the attack surface. Once that task is complete, I find a huge number of vulnerabilities, simply because a significant part of the application hadn’t been tested. The system owners, and the various players didn’t fully understand the application or how it behaved.

So, I think SecDevOps represents an interesting point in history, where we are trying to merge all the activities that have been going on in security with the activities that have been going on in DevOps so we can build an environment where we can create and deploy secure applications.

This relates closely to my ideas about using AppSec to measure quality. Ultimately, the quality of the application is paramount when we are trying to eliminate the unintended consequences of software.

DevSecOps initially sounds better because development goes first. But I agree with the view of DevSecOps as being more about applying development to security operations (SecOps).

This all ties together with the risk workflows that make things more connected and accountable.

1.1.10 Draft notes - DevOps

Stages of AppSec automation

Start with static analysis which don’t need a live environment to deploy the application

1. Have a CI that builds code automatically for every commit of every branch
2. Ran ‘a’ tool after build (doesn’t really matter which one what matters is that it uses the materials created on step 1)
   • use for example FindBugs with FindSecBugs ²
3. Find tune scan targets (i.e. exactly what needs to be scanned)
4. Filter results, customize rules to reduce noise
5. Gather reports and scans and put them on git repo
6. create a consolidated dashboard (for management and business owners)
7. add more tools
8. loop from 5

after this is mature, add a step to deploy the app to a live environment (or simulator)

1.2 Dev-Ops

1.2.1 Disposable IT infrastructure

Following the ‘everything is code’ concept, what you really want is an environment where your IT infrastructure is disposable. By this I mean that it should be very easy to rebuild, or recreate. Consider the developer’s environment, and how long it takes to create infrastructure. Then consider what happens to that infrastructure when the company hires more developers or the development team expands.

This is also an issue from a security point of view because it means that the developers, and even normal IT users, have a lot of black magic in their infrastructure. And if there is any Malware or malicious code installed in an application, then it means that it is in there for a certain period of time.

So, what you want is a situation where most of your infrastructure is automatically rebuilt. You want environments where developers’ laptops reboot every Monday, where you overhaul the infrastructure from the bottom up, and where business owners use Chromebooks so that every install is fresh.

Your infrastructure should be disposable, you shouldn’t care about it, and you should be able to easily rebuild, delete, or destroy, because that means all your data is backed up, and all your data is safe.

It also promotes the idea that in most scenarios, you shouldn’t be able to access all the data or assets that your current user role has access to (i.e. you should only have access to that you need to do the job at hand).

It would be great if we had a Git-based operating system, with native support for version control, even at the operating system level. This would provide full control of what is going on and what is changed (on install and over time), by using git diffs and branches.

1.2.2 Don’t kill the Ops

• be careful that DevOps is not a cost saving exercise, where it is seen as a way to kill/reduce Ops.
  • Dev + Ops -> DevOps -> Dev
• seen cases where there is an crazy idea that the ‘Ops’ team will be made redundant

1.2.3 Everything is code

It is critical to understand that everything that happens in an organisation, from the development, to the deployment, to the configuration, to the retirement of an app, is code.

This is tied to the concept of disposable infrastructure, where it is possible to rebuild everything from scratch automatically.

Since everything is code, everything should be versioned, stored in a Git repository, and tested.

Git is a very important component, since it allows the use of branches for managing multiple versions and deployments. The Git repo also captures all the configurations, settings, and mappings.

This is a big change, because a lot of what happens in any organisation is not documented, but instead is stored in somebody’s head, or in a script. The movement to use continuous integration workflows and embrace DevOps practices, represents a good opportunity to capture existing reality.

Basically, there should be no black-magic or non-versioned configuration, anywhere in the development and deployment pipeline.

1.2.4 History of DevOps

1.2.5 Horizontal DevOps

The best model I have seen for integrating DevOps in a company is where teams are created that span multiple groups. Instead of having a top-down approach to the deployment of operations, where you create the central teams, the standards, the bills, etc., and then push it down, the central DevOps team hires or trains DevOps engineers and then allocates them to each team.

The logic is that each team spends a certain amount of time with a DevOp engineer, who trains the teams in DevOp activities and best practices, and thereby embeds the best practices in the development life cycle.

The result is horizontal teams, which have several advantages. They have two masters: they answer to DevOps but they also answer to the team lead, and they share best practice. The creation of horizontal teams is a much better way to scale, and it encourages teams to collaborate. The teams know they aren’t required to spend all their time working in DevOps, and they know there is someone who can help them.

1.2.6 In DevOps Everything is Code

A common gap in DevOps workflows is, ironically, Application Security activities on the code the DevOps team writes (Secure coding, Static/Dynamic analysis, Threat Models, Security Reviews, Secure Coding Guidelines, Security Champions, Risk Workflows, etc…)

One cause for this gap is the fact that many DevOps teams come from network and infrastructure backgrounds, or network security backgrounds (i.e. traditional InfoSec), rather than from development (i.e. coding).

This leads to the lack of realization that every single configuration change, or environment setup scripts such as Chef/Puppet/Ansible files or AWS/Azure/G-Cloud setting, that exist in a DevOps pipeline, are actually code, and these need to be:

• versioned
• reviewed
• tested
• released
• rolled back
• and finally retired/deleted

This ‘DevOps code lifecycle’ needs everything that we talk about in AppSec.

What makes this ‘DevOps code’ even more important than ‘normal code’, is the fact that ‘DevOps code’ tends to run with full admin privileges. Any vulnerability in this code, any exploit or blind spot, any lack of settings or even malicious changes, will have a tremendous impact on the company’s risk profile.

You need to look at the build servers (Jenkins, Bamboo, TeamCity, Travis) and pay attention to what its status is, and what code is running on it.

I like the idea of a pristine build environment, completely isolated from other build servers and networked devices, and supported and maintained by the DevOps team. In there, application builds are created very cleanly, with very few side effects, and with a full understanding of what is going on. Ideally, the build service should have read-only access to certain dependencies, because it should not be modifying them.

The key paradigm shift here is to realize that we need all the AppSec practices in everything that happens in the DevOps world. A good example is an identity solution I participated in where the code itself, in isolation, was rock solid.

But when the code went to production it became a horror show. There were bugs that only manifested themselves after being deployed. Functionality was missing, and at the end, the original brief was very far from where it was supposed to be.

The main problem was a major lack of integration testing, namely end-to-end testing where you can test code as it happens in production.

This is the power of DevOps.

You need to be able to take everything apart, and rebuild everything that is interconnected. You can use surrogate dependencies to mock-up certain external dependencies, but the idea is to have as much code as possible running at any given time.

To make all this work, the developer needs to have access to reliable and rebuildable environments. The developer needs a development workflow that proceeds from running a change that executes in a purely traditional unit test, to full-blown integration tests running on live instances.

From here the code/configuration change can proceed from running from a single machine, to a local or quasi-local environment, to the cloud environment that runs with the back-end and front-end components, and with all the components integrating.

1.2.7 Infrastructure as code

• all changes and scripts and clicks:
  • are code
  • need to be stored in git
  • need tests (can run locally and in production)

1.2.8 Patch speed as a quality metric

Making small changes to an application (i.e. Patching) should be an easy and smooth event.

However, if it is a problem, it means that there are issues either in deploying the app, or in rebuilding the required infrastructure.

These issues need to be addressed sooner rather than later, specially since they affect the risk appetite for management and business owners to allow changes and refactoring to occur.

For patching, you really want to seen an Kanban workflow with a healthy fast, smooth flow and an low WIP (Work In Progress)

What happens when rollbacks are required?

Before an incident even happens, open a JIRA ticket on ‘lack of patching’. This way, when a lack of patching causes problems, you are ready to capture the incidents that occur.

History, and any experience of Murphy’s Law, should provide evidence on the cost of non-patching. A lack of patching acts as a canary in a coal mine, in so far as it points to bigger problems up ahead. You should think of patching as a type of fire drill.

When is patching easy

Patching is easy in the following circumstances:

• when it is easy to run the new version in an isolated environment (with and without production data)
• when there is a good understanding of what is going to change (files, ports, behavior, inter-dependencies, schema’s changes)
  • i.e. a diff with the current version of everything
• when there are Tests (e2e and integration) that run though the affected systems’ success criteria and confirm any changes (i.e. the side effects)
• when it easy to rollback to previous version(s)

1.2.9 Performing root-cause analysis

• some environments make it hard to perform root-cause analysis, because it is seen as a blame exercise, instead of a learning opportunity
• Root-cause analysis are key for any bug (namely one the ones with Security implications)
  • lack of ‘Root-cause analysis’ is a risk (which needs to be accepted)
    • means business owners doesn’t want to spend the time to find other (similar issues)
• add story of project manager that asked “please don’t find more security issues during the retest (they are out of scope)”
• in an DevOps world, it is key for the root-cause analysis that there is a way replicate the issue (in an simulated environment).
  • the end-result of a root-cause analysis should be a Test (very close to a script) (that passes when the issue can be replicated correctly and reliably)

1.2.10 Run Apps Offline

The ability to run applications offline, i.e. without live dependencies of QA servers, or even live servers, is critical in the development process. That capability allows the developers to code at enormous speed, because usually the big delays and expensive calls are to those services that allow all sorts of versioning, and all sorts of development techniques to occur. The ability to run your apps offline also signifies that the application development environment has matured to a level where you now have, or have created, mocked versions of your dependencies.

Ideally, the faster you can run the dependencies, even running them as real code, the better. The important thing is to be sure you are running them locally, without a network connection, and without an umbilical code to another system.

Of course you will always test against those other systems. The test that you are going to run locally against that mocked version should also pass against the live system. If the dependencies don’t pass, you have a problem and you have to fix it.

When you are developing, you should have all the required dependencies at that moment in time. This makes a huge difference when you are developing a new version, or a new feature, where you can already simulate how those dependencies behave. This allows for much better demos, a much faster interaction and development loop, and, ultimately, it accelerates developers’ productivity.

The answers to the following simple questions are also very important: Can you run your app offline? Can you run your service offline? Can you run your application that you are coding offline? These are things that management can use to measure the pulse, and the quality, of the current development environment, and the continuous integration of particular teams.

So, if you have three teams that code in the way described above, and one team that doesn’t, you can guess which team is probably not shipping regularly. This same team will be unable to develop things very quickly, and it will be unable to respond quickly to customer and business requirements. More importantly, the team that doesn’t run apps offline will be the team that has to deal with a whole number of bugs and QAs every time it releases. The teams that run their apps offline, with QA cycles, won’t encounter these kinds of obstacles.

1.2.11 When devs are cloud admins

• When moving to the cloud, Devs are becoming SysAdmins
  • which is a massive issue and is creating a large number of security issues
• in some cases this move is literally ‘throwing the baby with the bath water’ where lack of innovation, speed and features from the local admin/IT teams, pushed the developers to have free reign in the cloud (like AWS) where they can create servers and VMs in seconds (vs days, weeks or months)

1.3 Sec DevOps patterns

• provide examples

2. Risk Workflow

2.1 Concepts

2.1.1 Abusing the concept of RISK

As you read this book you will notice liberal references to the concept of RISK, especially when I discuss anything that has security or quality implications.

The reason is I find that RISK is a sufficiently broad concept that can encompass issues of security or quality in a way that makes sense.

I know that there are many, more formal definitions of RISK and all its sub-categories that could be used, but it is most important that in the real world we keep things simple, and avoid a proliferation of unnecessary terms.

Fundamentally, my definition of RISK is based on the concept of ‘behaviors’ and ‘side-effects of code’ (whether intentional or not). The key is to map reality and what is possible.

RISKs can also be used in multi-dimensional analysis, where multiple paths can be analyzed, each with a specific set of RISKs (depending on the chosen implementation).

For example, every threat (or RISK) in a threat model needs to have a corresponding RISK ticket, so that it can be tracked, managed and reported.

Making it expensive to do dangerous actions

A key concept is that we must make it harder for development teams to add features that have security, privacy, and quality implications.

On the other hand, we can’t really say ‘No’ to business owners, since they are responsible for the success of any current project. Business owners have very legitimate reasons to ask for those features. Business wishes are driven by (end)user wishes, (possibly defined by the market research department and documented in a MRD (Market requirement Document). Saying no to the business means saying no to the customer. The goal, therefore, is to enable users to do what they want, with an acceptable level of risk.

By providing multiple paths (with and without additional or new RISKs) we make the implications of specific decisions very clear.

What usually happens is that initially, Path A might look easier and faster than either of Paths B or C, but, after creating a threat model, and mapping out Path A’s RISK, in many cases Path B or C might be better options due to their reduced number of RISKs.

It is important to have these discussions before any code is written and while there is still the opportunity to propose changes. I’ve been involved in many projects where the security/risk review only occurs before the project goes live, which means there is zero opportunity to make any significant architectural changes.

Release Management (RM) should be the gating factor. RM should establish quality gates based on business and security team minimum bar.

2.1.2 Accepting risk

Accepting risk is always a topic of discussion, mostly caused by lack of information. The JIRA Workflow aims to make this process more objective and pragmatic.

Risks can be accepted for a variety of reasons:

• the system containing the risk will be decommissioned in the short term, and does not handle or process sensitive information
• the risk can only occur under very specific circumstances
• the cost of fixing (both money and other resources) is much higher than the cost of fixing operational issues
• the business is okay with the probability of occurrence and its financial/commercial impact
• the system is still under development and it is not live
• the assets managed/exposed by the affected risks are not that important
• the risk represents the current state of affairs and a significant effort would be required to change it

Risks should never be accepted for issues that

• threaten the operational existence of the company (e.g. losing a banking license)
• have significant financial impact (e.g. costs, fines due to data leaks)
• have significant business impact (e.g. losing customers)
• greatly impact the company’s reputation (e.g. front page news)

Not being attacked is both a blessing and a curse. It’s a blessing for a company that has not gone through the pain of an attack, but a curse because it is easy to gain a false sense of security.

The real challenge for companies that have NOT been attacked properly, or publicly, and don’t have an institutional memory of those incidents, is to be able to quantify these risks in real business, financial, and reputational terms.

This is why exploits are so important for companies that don’t have good internal references (or case studies) on why secure applications matter.

As one staff member or manager accepts risk on behalf of his boss, and this action is replicated throughout a company, all the way to the CTO, risks are accumulated. If risks are accepted throughout the management levels of a company, then responsibility should be borne in a similar way if things go wrong. Developers should be able to ensure that it is their manager who will get fired. That manager should make sure that his boss will get fired, and so on, all the way to the CTO and the CISO.

• business owners (or who controls the development pipeline) make large number of decisions with very little side effects
• add explanation of why pollution is a much better analogy than ‘Technical Debt’ (in measuring the side effects of coding and management decisions).
  • Risk is a nice way to measure this polution
• ”…you are already being played in this game, so you might as well expose the game and tilt the rules in your favour…“ *

There is a video by Host Unknown, that lays out the consequences of thoughtlessly accepting risk. It is a comic piece, but with a very serious message. (Host Unknown presents: Accepted the Risk)

2.1.3 Can’t do Security Analysis when doing Code Review

One lesson I have learned is that the mindset and the focus that you have when you do security reviews are very different than when you work on normal feature and code analysis.

This is very important because as you accelerate in the DevOps world, it means that you start to ship code much faster, which in turn means that code hits production much faster. Logically, this means that vulnerabilities also hit production much faster.

In the past, almost through inertia, you prevented major vulnerabilities from propagating into production and being exposed to production data. Now, as you accelerate, vulnerabilities, and even maliciously introduced vulnerabilities, will be exposed into production more quickly.

This means that you must have security checks in place. The problem is, implementing those security checks requires a completely different mindset than implementing code reviews.

When you do a code review, you tend to visualize a slice of a model of the application. Your focus is fixed entirely on the problem at hand, and it is hard to think outside of that.

When you implement a security review, this approach becomes counter-intuitive, because many of the security reviews that you do are about following rabbit holes, and finding black spots. Your brain is not geared for this kind of work if you are more used to working on code reviews.

It doesn’t help that we still don’t have very good tests, which don’t focus on the behavior or the side effects of the components. Instead, tests tend to focus more on specific code changes which might be a subset of the behavior changes of the code change.

You need systems that can flag when something is a problem, or needs to be reviewed. Then, with a different mindset or even different people, or different times, go through the code and ask, “What are the unintended side effects? Does this match the threat model that was created?”

In a way, the point of the threat models is to determine and confirm the expected behavior. Ultimately, in security reviews you double-check these environments.

This requires a different mindset, because now you must follow the rabbit holes, and you must ask the following questions:

a] how does data get in here? b] what happens from here? c] who consumes this? d] how much do I trust this?

You ask the STRIDE questions, where you go through proofing, tampering, information disclosure, repudiation, denial of service, and you ask those questions across the multiple layers, and across multiple components. The better the test environment, and the better the technology you have to support you, the easier this task becomes. Of course, it becomes harder, if not impossible, when you don’t have a good test environment, or good technology, because you don’t have enough visibility.

Ideally, the static analysis should significantly help the execution of a security analysis task. The problem is, they still don’t expose a lot of the internal models and they don’t view themselves as tools to help with this analysis. This is crazy when you think about their assets.

2.1.4 Creating Small Tests

When opening issues focused on quality or security best practices (for example, a security assessment or a code review), it’s better to keep them as small as possible. Ideally, these issues are placed on a separate bug-tracking project (like the Security RISK one), since they can cause problems for project managers who like to keep their bug count small.

Since this type of information only exists while AppSec developers are looking at code, if the information isn’t captured, it will be lost, either forever, or until that bug or issue becomes active. It is very important that you have a place to put all those small issues, examples, and changes.

Capturing the small issues also helps to capture the high-level and critical security issues that are made of multiple low or medium issues. Their capture also helps to visualize the patterns that occur across the organization, for example, in an issue that effects dozens of teams or products.

This approach also provides a way to measure exactly what will occur during a quality pass or sprint, for example, when focusing on cleaning and improving the quality of the application.

The smaller the issue, the smaller the commit, the smaller the tests will be. As a result, the whole process will be smoother and easier to review.

It also creates a great two-way communication system between Development and AppSec, since it provides a great place for the team to collaborate.

When a new developer joins the team, they should start with fixing bugs and creating tests for them. This can be a great way for the new team member to understand what is going on, and how the application works.

Fixing these easy tests is also a good preparation for tackling more complex development tasks.

2.1.5 Creating Abuse Cases

Developer teams tend to focus on the ‘Happy Paths’, where everything works out exactly as expected. However, in agile environments, creating evil user stories linked to a user story can be a powerful technique to convey higher level threats.

Evil user stories (Abuse cases)

Evil user stories have a dependency with threat modeling and can be an effective way of translating higher level threats and mitigations. They can be imagined as a kind of malicious Murphy’s law ³.

Take for example a login flow. After doing a threat model on this flow one should have identified the following information:

Attackers: * registered users * unregistered users

Assets: * credentials * customer information

With this information, the following threats can be constructed:

• unregistered users bypassing the login functionality
• unregistered users enumerating accounts
• unregistered users brute-forcing passwords
• registered users accessing other users’ information

At the same time the team has constructed the following user stories that should be implemented: * “as a registered user I want to be able to login with my credentials so I can access my information”

Combined with the outcome of the threat model, the following evil user stories can be constructed: * “as an unregistered user I should not be able to login without credentials, and access customer information” * “as an unregistered user I should not be able to identify existing accounts and use them to attempt to login” * “as a user I should not be able to have unlimited attempts to log in and hijack someone’s account” * “as an authenticated user I should not be able to access other users’ information”

2.1.6 Deliver PenTest reports using JIRA

Using JIRA to deliver PenTest reports is much more efficient than creating a PDF to do so. Using JIRA allows for both a quick validation of findings, and direct communication between the developers and the AppSec team.

If customized properly, there are a number of tools that can act as triage tools between PenTest results and JIRA:

• ‘defect dojo’
• ‘bag of holding’
• ThreadFix
• Other…

Threat Modeling tools could also work well between PenTest results and JIRA.

2.1.7 Email is not an Official Communication Medium

Emails are conversations, they are not official communication mediums. In companies, there is a huge amount of information and decisions that is only communicated using emails, namely:

• risks
• to-dos
• non-functional requirements
• re-factoring needs
• post-mortem analysis

This knowledge tends to only exist on an email thread or in the middle of a document. That is not good enough. Email is mostly noise, and once something goes into an email, it is often very difficult to find it again.

If information is not at the end of a link, like on a wiki page, bug-tracking system or source control solution like Git, it basically doesn’t exist.

It is especially important not to communicate security risks or quality issues in email, where it is not good enough to say to a manager, ”… by the way, here is something I am worried about…“.

You should create and send a JIRA RISK ticket to the manager.

This will allow you to track the situation, the information collected and the responses; in short, you can track exactly what is going on. This approach also gets around the problem of someone moving to other teams or companies. Their knowledge of a particular issue remains behind for someone else to use if they need to.

Emails are not a way to communicate official information; they are just a nice chat, and they should be understood as such. Important official information, in my view, should be hyperlinked.

The hyperlinkability of a piece of information is key, because once it has a hyperlinked location, you can point others to it, and a track record is created.

The way I look at it, if information is not available on the hyperlink, it doesn’t exist.

The Slack revolution

There is a Real-time messaging revolution happening, driven by tools like Slack or Matter-most, which are quickly becoming central to development and operational teams (some still use old-school tools like Basecamp, Jabber, IRC, IM or Skype).

One of the real powers of this new generation of collaboration tools is the integration with CI/CD and their ability to become the glue between teams and tools.

The problem is that all data is short-lived, and will soon disappear (there are some limited search capabilities, which are as bad as email).

Using Wikis as knowledge capture

JIRA issues can also be hard to find, especially since they tend to be focused on specific topics.

Labels, queries and reports help, but the best model is capture their knowledge by linking to them on Wikis (e.g. MediaWiki or Confluence) or document management tools (e.g. Umbraco or Sharepoint). The idea is to document lessons learned, how-to’s, and best practice.

2.1.8 Good Managers are not the Solution

When we talk about risk, workflows, business owners making decisions about development, and QA teams that don’t write tests, we often hear the comment, “If we had good managers, we wouldn’t have this problem”.

That statement implies that if you had good managers, you wouldn’t have the problem, because good managers would solve the problem. That is the wrong approach to the statement. Rather, if you had good managers, you wouldn’t have the problem, because good managers would ask the right questions before the problem even developed.

These workflows aren’t designed for when you have a good manager, a manager who pushes testing, who demands good releases, who demands releases every day, or who demands changes to be made immediately.

These workflows are designed for bad managers (I use the term reluctantly). Bad managers are not knowledgeable, or they are exclusively focused on the short-term benefits of business decisions, without taking to account the medium-term consequences of the same decisions. This goes back to the idea of pollution, where the manager says “Just ship it now, and we will deal with the pollution later”. With start-ups, sometimes managers will even say, “Push it out or we won’t have a company”.

The risk workflow, and the whole idea of making people accountable is exactly because of these kinds of situations, where poor decisions by management can cause huge problems.

We want to empower developers, the technical guys who are writing code and have a very good grasp of reality and potential side effects. They are the ones who should make technical decisions, because they are the ones who spend their time coding, and they understand what is going on.

2.1.9 Hyperlink Everything you do

Whether you are a developer or a security person, it is crucial that you link everything you do to a location where somebody can just click on a link and hit it. Make sure whatever you do is hyperlinkable.

This means that what you create is scalable, and it can be shared and found easily. It forms part of a workflow that is just about impossible if you don’t hyperlink your material.

An email is a black box, a dump of data which is a wasted opportunity because once an email is sent, it is difficult to find the information again. Yes, it is still possible to create a mess when you start to link things, connect things, and generate all sorts of data, but you are playing a better game. You are on a path that makes it much easier in the medium term for somebody to come in, click on the link, and make sure it is improved. It is a much better model.

Let others to help you.

If you share something with a link, in the future somebody can proactively find it, connect to it and do something about it. That is how you scale. Once you send enough links out to people, they learn where to look for information.

Every time I write something that is more than a couple of paragraphs long I try to include a link so that my future self, or somebody else in the future, will be able to find it and propagate that information without my active involvement.

Putting information in a public hyperlinked position encourages a culture of sharing. Making information available to a wider audience, either to the internet or internally in a company, sends the message that it is okay to share.

Sharing through hyperlinking is a powerful concept because when you send information to somebody directly it is very unlikely that you will note on each file whether it is okay to share.

But if you put data on a public, or on an internal easy-to-access, system, then you send a message to other players that it is okay to share this information more widely. Sending that link to other people has a huge impact on how that idea, or that concept, will propagate across the company and across your environment.

The thing to remember is that you are playing a long game. Your priority is not to get an immediate response, it is to change the pattern, stage the flows.

2.1.10 Linking source code to Risks

If you add links to risk as source code comments, you deploy a powerful and very useful technique with many benefits.

When you add links to the root cause location, and all the places where the risk exists, you make the risk visible. This reinforces the concept of cost (i.e. pollution) when insecure, or poor quality, code is written. Linking the source code to risk becomes a positive model when fixes delete the comments. When the comments are removed, the AppSec team is alerted to the need for a security review. Finally, tools can be built that will scan for these comments and provide a ‘risk pollution’ indicator.

• add coffee-script example from ‘Start with tests’ presentation

2.1.11 Mitigating Risk

One strategy to handle risk is to mitigate it.

Sometimes, external solutions, such as a WAF, provide a more effective solution.

If a WAF fixed the issue, and there are tests that prove it, then it is a valid fix. In this case, it is important to create a new RISK which says “Number of security vulnerabilities are currently being mitigated using a WAF. If that WAF is disabled, or if there is a WAF bypass exploit, then the vulnerabilities will become exploitable”

2.1.12 Passive aggressive strategy

Keep zooming in on the answers to refine the scope and focus of the issue.

Not finding something is a risk, and not having time to research something is a risk.

Sometimes, it is necessary to open tickets that state the obvious: * when we are not using SSL (or have an HTTP to HTTPS transition), then we should open a risk to be accepted with: “Our current SSL protection only works when the attacker we are trying to protect against (via man-in-the-middle) is not there“

2.1.13 Reducing complexity

The name of the game is to reduce the moving parts of a particular solution, its interactions, inputs, and behaviours.

Essentially, the aim is to reduce the complexity of what is being done.

2.1.14 Start with passing tests

• add content from presentation

2.1.15 Ten minute hack vs one day work

• 10m hacks vs 1 day proper coding will create more RISK tickets and source comments
• I get asked this a lot from developers
• There has to be a quantitive difference, which needs to be measured

That code sucks

• Ok can you prove it?
  • Code analysers in IDE can help identify known bad patterns and provide awareness
• Why is it bad?

In new code legacy - if it also hard to refactor and change, then yes

2.1.16 The Pollution Analogy

When talking about risks, I prefer to use an pollution analogy rather than technical debt. The idea is that we measure the unintended negative consequences of creating something, which in essence is pollution.

In the past, pollution was seen an acceptable side effect of the industrial revolution. For a long time, pollution wasn’t seen as a problem in the same way that we don’t see security vulnerability as a problem today. We still don’t understand that gaping holes in our infrastructure, or in our code, are a massive problem for current and future generations.

We are still in the infancy of software security, where we are in the 1950s in terms of pollution. David Rice gave a great presentation⁴ where he talks about the history of pollution and how it maps perfectly with InfoSec and AppSec.

Finding a security vulnerability is questioning the entire development pipeline and quality control. How it is possible that these massive gaps, these massive security vulnerabilities and code patterns weren’t picked up before. Weren’t they understood by the development teams, by the testers, by QA, by the clients?

Even worse, can’t the current NOC and log monitoring detect those when those vulnerabilities are initially probed and ultimately exploited?

We need to make this pollution visible.

I view the proposed risk model as a way to measure that pollution, and to measure the difference between app A and app B.

One positive side of using the pollution analogy, is that security doesn’t need to say no all time. Instead of saying “You can’t do XYZ”, AppSec says “If you want to go for feature A in a specific timeline, in a specific scope with a specific brief, then you will have this residual pollution, and these unintended consequences”.

You could also have a situation where security says, “Well, if you go in ‘that’ direction, currently we don’t know what are its consequences or security site-effects, so the business owner will have to accept the risk that there is an unknown set of risks which are also not clearly understood”.

The path of using pollution as an analogy is an evolution from the current status quo. At least with it we can measure and label different apps using common metrics. But ideally we want to create a clean code that doesn’t pollute

This is analogous to the evolution of pollution.

The first phase of industrial pollution saw pollution as a necessary side effect of progress, with some fines for the worse offenders (the ‘polluter pay’ model).

The second phase established the green movement where business had to behave decently, pollute less and be rewarded by the market.

The third phase is most interesting to this analogy, because in the third phase companies started to produce things in different ways, which not only created better products, but also dramatically reduced or eliminated pollution.

This is where we want to get to in application security. We want to get to the point where we create software that is better, faster, cheaper and secure (i.e. without pollution)

2.1.17 Triage issues before developers see them

Issues identified in vulnerability scans, code analysers, and penetration tests, must be triaged by the AppSec team to prevent overloading developers with duplicate issues or false positives. Once triaged, these issues can be pushed to issue trackers, and exposed to the developers.

Unfortunately, this is still an underdeveloped part of the AppSec pipeline. Even commercial vendors’ UI don’t really support the customisation and routing of the issues they ‘discover’.

The following tools are attempting to do just that:

• Bag of Holding https://github.com/aparsons/bag-of-holding
• Defect Dojo https://github.com/OWASP/django-DefectDojo
• ThreadFix

With these triage tools, AppSec specialists can identify false positives, accepted risks, or theoretical vulnerabilities that are not relevant in the context of the system. This ensures that developers should only have to deal with the things that need fixing.

Create JIRA workflow to triage raw issues

The creation of a security JIRA workflow for the raw issues to act as a triage tool, and push them to team boards after reviewing, would be another good example of the power of JIRA for workflows.

2.1.18 Using AppSec to measure quality

• add content from presentation AppSec and Software Quality
• academic research
  • https://insights.sei.cmu.edu/sei_blog/2016/06/using-quality-metrics-and-security-methods-to-predict-software-assurance.html

2.1.19 Employ Graduates to Manage JIRA

One of the challenges of the JIRA RISK workflow is managing the open issues. This can be a considerable amount of work, especially when there are 200 or more issues to deal with.

In large organizations, the number of risks opened and managed should be above 500, which is not a large quantity. In fact, visibility into existing risks starts to increase, and improve, when there are more than 500 open issues.

The solution to the challenge of managing issues isn’t to have fewer issues.

The solution is to allocate resources, for example to graduates, or recently hired staff.

These are inexpensive professionals who want to develop their careers in AppSec, or they want to get a foot in the company’s door. Employing them to manage the open issues is a win-win situation, as they will learn a great deal on the job, and they will meet a lot of key people.

By directing graduate employees or new hires to manage the open issues, developers’ time is then free to fix the issues instead of maintaining JIRA.

The maintenance of issues is critical for the JIRA RISK workflow to work, because one of its key properties is that it is always up-to-date and it behaves as a ‘source of truth’.

It is vital that risks are accepted and followed up on and that issues are never moved into the developer’s backlog where they will be lost forever.

We can’t have security RISKs in backlog; issues must either be fixed or accepted.

2.1.20 Risk Dashboards and emails

It is critical that you create a suite of management dashboards that map the existing security metrics and the status of RISK tickets:

• Open, In Progress
• Awaiting Risk Acceptance, Risk Accepted
• Risk Approved, Risk not Approved, Risk Expired
• Allocated for Fix, Fixing, Test Fix
• Fixed

Visualizing this data makes it real and creates powerful dashboards. The dashboards need to be provided to all players in the software development lifecycle.

You should create a sequence of dashboards that starts with the developer (who maps the issues that he is responsible for), then his technical architect, then the business owner, the product owner … all the way to the CTO and CISO.

Each dashboard clarifies which risks they are responsible for, and the status of application security for those applications/projects.

To reinforce ownership and make sure the issues/risks don’t ‘disappear’, use either the full dashboard, or specific tables/graphs, to create emails that are sent regularly to their respective owners.

The power of risk acceptance is that each layer accepts risk on behalf of the layer above. This means that the higher one is in the company structure, the more risks are allocated and accepted (all the way to the CISO and CTO).

The CISO plays a big part in this workflow, since it is his job to approve all ‘Risk Accepted’ issues (or raise exceptions that need to be approved by higher management).

This workflow drives many security activities because it motivates each player to act in his own best interests (which is to reduce the number of allocated Risk Accepted items).

The idea is to social-engineer the management layer into asking the developers to do the non-function-requirements tasks that the developers know are important, but seldom get the time to do.

2.2 For Developers

2.2.1 5000% code coverage

A big blind spot in development is the idea that 100% code coverage is ‘too much’.

100% or 99% code coverage isn’t your summit, your destination. 100% is base camp, the beginning of a journey that will allow you to do a variety of other tests and analysis.

The logic is that you use code coverage as an analysis tool, and to understand what a particular application, method, or code path is doing.

Code coverage allows you to answer code-related questions in much greater detail.

Let’s look at MVC Controller’s code coverage as an example. We must be certain that there is 100% code coverage on all exposed controllers. Usually, there are traditional ‘unit tests’ for those controllers which give the impression that we have a good coverage of their behavior. However, that level of coverage might not be good enough. You need the browser automation, or network http-based QA tests, to hit 100% of those controllers multiple times, from many different angles and with all sorts of payloads.

You need to know what happens in scenarios when only a couple of controllers are invoked in a particular sequence. You need to know how deep into the code we get, and what parts of the application are ‘touched’ by those requests or workflows.

This means that you don’t need 100% code coverage. Instead, you need 1000%, or 2000%, or even 5000% coverage. You need a huge amount of code coverage because ultimately, each method should be hit more than once, with each code-path invoked with multiple values/payloads.

In fact, the code coverage should ideally match the use cases, and every workflow that exists.

This brings us to another interesting question. If you take all client-accepted use cases which are invoked and simulated with tests, meaning that you have matched the expected ‘user contract’, (everything the user expects to happen when interacting with an application) if the coverage of the application is not at 100%, what else is there?

Maybe the executed tests only covered the happy paths?

Now let’s add the abuse and security use cases. What is the code coverage now?

If, instead of 100%, you are now at 70% coverage of a web application or a backend API, what is the rest of the code doing?

What code cannot currently be invoked using external requests? In most cases, you will find dead code or even worse, high-risk vulnerabilities that exist there, silently waiting to be invoked.

This is the key question to answer with tests: “Is there any part of the code that will not be triggered by external actions?”. We need to understand where those scenarios are, especially if we want to fix them. How can we fix something if we don’t even understand where it is, or we are unable to replicate the abuse cases?

2.2.2 Backlog Pit of Despair

In lots of development teams, especially in agile teams, the backlog has become a huge black hole of issues and features that get thrown into it and disappear. It is a mixed bag of things we might want to do in the future, so we store them in the backlog.

The job of the product backlog is to represent all the ideas that anyone in the application food chain has had about the product, the customer, and the sales team. The fact that these ideas are in the backlog means they aren’t priority tasks, but are still important enough that they are captured. Moving something into the backlog in this way, and identifying it as a future task, is a business decision.

However, you cannot use the backlog for non-functional requirements, especially the ones that have security implications. You have to have a separate project to hold and track those tickets, such as a Jira or a GitHub project.

Security issues or refactoring needs, still exist, regardless of whether the product owner wants to pursue them, whether they are a priority, or whether customers are asking for them.

Security and quality issues should either be in a fixed state, or in a risk acceptance state.

The difference is that quality and security tickets represents reality, whereas the backlog represents the ideas that could be developed in the future. That is why they have very different properties, and why you shouldn’t have quality and security tickets in the backlog Pit of Despair ⁵.

2.2.3 Developer Teams Need Budgets

Business needs to trust developer teams.

Business needs to trust that developers want to do their best for their projects, and for their company.

If business doesn’t learn to trust its developer teams, problems will emerge, productivity will be affected and quality/security will suffer.

A great way to show trust is to give the developer team a budget, and with it the power to spend money on things that will benefit the team.

This could include perks for developers such as conference attendance, buying books, or buying things which are normally a struggle to obtain. It is often the case that companies will purchase items for a team when requested, but first the team must struggle to overcome the company’s apathy to investing in the developer team. The balance of power never lies with the team, and that imbalance makes it hard to ask for items to be purchased. Inconsistencies are also a problem: sometimes it can be easier to ask for £5,000, or for £50,000, than it is to ask for £50 .

Companies need to treat developer teams as the adults they are, and they need to trust them. My experience in all aspects of organizations is that it is difficult to spend money.

When you spend money, especially in an open-ended way, your expenditure is recorded and it becomes official. If your investment doesn’t yield good results for the company, you will be held accountable.

Therefore, allocating a budget to the developer teams will keep the teams honest, and will direct their focus to productive investments. Purchases could range from buying some tools for the developers, to buying a trip, or even to outsourcing some work to a freelancer.

Spending on the operational expenses for the team will yield benefits both to developer teams and to business.

2.2.4 Developers should be able to fire their managers

Many problems developer teams deal with arise from the inverted power structure of their working environment. The idea persists that the person managing the developers is the one who is ultimately in charge, responsible, and accountable.

That idea is wrong, because sometimes the person best-equipped to make the key technological decisions, and the difficult decisions, is the developer, who works hands-on, writing and reading the code to make sure that everything is correct.

A benefit of the ‘Accept Risk’ workflow, is that it pushes the responsibility to the ones that really matter. I’ve seen cases when upper-layers of management realise that they are not the ones that should be accepting that particular risk, since they are not the ones deciding on it. In theses cases usualy the decision comes down to the developers, who should use the opportunity to gain a bigger mandate to make the best decisions for the project.

Sometimes, a perverse situation occurs where the managers are no longer coding. They may have been promoted in the past because they used to be great programmers, or for other reasons, but now they are out of touch with programming and they no longer understand how it works.

Their job is to make the developers more productive. They work in customer liaison, they manage the team and its results, they organise, review, handle business requirements and expectations, and make sure everything runs smoothly. That is the job of the manager, and that manager also acts as the voice of the developer team.

This situation promotes inefficiencies and makes the managers more difficult to work with. They don’t want to share information, but they do want to take ownership of developers’ work or ideas that they didn’t have themselves. This environment gets very political very fast, and productivity is effected.

The manager I describe above should ideally be defending the developer team, and should act like an agent for that team. Logically, a developer, or a group of developers, should therefore have the power to nominate, appoint, and sack the manager if necessary.

The developers should hold the balance of power.

Developers should also be able to take decisions on pay, perks, and budgets. Business should treat the developer teams as the grownups they are, because the developer teams are ultimately accountable for what is created within the company.

2.2.5 Every Bug is an Opportunity

The power of having a very high degree of code coverage (97%+) is that you have a system where making changes is easy.

The tests are easy to fix, and you don’t have an asymmetric code fixing problem, where a small change of code gives you a nightmare of test changes, or vice versa.

Instead, you get a very interesting flow where every bug, every security issue, or every code change is an opportunity to check the validity of your tests. Every time you make a code change, you want the tests to break. In fact you should worry if the tests don’t break when you make code changes.

You should also worry if the tests break at a different location than expected. This means that the change you made is not having the desired effect. Either your tests are wrong, or you don’t have the right coverage, because you expected the test to break in one location but it broke somewhere else, maybe even on a completely different test that happened to pass that particular path.

To enforce the quality of your test, especially when you have a high degree of coverage, use every single bug fix, and every single code change as opportunities to confirm that your tests are still effective.

Every code change gives the opportunity to make sure that the understanding of what should change is what did change. Every bug allows you to ask the questions, “Can I replicate this bug? Can I correctly identify all that is needed to replicate the creation, or the expectation of this bug?”

2.2.6 Every project starts with 100% code coverage

Every application starts with 100% code coverage because by definition, when you start coding, you cover all the code.

You must ensure that you have a very high degree of code coverage right from the beginning. Because if you don’t, as you add code, it is easier to let code coverage slip, and then you realise that large chunks of your code base are not tested (any code that is changed without being tested, is just a random change).

Reasons why code coverage slips:

• code changes are not executed using tests
• code coverage is not being used as a debug tool
• code coverage is not shown in the IDE in real-time
• demo or go-live deadline
• mocks and interfaces are used to isolate changes (ironic side effect)
• certain parts of the application are hard to test
• weak testing technology and workflows
• tests are written by separate teams (for example separate dev and QA teams)
• lack of experience by developers, testers and management on the value and power of 100% code coverage

All of these are symptoms of insecure development practices, and will have negative side effects in the medium term.

It is very important to get back to 100% code coverage as soon as possible. This usually will require significant efforts in improving the testing technology, which is usually where the problems start (i.e. lack of management support for this effort)

Some of unmanaged languages, or even managed languages (like Java with the way exceptions are handled), have difficulties in measuring certain code paths. There is nothing wrong in changing coding patterns or conventions to improve an tool’s capabilities, in this case the code coverage technology’s ability to detect code execution paths.

It is very important that the development team has the discipline and habit of always keeping the application on 100% code coverage. Having the right tools and worlflows will make a massive difference (for example Wallaby for Javascript or NCunch for .NET)

For the teams that are really heavy into visualisation and measuring everything, it is very powerful to use dashboards like the ones created by Kibana to track code coverage in real-time (i.e. see test’s execution as they occur in multiple target environments: dev, qa, pre-prod and prod).

2.2.7 Feedback Loops

The key to DevOps is feedback loops. The most effective and powerful DevOps environments are environments where feedback loops, monitoring, and visualizations are not second-class citizens. The faster you release, the more you need to understand what is happening.

DevOps is not a silver bullet, and in fact anyone saying so is not to be trusted. DevOps are a profound transformation of how you build and develop software.

DevOps are all about small interactions, small development cycles, and making sure that you never make big changes at any given moment. The feedback loop is crucial to this because it enhances your understanding and allows you to react to situations very quickly.

The only way you can react quickly is if you have programmatically codified your infrastructure, your deployment, and your tests. What you want is automation of your understanding, and of your tests. The feedback loops you get make this possible.

The success stories of DevOps are all about the companies who moved from waterfall style, top-down development, to a more agile way of doing things. The success of DevOps has also allowed teams that were agile but weren’t working very well, to ship faster.

I like the idea of pushing as fast as possible to production. It is essential to have the two pipelines I mention elsewhere in the book, where, if you codify with specific patterns that you know, you go as fast as possible. If you add changes that introduce new patterns or change the attack surface, you must have a separate pipeline that triggers manual or automated reviews.

The feedback loops allow you to understand this. The feedback loops allow you to understand how to modify your attack surface, how to change how your application behaves, and how to change key parts of how everything works.

2.2.8 Make managers accountable

Managers need to be held accountable for the decisions they make, especially when those decisions have implications for quality and security. There are several benefits to increased accountability. Organisations will be encouraged to accept risk, and, before they click on the ‘Accept Risk’ button, managers will be compelled to ‘read’ the ticket information. However, the introduction and development of a more accountable management structure is a long game, to be played out over many months, or even years. Sometimes, it requires a change in management to create the right dynamic environment.

Usually, a new manager will not be too enthusiastic when he sees the risks that are currently accepted by a business and are now his responsibility. He knows he has a limited window, of a few months perhaps, where he can blame his predecessor for any problems. After that time has passed, he ‘owns’ the risk, and has accepted it himself.

This workflow creates historical records of decisions, which supports and reinforces a culture of accountability.

In agile environments, the Product Owner is responsible for making the right decisions.

2.2.9 Risk lifecycle

Here are the multiple stages of Risks:

 1 digraph {
 2   size ="3.4"
 3   node [shape=underline]
 4 
 5   "Risk"                   -> "Test (reproduces i\
 6 ssue)" [label = "write test"]    
 7   "Test (reproduces issue)" -> "Risk Accepted"   \
 8        [label = "accepts risk"]
 9   "Test (reproduces issue)" -> "Fixing"          \
10        [label = "allocated for fix"]  
11   "Test (reproduces issue)" -> "Regression Tests"\
12        [label = "fixed"]
13 }

It is key that a test that replicates the issue reported is created.

The tests result for each of the bottom layers mean different things:

• Risk Accepted : Issues that will NOT be fixed and represent the current accepted Risk profile of the target application or company
• Fixing: Issues that are currently being actively addressed (still real Risks)
• Regression Tests: Past issues that are now being tested to confirm that they no longer exist. Some of these tests should run against production.

2.2.10 Risk profile of Frameworks

Many frameworks work as a wrapper around a ‘raw’ language and have built in mechanisms to prevent vulnerabilities.

Examples are:

• Rails for Ruby
• Django for Python
• AngularJS, Ember, ReactJS for javascript
• Razor for .Net

Using these frameworks can help less experienced developers and act as a ‘secure by default’ mechanism.

This means that when using the ‘secure defaults’ of these frameworks, there will be less Risk tickets created. Ideally Frameworks will make it really easy to write secure code, and really hard (and visible) to write dangerous/insecure code (for example how AngularJS handles raw HTML injection from controllers to views).

This can backfire when Frameworks perform complex and dangerous operations ‘automagically’. In those cases it is common for the developers to not really understand what is going on under the hood, and high risk vulnerabilities will be created (sometimes even using the code based on the Framework’s own code samples)

• Add case-studies on issues created by ‘normal’ framework usage:
  • SpringMVC Auto-Binding (also called over-posting)
  • Razor controls vulnerable to XSS on un-expected locations (like label)
  • Ember SafeHtml
  • OpenAM SQL Query Injection on code sample (and see in live app)
  • Android query SQL Injection (on some parameters)

2.2.11 Security makes you a Better Developer

When you look at the development world from a security angle, you learn very quickly that you need to look deeper than a developer normally does. You need to understand how things occur, how the black magic works, and how things happen ‘under the hood’. This makes you a better developer.

Studying in detail allows you to learn in an accelerated way. In a way, your brain does not learn well when it observes behavior, but not cause. If you are only dealing with behavior, you don’t learn why something is happening, or the root causes of certain choices that were made in the app or the framework.

Security requires and encourages you to look deeper, to find ways to learn the technology, to understand how the technology works, and how to test it. I have a very strong testing background because I spend so much time trying to replicate things, to make things work, to connect A to B, and to manipulate data between A and B.

What interests me is that when I return to a development environment, I realize that I have a bigger bag of tricks at my disposal. I always find that I have greater breadth, and depth, of understanding of technology than a lot of developers.

I might not be the best developer at some algorithms, but I often have a better toolkit and a more creative way of solving problems than more intelligent and more creative developers. The nature of their work means that their frames of reference are narrower.

Referencing is important in programming because often, once you know something is possible, you can easily carry out the task. But when you don’t know if something is possible, you must consider how you investigate the possibilities, how long your research will take, and how long it might be before you know if something will succeed or not. Whereas, if you know something is possible, you know that it is X hours away and you can see its evolution, or if it is a lost cause.

Having those references makes a big difference because when you look at a problem it is important to know which rabbit holes you can follow and which ways will create good and sound solutions and which ones don’t. This is especially important when we talk about testing the app: there tends to be a lot of interesting lateral thinking and creative thinking required to come up with innovative solutions to test specific things.

My argument is that when you do application security, you become a better developer. Your toolkit expands, your mind opens, and you learn a lot. In the last few months alone, I had to learn and review different languages and frameworks such as Golang, Spark, Camel, Objective C, Swift and OpenIDM.

After a while learning new systems becomes easier, and the ability to make connections makes you a better developer because you are more skilled at observing concepts and technologies.

2.2.12 Test lifecycle

• explain test flow from replicating the bug all the way to regression tests

1 digraph {
2   size ="3.4"
3   node [shape=box]
4   "Bug"-> "Test Reproduces Bug"
5        -> "Root cause analysis"
6        -> "Fix"
7        -> "Test is now Regression test"
8        -> "Release"
9 }

2.2.13 The Authentication micro-service cache incident

A good example of why we need tests across the board, not just normal unit tests, but integration tests, and tests that are spawned as wide as possible, is the story of a authentication module that was developed as an re-factoring into a separate micro-service.

When the module was developed, it contained a high degree of code coverage, in fact it had 100% unit test coverage. The problems arose when it went live, and several issues occurred. One of the original issues occurred because the new system was designed to improve the way the database or the passwords were stored. This meant that once it was fully deployed some of existing dependent services stopped working.

For example, one of the web services used by customer service stopped working. Suddenly, they couldn’t reset passwords, and the APIs were no longer available. Because end-to-end integration tests weren’t carried out when the website started, some of the customer service failed, which had real business impact.

But the worst-case example occurred when a proxy was used in one of the systems, and the proxy cached the answer from the authentication service. In that website, anybody could log in with any password because the cache was caching the ‘yes you are logged in’.

This illustrates the kind of resilience you want these authentication systems to have. You want a situation where, when you connect to a web service, you don’t just get, for example, a 200 which means okay, you should get the equivalent of an cross-site-request-forgery token. Ideally you would get an transaction token in the response received, i.e. “authentication is ok for this XYZ transaction, this user and here is an user object with user details”.

(todo: add example of 2fA exploit using Url Injection)

When you make a call to a web service and ask for a decision, and the only response you get is yes or no, this is quite dangerous. You should get a lot more information.

The fundamental problem here is the lack of proper end-to-end testing. It is a QA and development problem. It is the fact that in this environment you cannot spin off the multiple components. If you want to test the end-to-end user log in a journey, you should be able to spin off every single system that uses that functionality at any given time (which is what DevOps should also be providing).

If that had been done from the moment the authentication module was available, these problems would have been identified months in advance, and several incidents, security problems, and security vulnerabilities would have been prevented from occurring.

2.2.14 Using Git as a Backup Strategy

When you code, you inevitably go on different tangents. Git allows you to keep track of all those tangents, and it allows you to record and save your progress.

In the past, we used to code for long periods of time and commit everything at the end. The problem with this approach is that sometimes you follow a path to which you might want to return, or you might follow a path that turns out to be inefficient. If you commit both early and often, you can keep track of all such changes. This is a more efficient way of programming.

Of course, before you make the final commit, and before you send the push to the main repository, you should clean up and ensure the code has a certain degree of quality and testing.

I find that every time I code, even for a short while, my instinct is to write a commit, and keep track of my work. Even if I add only a part of the code, or use the staging capabilities to do some selective commits, I find that Git gives me a much cleaner workflow. I can be confident in my changes from start to end, and I rarely lose any code or any snippet with which I was happy.

2.2.15 Creating better briefs

Developers should use the JIRA workflow to get better briefs and project plans from management. Threat Models are also a key part of this strategy.

Developers seldom find the time to fulfil the non-functional requirements of their briefs. The JIRA workflow gives developers this time.

The JIRA workflow can help developers to solve many problems they have in their own development workflow (and SDL).

2.3 For management

2.3.1 Annual Reports should contain a section on InfoSec

Annual reports should include sections on InfoSec and AppSec, which should list their respective activities, and provide very detailed information on what is going on.

Most companies have Intel dashboards of vulnerabilities, which measure and map risk within the company. Companies should publish that data, because only when it is visible can you make the market work and reward companies. Obliging companies to publish security data will make them understand the need to invest, and the consequences of the pollution that happens when you have rented projects with crazy deadlines and inadequate resources, but somehow manage to deliver.

The ability to deliver in such chaotic circumstances is often due to the heroic efforts of developers, who work extremely hard to deliver high quality projects, but get rewarded only by being pushed even more by management. This results in extraordinary vulnerabilities and risks being created and bought by the company. Of course, the company doesn’t realize this until the vulnerabilities are exploited.

In agile environments, it’s important to provide relative numbers such as:

• Risk issues vs velocity
• Risk issues vs story points

By analyzing these numbers over time, the tipping point, where quality and security are no longer in focus, can become painfully clear.

2.3.2 Cloud Security

One way in which cloud security differs from previous generations of security efforts, such as software security and website security, is that in the past, both software and website security were almost business disablers. The more features and the more security people added, the less attractive the product became. There are very few applications and websites that make the client need more security to do more business, which results in the best return on investment.

What’s interesting about cloud security is that it might be one of the occasions where security is a business requirement, because a lack of security would slow down the adoption rate and prevent people from moving into the cloud. Accordingly, people care about cloud security, and they invest in it.

While thinking about this I realized that the problem with security vulnerabilities in the cloud is that any compromise doesn’t just affect one company, it affects all the companies hosted in the cloud. It’s a much bigger problem than the traditional scenario, where security incidents resulted in one company being affected, and the people in that company worked to resolve the problem. When an incident happens in the cloud, the companies or assets affected are not under the control of their owners. The people hosting them must now manage all these external parties, who don’t have any control over their data, but who can’t work because their service is down or compromised. The problem is horizontal; a Company A-driven attack will affect Companies B, C, D, E, F, all the way to X, Y, Z. As a result, it’s a much tougher problem to solve.

While catastrophic failures are tolerated in normal websites and applications, cloud-based worlds are much less tolerant. A catastrophic failure, where everything fails or all the data is compromised or removed, means the potential loss of that business. That hasn’t happened yet, but doubtless it will happen. Cloud service companies will then have to show that they care more about security than the people who own the data.

Cloud providers care more than you

One of the concepts that Bruce Schneier talked about at the OWASP IBWAS conference in Spain is that a cloud service cares more about the security of their customers than the customer does. This makes sense since their risk is enormous.

In the future, cloud companies will be required to demonstrate this important concept. They should be able to say, “Look, I have better systems in place than you, so you can trust me with your data. I can manage more data for you than you can”. This will be akin to the regulatory compliance that handles data in the outside world.

One way to do this is with publishing of RISK data and dashboards (see OWASP Security Labelling System Project).

You can imagine a credit card company wanting or needing to demonstrate this. But you can also see the value of it to the medical industry, or any kind of industry that holds personal information. If a company can’t provide this type of service it must outsource the service, but to be able to do that the security industry must become more transparent. We need a lot more maturity in our industry, because companies need to show that they have adequate security controls. It is not sufficient to be declared compliant by somebody who goes for the lowest common denominator, gets paid, and tells the company it’s compliant.

Genuinely enhanced visibility of what’s going on will allow people to measure what’s happening, and then make decisions based on that information. The proof of the pudding will not be how many vulnerabilities the cloud companies have, but how they recover from incidents.

The better a company can sustain an attack, the better that company can protect data. A company who says, “I received x, y, z number of attacks and I was able to sustain them and protect them this way” is more trustworthy than the company that is completely opaque. The key to making this work is to create either technology, or standards of process, that increase the visibility of what is going on.

2.3.3 Code Confidence Index

Most teams don’t have confidence in their own code, in the code that they use, in the third parties, or the soup of dependencies that they have on the application. This is a problem, because the less confidence you have in your code, the less likely you are to want to make changes to that code. The more you hesitate to touch it, the slower your changes, your re-factoring, and your securing of the code will be.

To address this issue, we need to find ways to measure the confidence of code, in a kind of Code Confidence Index (COI).

If we can identify the factors that allow us to measure code confidence, we would be able to see which teams are confident in their own or other code, and which teams aren’t.

Ultimately, the logic should be that the teams with high levels of Code Confidence are the teams who make will be making better software. Their re-factoring is better, and they ship faster.

2.3.4 Feedback loops are key

A common error occurs when the root cause of newly discovered issues or exploits receives insufficient energy and attention from the right people.

Initially, operational monitoring or incident response teams identify new incidents. They send the incidents are to the security department, and after some analysis the development teams receive them as tickets. The development teams receive no information about the original incident, and are therefore unable to frame the request in the right perspective. This can lead to suboptimal fixes with undesired side effects.

It is beneficial to include development teams in the root cause analysis from the start, to ensure the best solutions can be identified.

2.3.5 Getting Assurance and Trust from Application Security Tests

When you write an application security test, you ask a question. Sometimes the tests you do don’t work, but the tests that fail are as important as the tests that succeed. Firstly, they tell you that something isn’t there today so you can check it for the future. Secondly, they tell you the coverage of what you do.

These tests must pass, because they confirm that something is impossible. If you do a SQL injection test, in a particular page or field, or if you do an authorization test, and it doesn’t work, you must capture that.

If you try something, and a particular vulnerability or exploit isn’t working, the fact that it doesn’t work is a feature. The fact that it isn’t exploitable today is something that you want to keep testing. Your test confirms that you are not vulnerable to that and that is a powerful piece of information.

You should have tests for both the things you know are problems, and the things you know are not problems. Such tests give you confidence that you have tested your application for certain things.

Be sure that you have enough coverage of those tests. You also want to relate this to code coverage and to functionality, because you want to make sure that there is direct alignment between what is possible on the application and what is invoked by the test they should match.

The objective is to have much a stronger assurance of what is happening in the application, and to detect future vulnerabilities (created in locations that were not vulnerable at the time of the last security assessment).

2.3.6 I don’t know the security status of a website

Lack of data should affect decision-making about application security.

Recently, I looked at a very interesting company that provides VISA compatible debit-card for kids, which allows kids to get a card whose budget can be controlled online by their parents. There is even a way to invest in the company online via a crowdfunding scheme.

I looked at this company as a knowledgeable person, able to process security information and highly technical information about the application security of any web service. But I was not able to make any informed security decision about whether this service is safe for my kids. I couldn’t understand the company’s level of security because they don’t have to publish it and, therefore, I don’t have access to that data.

As a result, I must take everything on the company’s website at face value. And because there is no requirement to publish any real information about their product, the information given is shaped by the company’s marketing strategy. I have no objective way of measuring whether the company has good security across their SDL, has good SecDevOps capabilities, if are there are any known security risks I should be aware, or more importantly, if my kids’ data is protected and secure.

This means that my friends who recommended that service to me are even worse off than I am. They are not security savvy users and they can only rely on the limited information given on the company’s website.

If there are three or four competing services at any moment in time, they will not be able to compete on the security of their product. It is not good enough if a company only invests in security in case security becomes a problem, or causes embarrassment in the future. It is like saying, “Oh, let’s not pollute our environment because we might get caught”.

In business today, security issues are directly related to quality issues. Application security could be used to gain a good understanding of what is going on in the company, and whether it is a good company to invest in, or a good company to use as a consumer.

This approach could scale. If I found problems, or if data was open, I could publish my analysis, others could consume it, and this would result in a much more peer-reviewed workflow for companies.

This reflects my first point: if I can’t understand a company’s level of security because they don’t have to publish it, this should change. And if it does, it will change the market.

Having a responsible disclosure program or public bug bounty program is also a strong indicator of quality and security.

In fact, a company that doesn’t have a public bug bounty program is telling the world that they don’t have an AppSec team.

2.3.7 Inaction is a risk

Lacking the time to perform ‘root cause analysis’, or not understanding what caused a problem, are valid risks in themselves.

It is key that these risk are accepted

This is what makes them ‘real’, and what will motivate the business owner to allocate resources in the future. Specially when a similar problem occur.

2.3.8 Insurance Driven Security

• insurance companies are starting to operate in the AppSec work
• they will need objective way to measure RISK
• workflows like this one could provide that

2.3.9 Is the Decision Hyperlinked?

I regularly hear the following statements: “The manager knows about it”, “I told you about this in the meeting”, “Everyone is aware of this”, and so on. However, if a decision is not in a hyperlinkable location, then the decision doesn’t exist. It is vital that you capture decisions, because without a very clear track of them, you cannot learn from experience.

Capturing decisions is most important for the longer term. When you deal with your second and third situations, you start building the credibility to say, “We did this in the past, it didn’t work then, and here are the consequences. Here is the real cost of that particular decision, so let’s not repeat this mistake”.

It is essential to do postmortems and understand the real cost of decisions. If a comment is made along the lines of, “Oh, we don’t have time to do this now because we have a deadline”, after a huge amount of manpower and time has been spent fixing the problem, you need to be able to say, “Was that decision the correct one? Let’s now learn from that, and really quantify what we are talking about”.

Completing this exercise will give you the knowledge to say the next time, “We need a week, or two weeks, or a month to do this”. Or you could say, “Last time we didn’t do this and we lost six months”. So, it is key not only to capture the decisions, but also to ensure you do a very good postmortem analysis of what happens after risks are accepted.

When there are negative consequences because of a bad decision, such as security issues or problems of technical debt, it is important that the consequences are hyperlinked back to the original issue for future reference.

In a way, the original issues are the foundations of the business case for why a problem occurred, and why you don’t want to repeat the problem in the future.

2.3.10 Measuring companies’ AppSec

Reviewing a company’s technology allows us to understand its future. As more and more companies depend on their technology, and as more investment is directly connected to software developed by companies, there is a need for companies to provide independent analysis of their technology stacks.

Typically, this is something an analyst does, where they should be measuring both the maturity of the company’s software development, and the maturity of its environment. The analyst should use public information to compile their evidence, and they should then use this evidence to support their data. Having good evidence means having explicit data on the record which clearly supports the analysis.

The power of the analyst is that they can take the time, and have the ability to create peer-reviewed, easy-to-measure, easy-to-understand analysis of companies.

Let’s look at investments, as it is now easier and easier for people to invest in companies. If your idea is to invest in technology at the earliest stages of a company’s growth, you want to be really sure it is going in the right direction. You should be able to get good metrics of what is going on, and you need a good understanding of what is happening inside these companies. Questions to ask include the following:

• Are they using an old technology stack?
• How much are they really investing in it?
• How much are they really busking the whole thing? (because you know that is something that can easily happen, where marketing paints a very different rosy picture of reality)
• Are they having have scalability problems caused by past architectural decisions?
• Are they able to refactor the code and keep it clean/lean and focused on the target domain models?

If you have a company which is growing very fast, it makes a massive difference if the company is on a scalable platform or not.

In the past, companies could afford to let their site ‘blow up’ and then address problems after they occurred. That is no longer possible, due to the side effects, and particularly the security side effects, of any crashes or instability. Today, the more successful a company becomes, the more attackers will focus on it. If there are serious security vulnerabilities, they will be exploited faster than the company can address them, and this will affect real users and real money. We can see this happening with cars and the IoT (Internet of Things).

There is still a huge amount of cost in moving away from products/technologies, which is I call the ‘lock-in index’, and is whatever ‘locks’ the customer to the software.

The idea is that investors and users should have a good measure of how ‘locked-in’ a company is, and the technological quality of the application/service they are about to invest into.

2.3.11 OWASP Top 10 Risk methodology

• add details about Risk methodology that in included with the OWASP Top 10
• mention other Risks methodologies
  • applying OWASP ASVS allows for a more risk based approach
  • OwaspSAMM and BSIMM can measure the maturity level of an organisation

2.3.12 Relationship with existing standards

It is important have a good understanding of how a company’s Risk profile maps to existing security standards alike PCI DSS, HIPAA, and others.

Most companies will fail these standards when their existing ‘real’ RISKs are correctly identified and mapped. This explains the difference between being ‘compliant’ and being ‘secure’.

Increasingly, external regulatory bodies and laws require some level of proof that companies are implementing security controls.

To prevent unnecessary delays or fines, these requirements should be embedded in the process in the form of regular scans, embedded controls (e.g. in the default infrastructure), or in user stories.

For example, in DevOps environments all requirements related to OS hardening should be present in the default container. Development teams should not have to think about them.

2.3.13 Responsible disclosure

Having a responsible disclosure process and/or participating in programs like HackerOne or Bugcrowd have several advantages:

• it shows a level of confidence to the customers
• it provides additional testing cycles
• it sends a message that they have an AppSec program

2.3.13.1 JIRA workflow for bug bounty’s submissions

• add diagram (from XYZ)
  • real world example of using an JIRA workflow to manage submissions received

2.3.13.2 Bug bounties can backfire

http://www.forbes.com/sites/thomasbrewster/2016/07/13/fiat-chrysler-small-bug-bounty-for-hackers/#58cc01f4606f

2.3.14 Third party components and outsourcing

• these dependencies need to be managed
• lots of control lost to 3rd parties
• open source and close source are very similar
  • at least with open source there is the chance to look at the code (doesn’t mean that somebody will)
  • companies need developers that are core contributors of FOSS
• outsource code tends to have lots of ‘pollution’ and RISKs that are not mapped (until it is too late)
  • liability and accountability of issues is a big problem
  • see OWASP Legal project for an idea of AppSec clauses to include in outsourcing contracts.
• copyright infringes will become more of a problem in the future; this can be hard to identify in 3rd party components

2.3.14.1 Tools to help manage dependencies

• OWASP Dependency checker can identify vulnerable components in use
• several commercial tools exist that can identify copyright infringements in components
  • Veracode has it
  • Sourceclear
  • Sonar (check which version)
  • nsp node security
  • … add more (see 13 tools for checking the security risk of open-source dependencies)

2.3.15 Understand Every Project’s Risks

It is essential that every developer and manager know what risk game they are playing. To fully know the risks, you must learn the answers to the following questions:

• what is the worst-case scenario for the application?
• what are you defending, and from whom?
• what is your incident response plan?

Always take advantage of cases when you are not under attack, and you have some time to address these issues.

2.3.16 Using logs to detect risks exploitation

Are your logs and dashboards good enough to tell you what is going on? You should know when new and existing vulnerabilities are discovered or exploited. However, this is a difficult problem that requires serious resources and technology.

It is crucial that you can at least detect known risks without difficulty. Being able to detect known risks is one reason to create a suite of tests that can run against live servers. Not only will those tests confirm the status of those issues across the multiple environment, they will provide the NOC (Network Operations Centre) with case studies of what they should be detecting.

2.3.16.1 Beware of the security myth

Often, no special software or expertise is needed to identify basic, potential, bad behavior. Usually, companies already have all the tools and technology they need in-house. The problem is making those tools work in the company’s reality. For example, if someone accesses 20 non-existing pages per second for several minutes, it is most likely they are brute-forcing the application. You can easily identify this by monitoring for 404 and 403 errors per IP address over time.

2.3.17 Using Tests to Communicate

Teams should talk to each other using tests. Teams usually have real problems communicating efficiently with each other. You might find an email, a half-baked bug-tracking issue opened, a few paragraphs, a screen shot, and if you are lucky, a recorded screencast movie.

This is a horrible way to communicate. The information isn’t reusable, the context isn’t immediately clear, you can’t scale, you can’t expand on the topic, and you can’t run it automatically to know if the problem is still there or not. This is a highly inefficient way to communicate.

The best way for teams to communicate is by using Tests.

Both within and across teams; top-down and bottom-up, from managers and testers to teams.

Tests should become the lingua franca of companies. Their main means of communication.

This has tremendous advantages, since in order for it to work, it requires really very good test APIs, and very good execution environments.

One must have an easy-to-develop, easy-to-install, and easy-to-access development environment in place, something that very rarely occurs.

By making tests the principal way teams communicate, you create an environment that not only rewards good APIs, it rewards good solutions for testing the application. Another advantage is that you can measure how many tests have been written across teams and you can review the quality of the tests.

Not only is this a major advantage to a company, it is also a spectacular way to review changes. If you send a test that says, “Hey! I would like XYZ to occur”, or “Here is a problem”, then the best way to review the quality of the change is to review the test that was modified in order to solve the problem.

In this scenario, you have a test that reflects the fixed state of a behaviour, and you have a test that communicates the changes you want to make. A review of both tests is often the best way to evaluate if the requested changes have been made and are working as desired.

2.3.18 Who is actually making decisions?

One of the interesting situations that occurs when we play the risk acceptance game at large organisations, is how we are able to find out exactly who is making business and technical decisions.

Initially, when a ‘Risk Accepted’ request is made, what tends to happen is that the risk is pushed up the management chain, where each player pushes to their boss to accept risk. After all, what is at stake is who will take responsibility for that risk, and in some cases, who might be fired for it.

Eventually there is a moment when a senior director (or even the CTO) resists accepting the risk and pushes it down. What he is saying at that moment in time, is that the decision to accept that particular risk, has to be made by someone else, who has been delegated (officially or implicitly) that responsibility.

In some cases this can now go all the way down to the actual developer/team doing the coding. Paradoxically, usually the developer didn’t realise until that moment, that he is one that is actually deciding how and when to fix it (or not).

Developers often hold a huge amount of power, but they just don’t know it.

Playing the risk acceptance game, and identifying who is deciding on risk, is a way of empowering the actual decision-maker. Once the person realises their role and power, they can make sure they have realistic time-scales to fix the issue accordingly, and when required, make the case for more time, more resources, or even a more defined role as a decision-maker.

This exercise is very important because, until you discover who clicks on the ‘Accept Risk’ button, there is little knowledge about who (if anybody at all) is making important decisions.

Risk issues can only be on one of two moving paths: ‘Fixing path’ and ‘Risk Acceptance path’. Since deciding not to act on a particular issue is a decision in itself, the Risk workflow makes sure that issues don’t ‘disappear’ in informal conversions, emails or documents.

This is also a good example why Pollution is a better analogy than Technical Debt. Features that are pushed with un-realistic deadlines or bad briefs will create a higher number of Risk tickets (i.e. pollution) which will have to be accepted by the business owners (who agreed on the development brief and time-scales)

2.4 For security Teams

2.4.1 Defensible findings

A big lesson I have learned from my consulting work is that every time you present evidence of a security issue or a vulnerability, you must have highly defensible findings.

So, I always use the same thinking: I must present data and information that are completely accurate and truthful.

This is very important, because if you present a report, especially if you present in an adversarial situation, where you have a team on the other side that doesn’t want to hear your findings, or will dispute every single line of them, everything must be true.

It doesn’t matter whether you are talking to the developer, the business owner, or the board; everything you do, and everything you say must be evidence-based.

And you can produce great evidence, but if finding number 17 contains exaggerations, or is less than watertight, that is exactly what your audience will focus on. And your credibility (and your hard work) will go down the drain.

Therefore, you must make sure that your approach is both pragmatic and objective. Without objectivity, you will lose your credibility. You must be certain that the statements that you have make sense from a business point of view. You can’t say the house is on fire if it isn’t on fire. You can’t say this is a highly critical vulnerability, when at the end of the day a compromise won’t bring the business down or cause too many problems.

In security, you do have vulnerabilities, data compromises, data confidentiality or integrity issues that will bring the business down, so you must put the problem you describe into perspective.

One of the things I find very interesting is that from my position in security and risk analysis I can express strong opinions about architectural and design issues. I give those opinions from my point of view: “Here are the side effects, here are a couple of threat models on features, look at the difference between A and B. You can go with that architecture, fine, but here are the side effects. If you go with this different architecture it is much simpler, there are fewer things to do, and less difficult situations to deal with”.

I have learned that everything you do must be supported by evidence, so you can prove your point and defend your findings. Ensure everything is hyperlinked, because when you hyperlink you build a body of evidence.

Having a thorough list of issues, for example via 500+ JIRA risk’s issues, is key to make this work. You start to build historical evidence that you can use to say “I worked on the same problem in the past, here are the patterns we should follow. Although I can’t prove why X, Y, Z is a problem today, this A, B, C issue had similar characteristics and it eventually blew up. So if you are going to accept the risks, be aware that the last time this happened it didn’t end well”.

It is a passive-aggressive way to deal with a problem. You aren’t saying “Don’t do it!”, but you are drawing their attention to the potential consequences of taking a particular course of action. They can choose to ignore your advice, but this is where you can elevate the issue, and make it a decision for the business owner.

2.4.2 Do security reviews every sprint

If you have an agile development environment, you need to implement security procedures and security reviews at the end of every sprint. In the period between the sprint finishing and going live, you need to do a push to get a sense of whether the original threats and issues, that were highlighted in the threat model, were done, or exist, in a verifiable way.

The central AppSec team shouldn’t do this test.

The target application security champion(s) should do this ‘smaller’ review, in their one-day-a-week allocated to AppSec activities. They should only ask the central AppSec team for help if they need it.

When you create a threat model before the application or feature is built, you know in advance which apps will need a more in-depth security review or analysis. This will depend on the size of the changes, what is being changed, or the assets being handled. This allows scheduling of the more experienced and knowledgeable AppSec professionals, who can be internal or external entities.

Note that these ‘sprint reviews’ are not meant to replace a final security push, or, when required, pentests (i.e. Application Security assessments).

2.4.3 Risk Workflow for Software Vendors

A software vendor is someone who delivers a software application that is executed by a client. The same concept applies to web applications and web apps, but let’s start by looking at a traditional software package.

The risk workflow in this case is very important, and there are multiple angles to consider. Let’s start with a simple one.

The first items to consider are the issues that evolve during the development of the software. Already, two types of risks exist. There are the risks that exist on the application, which should be known and captured on the risk register. The business owner must accept these risks, because ultimately he/she must decide how to prioritize the risks, and whether to fix them or not, depending on the priorities of the business.

Secondly, there are the risks that the company is willing to disclose to their customers. A mix between legal, governance, business and even marketing must decide on disclosure, because in most companies, security is still a marketing-driven exercise, as there is no regulation or requirement to disclose the truth.

There will be a list of security issues that the business will be willing to expose to customers, specially to clients that have signed NDAs, since those clients cannot disclose or talk about those issues publicly.

If you work on the software producer’s security team, you should know every risk that exists. To do so, you must go through the process of capturing, prioritizing, and understanding information, and then you must convince your business owners to accept the risk that is the flow.

Another grey area of responsibility are the insecure-by-design features, that are enabled by default, or are so key to the value of the software, that most clients will enable them.

This process raises a lot of interesting, ethical dilemmas. If you have a risk that no customer will notice, and is unlikely to be attacked, and nobody will pay attention to it, do you need to fix it?

Of course, you know you should fix the risk because it is the right thing to do. But, you also know that business is about making decisions, and about making risk decisions, and a lot of companies would choose not to address this kind of issue.

In fact, there is even a perverse reward system in some companies, where staff get rewarded for not knowing in situations where they plead ignorance of a problem and therefore evade accountability.

I would argue that ignorance is no longer a valid excuse, especially if you are dealing with vulnerabilities or problems that are widely known, and have been disclosed in different places.

If you are a client of those software packages, the situation is tricky because if the product is open-sourced, you have the code and you can do a security review. Of course, this doesn’t mean you are going to do a security review, but the option is there.

In most traditional products, the code is proprietary and you won’t be able to access it. You are therefore dependent on the vendor for information, which is usually limited. Your only option is to pay for an independent security review, which is something I have often done.

When you do discover a security problem with a product, you log it in your own risk register, where you acknowledge the problems that existed, and you discovered, in that product.

You report the problem to the vendor. Of course, the vendor should already know about the problem, and should have a solution for it. If they don’t know about it, this highlights the fact that they don’t have a risk process, and they weren’t paying attention to that problem.

If you are the vendor, and you receive those issues disclosed to you, you are now in a very weak position. Your client now has more power, and more leverage, than you do. In a way, you have switched from being the senior player, to the junior player, at the table.

A big problem we have as an industry is the fact that when client A discloses a problem, the vendor has no legal responsibility to disclose it to client B. The vendor can do a business and a marketing exercise for client A to keep him happy, give him more licenses, and control the damage limitation because they don’t have to tell all the other clients.

The client should ask the following questions:

a] Do you know about this? b] How many more of these do you have? c] Why didn’t you tell me in the first place?

If clients asked these questions, and vendors answered them, we would have a much better working market, and a much better model.

If I am working for the client, and we find issues, I ask a lot of questions on how fit-for-purpose the software is, especially if I start to find basic vulnerabilities and I know that the vendor doesn’t have a security team, or a good security posture.

This is an interesting, multi-dimensional problem and the risk workflows, and the mapping of the risks, will be a great way to measure this problem.

If you look at the insurance industry, and its ability to understand the real risk creative applications, although one might argue that not all these risks should be disclosed publicly, at least until they are fixed, the industry should at least disclose how many they have identified.

It goes back to the concept of a labelling system, where you create labels for software. In the same way that a label on a food product lists its ingredients, security labels on software products should list the security vulnerabilities that exist within.

2.4.4 Create an Technology Advisory Board

One of the biggest challenges in Agile and DevOps environments is the adoption rate of new technologies. In order to be as agile as possible there is a tendency to adopt new technology whenever it appears to have an advantage.

Common examples are cloud technology, analytic tools, continuous integration tools, container technology, web platforms and frameworks, and client-side frameworks.

To prevent the adoption of immature, insecure, or privacy violating components it is important to review desires and proposed solutions. The technology advisory board should take up this role. It should consist of people with security, privacy, and (some) legal knowledge.

It is important to not make this a new ‘Change advisory board’ with monthly review sessions doing a complete business impact analysis. Rather it should be in the form of a guild that is able to identify the maturity of the technology and possible impact on the ecosystem when things go wrong.

It can also guard for implementing multiple tools with the same purpose.

In this way the total ecosystem can be as lean and secure as possible.

2.5 JIRA RISK Workflow

2.5.1 Capture knowledge when developers look at code

It is vital that when a developer is looking at code, he can create tickets for ‘things noticed’ without difficulty. For example, ‘things noticed’ include methods that need refactoring, complex logic, weird code, hard-to-visualize architecture, etc. If this knowledge is not captured, it will be lost.

The developer who notices an issue, and opens a ticket for the issue, will be unable to do anything about it at that moment in time, since he will already be focused on resolving another bug.

Instead, more junior developers, graduate employees or interns could take responsibility for opening and managing these tickets.

They could even try to address the issues in the first instance, because the developer is responsible for merging the PRs.

2.5.2 Describe Risks as Features rather than as Wishes

When opening a risk JIRA ticket, it is essential to describe the exact behavior of that issue as a feature, rather than describing what you would like to see happening (i.e. your wish list).

For example:

• instead of saying ‘application should encode XYZ value’, you should say ‘XYZ value is not encoded’
• instead of saying ‘application shouldn’t be vulnerable to XSS or SQL injection’, you should say ‘application is vulnerable to SQL injection’. In this case, SQL Injection is a feature of the application, and while the application allows SQL Injection, the application is working as designed (whether that is intended or not is a different story).

When we describe vulnerabilities, we describe features, because vulnerability is a feature of an application.

If an application has a direct object reference vulnerability (OWASP Top 10), then that is a feature that allows User A to access user B data (by design, using the capabilities of the application).

For each of these cases, you should open risk tickets, since those risks represent existing features. Sometimes you open multiple risks for the same issue, allowing technical and business audiences to understand what is going on (SQL Injection doesn’t mean a lot to management, but ‘Access and modify all customer data’ does).

I remember a funny story where we found SQL injection in a pentest, and when we presented the findings, the business responded: ’… well, that is not a critical issue, we have good backups, so that SQL injection is not dangerous’. When we asked ‘what if we can drop all tables?’ , they said ‘We can recover from that very fast, no problem.’

We argued ‘well … we can modify data’ and they came back with, ‘We have read-only access and we can protect it from there.’. But finally, when we stated ‘we can log in as any user with a typical SQL Injection payload of: or 1=1’ that connected the dots, and the business said ‘we will fix that ASAP’

The reason that example clicked was because we showed them how to bypass the business logic of the application using the SQL Injection ‘feature’. They could tolerate, to a degree, data corruption or content changes. However, they reacted when they saw that the SQL Injection could bypass the application’s business logic and break their non-repudiation capabilities (i.e. they would lose the ability to understand what a user did on the site).

2.5.3 Git for security

• Why Git and DVCS (Distributed Version Control Systems (check name)) are so important for Quality and Security
• Why migrating to git is a good idea
• Analogy with Docker

How git workflows can be used for security

• add explation

git and svn

• The Git Svn story
  • ‘Kinda’ the same workflows, just different
  • Git developers to decision to make the opposite decision of Svn
    • why? … not just a dig at Svn. concept represents different approaches and focus
  • speed of git checkouts
  • virtual file systems
  • nightmare of moving files in svn ()
  • the power of one .git folder

2.5.4 Issue tracking solution

The issue tracking solution you choose is highly important.

To ensure maximum productivity always perform the following tasks:

• copy and paste your screenshots
• hyperlink your issues
• use markdown or wikitext

2.5.5 Risk accepting threat model

If you have trouble getting developer teams to create threat models, or to spend time on those threat models, then the solution is to make them accept the risk incurred from not having a threat model for the application.

The idea is not to be confrontational. Instead, stating that a feature has no threat model is a very pragmatic, focused, and objective statement.

The idea is that the developer team must accept that they don’t have a threat model. The logic is to create a ticket that says there is no threat model, and the ticket will be closed when the threat model is created. Alternatively, if the developers and their management team don’t want to spend the time creating a threat model, they must accept the risk of not having one.

This can be difficult to accept, but it’s an important part of the exercise.

2.5.6 Storing risk issues on JIRA

Is a JIRA ticket system a security risk

• Are they zero days?
• Is it increasing the attack surface
• ‘If it is on fire, fix it and document it later, if not on fire, document it’
• … better to know about it and to make it explicit that it will be exposed
• … in reality that info already exists on JIRA (maybe not so obvious)
• Monitoring the JIRA access could provide info about an attacker

2.5.7 The smaller the ticket scope the better

For bugs and tasks, the smaller the bug the better.

Having many small bugs and issues can be an advantage for the following reasons:

• easier to code
• easier to delegate (between developers)
• easier to outsource
• easier to test
• easier to roll back
• easier to merge into upstream or legacy branches
• easier to deploy

It is better to put them in a special JIRA project(s) which can be focused on quality or non-functional requirements.

Of course, this needs to be rational and kept in context. You should only create a couple of each instance/pattern, particularly when they are not being fixed. In such cases, create a ‘holding ticket’ that will store references to all the individual issues, which is good for systemic vulnerabilities.

You should also aggregate issues in Stories.

2.5.8 Key concepts

The following are the key concepts of this workflow:

• All tests should pass all the time
• Tests that check or confirm vulnerabilities should also pass
• Business owners must understand the risks of their decisions, and they must click on the ‘accept risk’ button

This workflow is about empowerment and assigning responsibility; it is not about apportioning blame.

Although I use JIRA for these examples, these concepts also apply to any bug-tracking system.

2.6 JIRA Technologies

This section covers the multiple technologies used

2.6.1 Confluence

• Atlassian wiki solution
• Tight integration with JIRA
• useful for reporting and Threat modeling
• used to create the materials used in weekly/monthly reports to owners of RISKs (i.e. the ones that have accepted the Risk)

2.6.2 JQL Query Language

• how it works
• security implications
• power examples of queries

2.6.3 Jira components

• used to map issues to specific projects
• allow easy filer per project

2.6.4 Copy and paste of images

• only available in the most recent version of JIRA but a feature that makes a massive difference
• the file attachment of images also works quite well

2.6.5 JIRA dashboards

• how to create them
• pie chart and issues list

2.6.6 JIRA Filters

• how to use them
• saving them

2.6.7 JIRA Kanban boards

• really good to track specific projects
• show couple examples
• add how to create on

2.6.8 JIRA Labels

• show examples
• how to use them
• common workflow
• using them to map OWASP Top 10 issues

2.6.9 JIRA workflows

• explain what they are what makes them powerful
• key components of it
• a good workflow tells a story and guides the user down specific paths

Tidy up your diagrams

It is important that your diagrams have layouts that look good and make sense.

The Diagram UI allows a bit of flexibility on where the Stages appear, so make use it and create nice diagrams

• show where they can be seen in the UI
• makes big difference when they are easy to read and understand
• I hide the labels since I find that they are harder to position and don’t provide that much more information
• add example of two diagrams diagrams. Same content: one messy and one clean
  • note how the clean one is much easier to read

Naming of states

• This is very important since they need to convey the desired action
• names have to be short or they don’t look good in the UI

Other workflows

Add screenshots for: - bug bounty - development - other …

A simple workflow

• add example of a simple workflow

2.7 Security Champions

• The Security Champions concept
• What is an Security Champion and what do they do?
• Becoming A Security Champion
• Do you have an heartbeat, you qualify!
• If you don’t have an Security Champion, get a mug
• Make sure your Security Champions are given time
• Making Security Champions AppSec Specialists
• Involvement in senior technical discussions
• Security Champions Don’t Take it Personally
• Supported by central AppSec
• Public references to Security Champions teams

2.7.1 AppSec memo from God

Having a board-level mandate is very important since it underlines the importance of AppSec.

The best way to provide a mandate to the existing AppSec team is to send a memo to the entire company, providing a vision for AppSec and re-enforcing its board-level visibility.

Sometimes called the ‘Memo from God’, the most famous one is Bill Gates’ ‘Trustworthy computing’ memo from January 2002 (responsible for making Microsoft turn the corner on AppSec)

2.7.1.1 Example of an AppSec memo

Here is a version of a memo I wrote for a CTO (in a project where I led the AppSec efforts) which contains the key points to make. Note that the contents of this book are released under a Creative Commons license (CC BY 3.0), which means you can reuse this text, in its entirety, in your organization.

From: CTO (or jointly with CEO)

As you must have noticed from the news, ‘cyber security’ is becoming a very hot topic, with daily reports of companies being exploited and assets compromised. At XYZ we have been lucky to have not (yet) been the target of such attacks, but as we grow and increase our visibility (and assets), we will become a target.

We are a digital company and everything we do happens via the applications we write and use. Historically, the focus of development has been in getting things done as fast as possible, with security being an afterthought and further down the ‘real’ priority list, which is typical of fast-growing companies like XYZ.

This is about to change, and we will be making a significant effort to improve the security of all aspects of XYZ operations and development.

In addition to the current network and physical security activities that we already have in place (firewalls, anti-virus, password management, user accounts restrictions, etc.), we will start a new Application Security practice which will focus on the enablement of our developers to write Secure and Resilient code/applications.

The key to writing secure code is to embed security into the SDL (Software Development Lifecycle) and to ensure that Application Security is an enabler, that allows development to happen faster and more efficiently.

Usually, security is seen as a tax, and as something that is always saying “No”, which explains why security is often avoided or bypassed. For Application Security, the analogy I would like you to think of is ‘Brakes in cars’ (i.e. technology that allows the car to go faster).

What this means is that for us to implement security correctly, we will need to improve our current development and deployment practices (aka Continuous Integration/Deployment) even more, so that all, or most, changes become small, incremental, and fully tested.

The test(ing) component is an area where significant efforts occur, where high-levels of code-coverage and testing are not just important, but a key requirement to have Application Security Assurance. We can’t protect something we don’t understand, visualize and control.

From a practical point of view, and looking at day to day development, we are kick-starting several activities which are outlined below:

1) Security Champions

The key to making Application Security scale is the existence of Security Champions in each team.

Security Champions are active members of development teams and act as their ‘voice’ on security issues.

We already have several Security Champions (see wiki) and if your team doesn’t have one, I strongly advise you to step up and volunteer to become one (this will give you a huge opportunity to learn and to improve your skills).

2) Secure Coding Standards

One of the most important development questions that needs to have a simple answer is ‘How do I code xyz securely?’. Using our existing development stack/tools (and maybe new ones), we aim to create an environment where such guidance is available in the IDE or at the distance of a link.

This kind of guidance only works if it is actively maintained, and I expect everybody to share their knowledge and help to create highly focused and accurate secure coding guidance that are relevant to XYX coding practices.

3) Application Security Automation

Since we ship code every day, we need to do security reviews every day, which means that we need to automate the discovery, and even the mitigation, of security vulnerabilities as much as possible.

The key objective is that when, not if, one of you creates a security vulnerability (which is as inevitable as bugs), we have systems, technology, and workflows in place to detect the vulnerability before that code hits a live server.

This means that we will be introducing Static (SAST) and Dynamic (DAST) software analysis tools and will be looking to expand our current Unit/Integration/E2E testing to incorporate ‘attack patterns’ and ‘architectural checks’.

4) Security data classification and Attack Surface

Another area we need to focus on is the mapping of the data we use, and the inputs and outputs of each of our applications.

When you look at the application you are currently working on, or have worked on in the past, I want you to think:

• ‘Do I need this data?
• ‘What would happen if this data was exposed to the public (or sold on the Dark Net)?’
• ‘What is the impact to our brand?’
• ‘Will this impact our partners and suppliers?’
• ‘How much money will we lose?’
• ‘What happens if this data is modified?’
• ‘Do I know my attack surface?’
• ‘Do I trust the data that I receive?’ (very relevant for ‘internal’ systems)

Note the reference to ‘internal’ systems, since Application Security is not only something that applies to our more outward-facing code. We need to remember that our attackers are getting more sophisticated, and the most dangerous ones will go after our money and assets.

Ironically, the further you go into the network, the more sophisticated and focused the attacker will be, and what matters is the value and exploitability of the application’s assets, unless the code lives in a completely isolated network and doesn’t talk with anything else.

5) JIRA based Risk/Issue acceptance and security visualization

Some of the key challenges when dealing with Application Security include ‘understanding what needs to be done’, ‘what are the risks?’ and ‘what should be the priority?’.

To help answer these questions, a new JIRA project (called RISK) has been created and will be used to hold all currently known security issues/vulnerabilities/compromises.

The workflow of these issues is the following:

1. Issue is opened (by anybody)
2. Issue is reviewed and expanded (for example to provide risk mappings, exploit details, references to similar issues)
3. Technical/business owners decide if a) the issue should be fixed asap (one to two weeks), or b) the issue is NOT going to be fixed (in the short term) and its risk needs to be accepted
4. The issues to be fixed will be:
   • moved into a ‘To Fix’ JIRA issue stage
   • linked into issues that are focused on fixing the issue (in their respective repos)
   • closed when there is verification (by Security Champions) that the issue has been fixed
5. The issues NOT to be fixed will be:
   • moved into a ‘To Accept Risk’ JIRA issue stage
   • assigned to the respective business/technical owner (who will have a button called ‘Accept Risk’ to click)

The key to this workflow is to improve visibility into the reality of Application Security compromises. As a company, our objective is not to create 100% secure applications, but to create applications and code whose risk is aligned with the current threat landscape and business risk appetite.

The other major advantage of mapping security issues like this is that we will be able to have an accurate visualization of the risk profile of our applications. For example, we will expand our use of tools like ELK to visualize (in quasi-real-time) the exploitation of these ‘accepted risk’ security issues, and ideally detect similar security vulnerabilities as they are being exploited.

6) Secure architecture, threat models and nonfunctional requirements

Application Security reviews and practices will also highlight a number of ‘nonfunctional requirements’ or ‘technical debt’ issues. These will not have the ‘the house is on fire’ risk profile, but will need development time to ensure the quality and resilience of our code base.

One practice that we will introduce is the creation of threat models for existing applications and new features. The Security Champions will help to create these threat models, and I expect you all to contribute, since although security-focused, they tend to promote a better understanding of the ‘real architecture’; they will help us to understand better how out technology works and interacts.

7) Hack anything that moves at XYZ

On the topic of security vulnerabilities we need a culture change where we celebrate and reward the discovery of security issues. Collectively, we need to understand that every issue we discover, and mitigate, is one less available to our attackers.

Therefore, you have my permission to (responsibly) hack anything XYZ-related, from external websites, to internal applications, networked resources, printers, cars, etc.

We will create an internal reward system and ensure there are some good professional perks for finding and reporting issues.

The word ‘responsibly’ means that if you find a way to blow up one of our websites, or access confidential data to which you shouldn’t have access, we expect you to create a ‘non-destructive’ PoC and use test accounts, not real customer data. Of course, we know that accidents happen, and we will use common sense.

Eventually, we will create a public ‘bug bounty’ program (after we’ve done a couple of internal rounds), so if you feel that your app will struggle with a public call for ‘..please hack XYZ…’, then you better start finding those issues.

…in conclusion

Times are changing and XYZ is changing. When faced with a scenario where security will be affected by a feature, we need to choose security, or clearly understand the trade-offs.

Security is now a board-level issue and if you feel that any area of our coding/technological world is not receiving the focus it requires, then your duty is to escalate it and fight for it. After all, XYZ is your company too.

These changes represent a great opportunity to make our technology stack and code even better. I hope that you are as excited as me to take XYZ to the next level.

2.7.2 AppSec Technologies and tools

The AppSec community and industry has some really powerful technologies and tools that not only help to find security issues, but can also be spectacular developer productivity tools.

(todo: expand on each technology, mention best players (open Source and proprietary) and provide examples of how they can be used very effectively)

• DAST - Dynamic scanning (Blackbox)
• SAST - Static scanning (WhiteBox)
• IAST - Interactive scanning
• WAF - Web Application Firewall
• Reporting
• Consolidation

2.7.3 Collaboration Technologies

The following technologies are crucial for Security Champions and JIRA workflows to work efficiently:

• Email - good when used in moderation, especially when emails contain links to online resources
• Mailing lists - still the best collaboration tool, as they scale a lot, are easy to filter, they reach a wide audience, are a great way to motivate new Security Champions when they see their name on the list, and they allow interested parties (and older non-active Security Champions) to stay connected to what is going on
• JIRA issues - discussion threads provide a lot of information and details about specific topics
• Wiki - key to capture knowledge in a more structured and long-term environment. Remember that wikis require maintenance and should be curated so that they remain relevant and don’t fall for the ‘tragedy of the commons’ problem. Wiki’s should use JIRA issues as evidence of what is said.
• Confluence - when integrated with JIRA, it creates a powerful way to create dashboards that present the data stored in the JIRA tickets
• Video conferences - tools like Join.me, BlueJeans, Google Hangouts, and Skype are great ways to make remote working and participation possible
• Slack - real-time collaboration tools are key to allow questions to be easily asked, and to allow for asynchronous collaboration, and catching up on specific topics
• Slack integrations - very powerful workflows can occur when SDL tools (and CI pipelines) feed data into specific channels. This is not only a good way to get a sense of what is going on, but also a good way to alert for possible issues or attacks. It gets even better when these integrations are interactive:
  • Hubot(s) - is a great example of this (where it can listen to messages posted and respond to them)
• Log visualization - tools like Splunk, ELK or Graphite, when supported by strong dashboards and visualizations, are one of the best ways to present information and collaborate
• Diagram technologies - Visio has been the gold standard for a long time (draw.io is a recent new player), but the problem is their lack of non-human readable data storage format. To promote collaboration and allow for ‘revision of what changed since last analysis’ (i.e. Diffs), diagramming technologies, created from textual descriptions, are much more powerful and useful, for example PlantUML or DOT (Graphwiz), which are easier to read (in source format) and can be stored in git (i.e. versioned controlled)

2.7.4 Conference for Security Champions

Every 6 to 12 months, it is a good idea to hold a conference exclusively dedicated to security champions, particularly for companies that have multiple locations, where its security champions don’t meet regularly in person.

At the conference, external speakers should present on specific topics.

If there are already several external AppSec consulting companies under contract to the hosting company, the consultants involved in existing projects are perfect candidates to present to the conference. They can use their own examples and stories, and it is easier to present internal materials if all participants are signed-up to the same NDA (Non-Disclosure Agreement).

Never underestimate the power of team collaboration, or of team members getting to know each other. Social events are important, and the model of the OWASP Summit is also a good example of a conference for security champions, as is the Microsoft Blue Hat security conference.

(find other examples)

2.7.5 If you have a heartbeat, you qualify!

It is important to understand that AppSec skills are not a key requirement to become a security champion. The essential quality is to want to become one.

I can make a good developer, who is interested and dedicated, into a good AppSec specialist in 6 months. If the developer is an expert in AppSec, then he should join the central AppSec team.

2.7.6 How to review Applications as a Security Champion

When you review applications as a security champion, you need to start by looking at the application from the point of view of an attacker. In the beginning, this is the best way to learn.

You should start thinking about data inputs, about everything that goes into the database, the application, all the entry points of the application. In short, think about everything an attacker could control, which could be anything from headers, to cookies, to sockets, to anything that enters the application.

Authorization is also a great way to look at the application. Just looking at how you handle data, and how you authorize things, is a great way to understand how the application works.

• add info about how this is best done using Threat Models and asking the STRIDE questions

2.7.7 Involvement in senior technical discussions

Involvement in senior technical discussions provides a great opportunity, and a perk, to security champions. Once a SC program is established, if the SCs should be involved in senior technical discussions. If they aren’t, it means that security is not being taken into consideration, and this is a problem.

2.7.8 Learning-resources

Books to learn from

• Hacking Exposed, both the normal version and Hacking Exposed: Web Applications.
• The Shellcoder’s Handbook is a great book from an application security point of view, and it walks you through the vulnerabilities.
• The Web Application Hackers Handbook provides a solid overview of problems in web applications and how to identify them
• (todo: add more books with a small description each)
  • Kevin Kelly books
  • The Circle
  • The Phoenix Project
  • Improving Web Application Security and Secure Code (from MS)
  • Geekonomics
  • Cathedral and the bazaar

(todo: add links to Amazon)

OWASP Materials

• Cheat Sheet series - one of the best AppSec resources on the internet
• Testing guide - good resource to start you AppSec skills.
• SAMM (Software Assurance Maturity Model)
• Developers Guide
• Code Review guide
• Top 10
• Top 10 for Mobile
• Top 10 Proactive Controls
• ASVS (Application Security Verification Standard)
• AppSensor guide
• Automated Threats Handbook

(todo: add links to projects)

Vulnerable by design applications

Another great resource is the OWASP WebGoat Project which just released a new version, and it has a selection of exercises in vulnerabilities so you can learn how they work and you can get clues if you get stuck. The first thing is to do the exercises and hack into these applications.

• Add more examples of apps, split by technology
  • .Net - WebGoat.Net, HackmeBank
  • Java - WebGoat
  • Node - NodeGoat, JuiceShop
  • Ruby - Railsgoat
  • php - Damn Vulnerable Web Application
  • Android - GoatDroid
  • iOS - iGoat
• OWASP Broken Web Applications Project
• [OWASP Vulnerable Application Directory] (https://www.owasp.org/index.php/OWASP_Vulnerable_Web_Applications_Directory_Project)

Hack your apps

You can hack anything that moves in your company, because you have an implicit mandate to protect your own company.

Go for your own application, or go for your colleague’s application. Sometimes that’s a bit easier to digest :)

BugBounties

Also go for things like bug bounties, which are basically companies that give you permission to hack them and find security issues.

(todo: mention existing programs like https://hackerone.com/)

2.7.9 Make sure your Security Champions are given time

It is very important that security champions get the time, the space, the mandate, and the information required to do their jobs.

The good news is, now that you have security champions (at least one per team), their work will allow you to see the differences between the teams and the parts of the company who can make it work, and those who struggle to make it happen.

To participate successfully in the security of their projects, security champions must execute the following tasks:

• review code
• fix code
• write tests
• know what is going on
• maintain the JIRA tickets
• create Threat Models
• be involved in the security practices of the teams

Adherence to these tasks leads to better code, better project briefs, up-to-date documentation and tests for the application.

Security champions should spend at least one day a week on these activities. Although allocating this time is easy for management to accept in theory, it is much harder to put into practice.

In the beginning, security champions will barely be able to spend a couple of hours a week at this work.

One of the things you should know from a central point of view is who is doing this kind of work, and who can spend the time doing it.

Fortunately, these things can be measured and tracked from the point of view of all the teams (using dashboards and graphs from the JIRA Risk Project).

2.7.10 Making Security Champions AppSec Specialists

• once you have SCs you need to grown them
• training is very important

2.7.11 Regular Hackathons

Regular hackathons have many benefits for security champions, teams and companies.

Security Champions should organize the hackathons, as hackathons are the next level of SCs applications. SCs can use them to find issues and learn new features.

They should be held every Friday or Monday afternoon, preferably with some drinks and pizza.

Ideally, a hackathon would happen every week, but if not it should be held every two, three, or four weeks. The important thing is to have movement.

The hackathon model fits well into the 10% of time developers need for research.

Inviting external experts, such as pentesters that regularly test the applications, has the benefit of providing new insights. Trying to do something, and then watching a more experienced and knowledgeable person do the same task, is one of the best ways to learn your job. You benefit from observing the workflow and strategies they employ to discover and exploit an issue.

Organizing teams into red (attackers) and blue (defenders) can also be very effective. However, planning your hackathon around teams makes it harder to organize, and needs a lot more preparation.

2.7.12 Rewarding Security Champions

• budget to sponsor the best one in last month (or week, if there is a lot of activity)
  • Participation in conferences (ticket + travel expenses)
  • Books
  • Bonus (some companies prefer old fashioned cash prices)
• How to measure who is the best of the month
  • Number of JIRA tickets: opened, fixed, tested
  • Number of Threat Models
  • Highest improvement
  • Innovative research
  • Shipped code (of an module used by multiple teams)
  • Above average documentation (Secure coding standards)
• provide presentation opportunities (for example at the company wide Security Champions conference, or to senior directors (a couple levels above the current developer’s position in the org chart))
• basically anything that you can do to a developer that he/she can put on the CV is a good reward
• It is very important to create explicit and open rules about these rewards, since the worse situation is when a particular Security Champions receives an award not because of his achievements but due to other (political or friendship based) decision.

2.7.13 Security Champions activities

What do Security Champions do in they allocated day-to-day time (4h to 8th)

• it is very important that JIRA tickets are use to map, track and allocate these tasks
  • the number of these tickets is what justifies the work the SC does
  • it also provides movement and management information about what is being done
  • it is important to have a good number of actionable (in the ‘Allocated for Fix’) stage (30 to 50 are a good number, as long as they are realistic and real)
  • you know the model is working when the managers start asking: “Do you have enough time of those SC activities?”, “Do you need help from other team members (i.e. more resources)”
    • eventually the SC should also be managing the AppSec tasks that are performed by other team members
• the time allocation is better if done in blocks of 4h (over two days), or 2h every day (at the same time)

2.7.13.1 Other AppSec projects to be involved in

• Map Attack Surface tool
• Web Services Visualisation tool
• Standard Schemas across the company

2.7.14 Security Champions Don’t Take it Personally

If you are a Security Champion, don’t take it personally if teams aren’t listening to you. Don’t think that you are the problem, that you aren’t good enough or that you are failing to communicate in some way.

In most cases, the problem isn’t you. The problem is actually the system; the company isn’t structured in a way that allows the security champion’s questions to be prioritised and answered. In other cases the Security Champion is not included in security-relevant architectural meetings and decisions.

So, if you find that you are struggling to get traction from a team, the team isn’t responding, or it fights you, then drop those requests (as long as the Risk as been accepted). If they treat you as a TAX, as somebody who is giving them work they don’t want to do, then also drop it.

In the Risk ticket, explain that you tried to persuade the team to accept the risks of not doing security, and that they are now responsible for their security, because you cannot help them.

In such cases, the problem lies not with the Security Champion, but with the company and the organisation, maybe even sometimes with the team itself. This is why it is important to have success stories you can point to and say, “Hey! It worked with that team, and it worked with that team. If it doesn’t work with this team, then I am not the problem”.

2.7.15 Involve Security Champions in Decision-making

Once a program starts being placed, security champions will often give feedback that they are not involved in the workflows and decisions. The job of the security champion is to ask, “What is this? Do I trust this? What happens with this?”, but they often don’t get the opportunity to ask these questions, because decisions are made without their input.

To illustrate this problem, a situation occurred recently where the security champion started to create threat models across a product, and thereby managed to retroactively involve himself in some of the decision-making.

He created a threat model with a lot of the developers and the other systems owners, and they found a couple of end points that were problematic. Once the teams were aware of the problems, realizing that data wasn’t trustworthy, and identifying vulnerabilities, then the developers themselves started to question the methods.

They found a nasty, exposed method that nobody knew about. It was a powerful and toxic method, that allowed a lot of data to be erased with very little accountability.

That method should never have got there; it wasn’t even validated, and there were other components missing.

This is exactly the kind of exercise you want security champions to be involved in, because you want them to question those things before they hit development, before they go live. You want to catch those problems as early as possible.

This is not to say “No” to functionality. It is rather to say, “If you want to do this, here are the side effects, here are the compromises, and here is how we should protect it”. At the very least you should say, “If you do this, you do it without due regard being given to the security implications. You must accept the fact that this feature has no security thinking behind it”.

It is very hard for a project owner to accept that the feature being pushed has not benefitted from the consideration of security implications and side effects. This leaves him accountable for how it works, and this makes project owners uncomfortable.

2.7.16 Supported by central AppSec

They need to be supported by a central AppSec team, but it is key that each team has one. If there are not enough champions they can create a guild or chapter and rotate over teams. It is also important that they are backed in their decisions by corporate security like e.g. the CISO.

Tasks (expand) * Code Reviews and Pen-tests (internally or via external managed services) * AppSec Automation (SAST and other tools) * Secure coding standards * Incident Response * Recruitment and Training

2.7.17 The Security Champions Concept

“If everyone is responsible for security, then nobody is” ⁶

• What are Security Champions?
• Why Security Champions
  • challenges to scale and propagate AppSec Knowledge
  • Keep AppSec focus and energy
  • have somebody responsible for AppSec
• What is the target audience of this section
  • … you want to become an SC …
  • … you want to set you a SC network …
• There is an heavy AppSec focus, but this is applicable to all of IT, Dev and Risk management practices
• explain how JIRA Risk Workflow is connected to the Security champions

“If everything is important, then nothing is.” from Patrick Lencioni

2.7.18 Threat Modeling workshops

• to happen every week
  • or every other week, alternating with a Security Champions meeting (this way there is always an SC activity happening every week at the same time and place)
• good place to ask questions and to present Threat Models created during the previous week
  • If there are not a lot of materials to present or show, that indicates that the quantify of Threat models being created is quite low (note that the objective is to create Threat Models for new features, which are happening all the time)
• another great learning resource, specially for new SCs (who are still getting their head around the workflow)
• see Threat Models section for more details

2.7.19 Training Security Champions

• Training is key to improve SC skills
• Best training is done on top of languages and frameworks they use
• Wiki pages with links to actual issues and relevant resources make a massive difference
• Using vulnerable by design apps (or older versions of current main apps with known vulnerabilities) are a great way to learn (by exploiting them)
• Writing exploits and finding vulnerabilities is a key step of the required accelerated learning curve

2.7.20 Weekly meetings

• explain why these meetings are so important
  • what happens at one of these meetings
  • what is the normal agenda
  • who turns up
• should happen every week, but a good compromise is for them to occur every other week
• everything shown and discussed at the Security Champions meeting needs to be done via one or more slides (to be added to that week’s slide deck)
• these slides are VERY important when creating learning materials
• create wiki pages with a full list of all past SC meetings (each entry is categorised by labels)
  • there are a great way to teach Security Champions and to call their attention to areas to research in their own apps
    • I’ve seen many cases where Security Champions will see a presentation on a topic/technology relevant to their current domain and think/say “Humm… I think we might be vulnerable to THAT vulnerability”

Initially, it will take considerable effort to generate content for these meetings; to find presenters; and to keep it engaging / interesting.

• think of the money that it costs the company to have all those resources in there. Make it count and don’t waste participants time
  • the developers are very busy and if the meetings are not relevant or not interesting they will just don’t turn up
• central AppSec team (to help kickstarting the meetings) and to keep the energy level up) must always be looking for topics to present at the next SC meeting. For example:
  • Threat Models
  • AppSec Questions
  • AppSec ideas
  • events from a point of view on an attacker
    • attacks
    • AppSec news,
    • basically any AppSec related topic that has not been presented recently)

2.7.20.1 Contents for weekly meetings

Here are some examples of what to present at these meetings:

• Latest news on AppSec (DDos, exploits, etc..)
• Latest bug-bounty findings and payments (a really good source of real-world examples)
• Issues found and issues fixed (on the SC’s application or service)
• Secure coding techniques
• Security tools and technologies (e.g. OWASP ZAP Project, OWASP dependency checker)
• Tools/techniques to improve the Developer’s Productivity (e.g. WallabyJS, NCrunch)
• Hack challenges
• Security reviews current in place
• Other OWASP tools and documents (ASVS, OwaspSAMM, AppSensor, Testing Guide)
• Testing techniques, workflows and technologies used (which might be different from the current development stack)

2.7.21 What is a Security Champion and what do they do?

Security Champions are a key element of an AppSec team, since they create an cross-functional team focused on Application Security

What is a Security Champion?

• Security Champions are active members of a team that may help to make decisions about when to engage the Security Team
• Act as the “voice” of security for the given product or team
• Aim to bridge the gap between security and dev teams
• Assist in the triage of security bugs for their team or area

What do they do?

• Actively participate in the AppSec JIRA and WIKI
• Collaborate with other security champions
  • Review impact of ‘breaking changes’ made in other projects
• Attend weekly meetings
• Are the single point of contact for their assigned team
• Ensure that security is not a blocker on active development or reviews
• Assist in making security decisions for their team
  • Low-Moderate security impact
    • Empowered to make decisions
    • Document decisions made in bugs or wiki
  • High-Critical security impact
    • Work with AppSec team on mitigations strategies
• Help with QA and Testing * Write Tests (from Unit Tests to Integration tests) * Help with development of CI (Continuous Integration) environments

2.7.22 What it takes to be a Security Champion

To become a security champion, it is essential that you want to be one.

You need a mandate from the business that will give you at least half a day, if not one full day per week, to learn the role. The business should also provide the means to educate and train you and others who wish to become security champions. Increasing and spreading knowledge will increase awareness and control.

You need to be a programmer, and understand code, because your job is to start looking at your application and understand its security properties. You should also know ‘the tools of the trade’, and how to implement them, in the most efficient way. Lastly, you must be able to identify useful metrics and instruct on how to obtain them.

2.7.23 To Add

add references to

• Creating a Security Champions Network

2.7.24 Security Champions

• So you want to be an SC?
• … here is how you do it…
• Need to be a developer

2.7.25 Becoming a Security Champion

To become a security champion, the most important property is that you want to be one.

You need a mandate from the business that will give you at least half a day, if not one day.

You need to be a developer who understands code, as it is difficult to be an AppSec SC and not be able to code.

Your job is to study your application and understand its security properties.

You need to look at the application from the point of view of an attacker. In the beginning the best way to get your head around it is to think about all the entry points of the application (i.e. inputs).

This includes everything the attacker can control: from GET/POST parameters, to WebServices calls, to headers, to cookies, to websockets, to anything that actually enters the application.

After that, authorization is also a great way to take a look.

Basically, start with creating threat models :)

Looking at how you handle data and how you authorize is a great way to understand how the application works.

2.7.26 Hack anything that moves

You should hack anything that moves in your company, because you have a mandate to protect your own company (in some companies it might be better to get this ‘hack everything’ authorization via official channels).

In terms of targets, you can go for your own company, for your own application or for your colleague’s application (which sometimes it’s a bit easier to digest).

2.7.27 BugBounties

Also go for bug bounties, which is a nice list of companies that give you permission to ‘hack/attack’ them and find security stuff.

2.7.28 Developers we need YOU in AppSec

• big oportunity for existing developers to move into appsec

2.7.29 Big market demand for AppSec Professionals

At the moment there is huge demand for AppSec devs. Historically the path info InfoSec via Network Security, which means that the majority of InfoSec professionals cannot move to AppSec, because they can’t code professionally (i.e. no real knowledge on how to build, test and ship software/apps)

It is also possible to learn on the job since there is such a shortage, which makes it easier to hire Devs who like AppSec and then train them up.

• add stats on salaries for: AppSec and projects that mention OWASP
  • http://blog.diniscruz.com/2009/09/owasp-driven-jobs.html
  • http://www.indeed.com/salary?q1=appsec&l1=

2.7.30 If you don’t have a Security Champion, get a mug

If your developer team doesn’t have an assigned security team champion, get one of these mugs.

That ‘Security Expert’ mug represents the fact that, without a security champion, when a developer has an application security question, he might as well ask the dude on that mug for help.

I also like the fact that the mug reinforces the idea that for most developer teams, just having somebody assigned to application security is already a massive step forward!!

Basically, we have such a skill shortage in our industry for application security developers that ‘if you have a heartbeat you qualify’

2.7.30.1 How to create the SC Mug

• Get a mug with lots of white space on the front and back
• write Security Champion at the front in large letters (but not so big that the text can’t be read from a distance)
• Alternatively, at the back write: It’s me, or Google, or Stack Overflow
• Or, if you have a small company stuffed animal or object, put it inside the mug

Put the mug in a central location, visible place to the team. It is important that the mug is in a neutral place, and not ‘assigned’ to anybody.

In some teams, I’ve observed a ritual when a Security Champion is appointed, presented with the mug, and expected to keep it on his/her desk.

2.7.31 Public references to Security Champions teams

Microsoft

The Microsoft Agile SDL describes them as Team Champions

In Simplified Implementation of the Microsoft SDL

Team Champions. The team champion roles should be filled by SMEs from the project team. These roles are responsible for the negotiation, acceptance, and tracking of minimum security and privacy requirements and maintaining clear lines of communication with advisors and decision makers during a software development project. * Security Champion/Privacy Champion. This individual (or group of individuals) does not have sole responsibility for ensuring that a software release has addressed all security issues, but is responsible for coordinating and tracking security issues for the project. This role also is responsible for reporting status to the security advisor and to other relevant parties (for example, development and test leads) on the project team. * Combination of Roles. As with the security and privacy advisor role, the responsibilities vested in the champion role may be combined if an individual with the appropriate skills and experience can be identified.

In “The Microsoft SDL Process Template – Making Secure Code Easier” Brian Harry blog entry says this about Security Champions

With the SDL Process Template, a security owner can easily tackle that initial question of “where do I start”? The Process Guidance page provides a security owner (and the entire team) with a brief overview of the SDL, five steps for Getting Started on an SDL project, and details on customizing the template and extending it for third party security tools. There is even more material supporting SDL implementation and customizing the SDL Process Template in the SharePoint library.

and

A security owner can accelerate the task of defining security requirements by opening up a query that includes all of the default SDL requirements – ready to triage and assign! There is also a custom work item to add your own requirements or recommendations

Mozilla

Mozilla has a good pages at Security and Security/Champions.

In the SC page they mention other types of Security Contributors:

• Contributor
• Security Contributor (Bug Bounty Reporters/Patch submitters)
• Security Mentors
• Security Group

Unfortunately this program has ended in 2012 following an internal reorganisation

OwaspSAMM

Owasp SAMM (Software Assurance Maturity Model) uses the term Team Champions

From Secure_SDLC_Cheat_Sheet

BSIMM

In BSIMM security champions are named ‘satellites’ and described in section 2.3 > The satellite begins as a collection of people scattered across the organization who show an above-average level of security interest or skill. Identifying this group is a step towards creating a social network that speeds the adoption of security into software development.

…others?

… add more

2.7.32 Draft notes - Threat Models

• Do Threat Models in layers
  • identify each STRIDE issue per layer
  • each layer it built on top of the previous one
  • connect the threats
  • map the Urls and Data objects and connect them across layers
  • created ‘refactored’ and ‘collapsed’ views of the diagrams (specially when there are ‘web services that act like proxies’ in the middle)
• Introduce the concept of Sinks in Threat Modeling
  • how to connect the multiple threats (one for each layer) so that they are chained
    • map this concept with the concept of Attack Trees
• Idea to create a Book focused on “Threat Modeling examples and patterns”

2.7.33 Capture the success stories of your threat models

One of the key elements of threat modeling is it’s ability to highlight a variety of interesting issues and blind spots, in particular within the architecture of the threat model. One of my favorite moments occurs when the developers and the architects working on a threat model realize something that they hadn’t noticed before.

In such cases, sometimes it is the developer who says, “Oh, I never realized that is how it worked!”. Other times when the architect says, “Well, this is how app was designed”, and the developer responds “Yeah, but that didn’t work, so we did it like this.”

What is actually happening when such exchanges take place is the mapping of reality, and the creation of a much better understanding of what that reality actually means within the company. Truth is being mapped, and the threat model becomes a single source of truth for the company.

It is very important not only to capture these success stories, but also to advertise and promote them. Promoting them allows you to explain one of the reasons why you want to work in threat modeling; because you want to understand what is going on, and you want to make sure that everybody working on a threat model is on the same page in terms of development, QA, testing, and so on.

2.7.34 Chained threat models

When you create threat models per feature or per component, a key element is to start to chain them, i.e. create the connections between them. If you chain them in a sequence you will get a much better understanding of reality. You will be able to identify uber-vulnerabilities, or uber-threats, that are created by paths that exist from threat model, A to threat model B, to threat model C.

For example, I have seen threat models where one will say, “Oh, we get data from that over there. We trust their system, and they are supposed to have DOS protections, and they rate limit their requests”.

However, after doing a threat model of that system, we find that it does not have any DOS protections, even worse, it doesn’t do any data validation/sanitisation. This means that the upstream service (which is ‘trusted’) is just glorified proxy, meaning that for all practices purposes, the ‘internal’ APIs and endpoints are directly connected to the upstream service callers (which is usually the internet, or other ‘glorified proxies’ services).

This example illustrates how, when you start chaining threat models, you can identify data that shouldn’t be trusted, or data that is controlled by the attacker. Usually the reverse also applies, where when you go downstream and check their threat models, you will find that they also trust your data and actions far too much.

Of course, the opposite of this scenario could also be true. One of the threat models might say, ”…we have a huge number of issues at this layer”. However, when you look at the layers above, you find they are doing a good job at validating, authorising and queuing the requests; they are all working to protect the more vulnerable layer, so the risk is low overall.

When you chain a number of threat models, you track them, document them, and you greatly increase your understanding of the threats. You can use this new knowledge in the future to ensure that you don’t expose that particular threat into new systems or new features.

2.7.35 Developers need data classification

Every type of data that exists in an organisation, especially the data that is consumed by applications, needs to have a Data Classification mapping.

Developers need to know if a particular piece of data is sensitive, and what value it holds for the business.

A good way to determine the expected level of confidentiality and integrity, is to ask what would happen ‘If a particular set of data were to be fully disclosed?’ (for example uploaded to PasteBin) or ‘If some of the data was being maliciously modified over a period of months?’.

These are really hard questions, and only by answering them, the developers (and business owners) can start to understand the value of an particular data set (given-to or generated-by their application).

Developers need to understand what they are dealing with, what is valuable to the business, and what needs to be protected.

See Microsoft’s Data Classification Wizard for a good list of data types that exist on large organisations (this will need to be tweaked per application)

todo: add references to Threat Modeling

2.7.36 Threat Models as better briefs

• diagrams created (DfDs for example) will represent reality much better than existing documentation
• It is key that Threat Models are used as ‘sources of truth’ (which are then maintained as code/architecture changes)

2.7.37 Threat Model Case Studies

• for each case study, add list of Risks that need to be added
• Examples of Case Studies to add:
  • Source code, Keys and passwords stored in Developer’s laptop
  • HTTP to HTTPS transition, lack of HSTS header, insecure cookies
  • Homeopathic Apigee127 web service
  • Smart-carts that control door access in buildings
  • Login brute of accounts
  • 4 version of HTML editing and Rendering
    • raw HTML
    • raw HTML with CSP
    • using Markdown
    • using Markdown with CSP
  • File upload solution vs GitHub fork
  • QA team with bad test environment
  • Insecure APIs with vulnerable by design methods

2.7.38 Threat Model Community

There is currently (late 2016) space within the application security world to develop a community focused on Threat Modeling. Such community would allow the many parties working on Threat Modeling to share information and provide a voice to all different stakeholders.

Questions to be considered by this community include:

• What are common patterns and threats across projects?
• What do developers understand about it?
• How are Threat Models consumed by managers?
• What do we name an issue/threat/risk?
• What schema can be used to store the data?
• How to version control the artifacts created?

These questions are important because they are the ones that really allow us to plan and understand the best way to structure Threat Models.

Open up the models

At the moment, 99.9% of Threat Models exist within companies in proprietary/closed environments. This doesn’t mean that these companies don’t want to share their models. It may just be that the information isn’t in a format that is easy to share.

One of the advantages of approaching this as a community, in an open way, with clear licenses and clear open standards on how to communicate, is that it forces us to solve the problem of separating confidential data from generic public data.

This community effort will also help to resolve the issue of data versioning, which is a very complex problem.

Version the models

Today, versioning (of Threat Models) is either done using the file system (for example appending v1.x to the file name), or even worse, not done at all (note: existing Threat Models applications, desktop or online based, don’t have an historical view of data).

This is not an effective way to work, doesn’t promote collaboration and doesn’t scale.

More importantly, this way of (quasi manual) versioning of Threat Models, prevents us from understanding the evolution of a particular Threat Model.

For example, imagine a Threat Model that starts is small, then grows bigger and then shrinks again, all depending on the desired (or implemented) features. To understand the present and future threats it is important to know the past.

Let’s say that you have a particular Threat Model of a particular feature of an application that is reasonably self-contained, or in a fairly good state. However, the addition of a new feature will cause the whole thing to explode. Essentially, what you are now dealing with is a situation where the new feature has either created a number of issues, or it hosts a number of vulnerabilities. These are much easier to visualise in a state where you can actually see the new connections and the impact of the change/feature request (by the business owner).

Reviewing Threat Models is much easier when only looking at what changed since the last version.

Existing efforts

Note: OWASP currently has an active Slack channel and an inactive project on Threat Modeling

2.7.39 Threat Model Confirms Pentest

A key objective of pentest should be to validate the threat model. Pentests should confirm whether the expectations and the logic defined in the threat model are true. Any variation identified is itself an important finding because it means there is a gap in the company’s understanding of how the application behaves.

There are three important steps to follow:

1. Take the threat models per feature, per layer and confirm that there is no blind spots or variations on the expectation
2. Check the code path to improve the understanding of the code path and what is happening in the threat model
3. Confirm that there are no extra behaviors

2.7.40 Threat Model per Feature

Creating and following a threat model for a feature is a great way to understand a threat model journey.

First, take a very specific path, a very specific new feature that you are adding, or take a property, such as a new field, or a new functionality.

Next, you want to create a full flow of that feature. Look at the entry point and the assets, and look at what is being used in that feature.

Now, you can map the inputs of the feature; you can map the data paths given by the data schema, and then you follow the data.

You can see for example how the data go into the application, what it ends up with, who calls who. This means you have a much tighter brief, and a much better view of the situation.

2.7.41 Threat Models mapped to Risks

• every risk identified in the Threat Model needs to be opened and tracked in the JIRA Risk project
• using Confluence to host the Threat Model content and have ‘live’ queries with the relevant risks (makes a massive difference in the maintainability of these Threat Models)
  • When special views are needed, Jira’s JQL Queries can be used to create some of the queries

2.7.42 Draft notes - Threat Models

• Do Threat Models in layers
  • identify each STRIDE issue per layer
  • each layer it built on top of the previous one
  • connect the threats
  • map the Urls and Data objects and connect them across layers
  • created ‘refactored’ and ‘collapsed’ views of the diagrams (specially when there are ‘web services that act like proxies’ in the middle)
• Introduce the concept of Sinks in Threat Modeling
  • how to connect the multiple threats (one for each layer) so that they are chained
    • map this concept with the concept of Attack Trees
• Idea to create a Book focused on “Threat Modeling examples and patterns”

2.7.43 When to do a threat Model

normal development flow

1 digraph G {   
2   size= "3.0"
3   node [shape=box]    
4   Idea -> "Project brief"
5        -> "Scheduling"
6        -> "Development"
7        -> "QA"
8        -> "Release"
9 }

Proposed development flow

 1 digraph G {   
 2   size= "4.5"
 3   node [shape=box]    
 4   Idea -> "Project brief"         
 5        -> "Scheduling"
 6        -> "Development"
 7        -> "QA"
 8        -> "Release"
 9 
10 "Project brief" -> "Threat Model"                
11 
12 "Threat Model" -> "Option A" -> "Risks"
13 "Threat Model" -> "Option B" -> "Risks"
14 "Threat Model" -> "Option C" -> "Risks"
15 "Risks" -> "To be accepted"
16         -> "Scheduling"
17 "Risks" -> "To check implementation"        
18         -> "QA"
19 
20 "To check implementation" -> "Pen-test"
21                           -> "Release"
22 
23 }

3. Appendix

• Appendix A: Creating Workflow in JIRA
• Appendix B: GitHub book workflow

3.1 Appendix A: Creating Workflow in JIRA

This section shows how to create the JIRA workflows without using any JIRA plugins

Key concepts of this workflow

• All tests should pass all the time
• Tests that check/confirm vulnerabilities should also pass
• The key to make this work is to:
  • Make business owners understand the risks of their decisions (and click on the ‘accept risk’ button)

3.1.1 Creating-a-Jira-project

For these examples we will use the version hosted JIRA cloud called (in Oct 2016) JIRA Software.

Note that the same workflow can be created on the on-premise versions of JIRA (including the older versions)

If you don’t have a JIRA server you can use, then use can create on using the Jira evaluation page and choosing the JIRA Software option. I would also add in the Documentation (aka Confluence) module since it is a very powerful wiki (which is called Confluence)

If you don’t have an account you will need to create one.

After clicking on Start now your cloud instance of JIRA will be created (my bet is that this is a docker container with a dedicated data store for each customer/trial)

3) login

4) create new project

5) choose Kanban Software Development

6) Name it ‘RISK - AppSec’ with the key ‘RISK’,

7) Your new JIRA Project dashboard should open and look something like this

3.1.2 Step-by-step instructions

Creating RISK workflow

as seen here http://blog.diniscruz.com/2016/03/updated-jira-risk-workflow-now-with.html

7) Go to JIRA Administration , Issues

8) Add an issue type

9) call it Risk

10) Go to Issue type schemes and click on ‘Add Issue Type Scheme’

11) Call it Risk Scheme and add the Risk Issue type into to (click Save to continue)

12) Associate that Risk Scheme

13) To the ‘RISK - AppSec’ project

14) Go to Workflows and add new one

15) call it ‘Risk Workflow’

16) Add status ‘In Progress

17 ) Create transition from Open to In Progress

18) Create a new Status called ‘Allocated for Fix’

19) add a transition to ‘Allocated for Fix’ state

20) how workflow looks like at the moment

21) Add status: Fixing, Test Fix and Fixed

with fixed set to the ‘Done’ Category

21) add transitions to those status

22) Add Status: Closed, ‘Awaiting Risk Acceptance’ , ‘Risk Accepted’, ‘Risk Approved’, ‘Risk Not Approved’, ‘Risk Approval Expired’

23) Add transitions (including a couple to reverse some of the steps)

24) Completed workflow should look like this

25) go to Workflow Scheme and chose to ‘Add workflow scheme’

26) Add Existing Workflow

27) Assign Risk Issue type to it

28) exit admin and go to the RISK project’s settings

29) in the Workflow page chose to Switch Scheme

30) Pick the ‘Risk Workflow Scheme’

31) Test workflow (fixing path)

31) Test workflow (Accept Risk)

3.2 Appendix B: GitHub book workflow

• explain the workflow used to create this book
  • audio transcripts and copy editing (and upwork)
  • Pull requests for copy editing
  • Labels for managing tasks and issues
• show how to report a problem with the book or suggest ideas

3.2.1 Book creation workflow

• what are all the actions that occur, from making a code change to having a preview done
• explain two modes (Github online editing and offline editing using Atom editor)

3.2.2 GitHub Leanpub hook

• how it works
• what it does

3.2.3 GitHub online editing

• explain how it works and the workflow used
• mention leanpub service hook and how every content change will eventually result in a new preview

3.2.4 GitHub repos used

• https://github.com/DinisCruz/Book_Jira_Risk_Workflow
  • hold content and raw files
  • better searching since the manuscript files are not there
  • used to create the stand-alone version of the book
• https://github.com/DinisCruz/Book_Jira_Risk_Workflow_Build
  • holds files in Leanpub friendly format
  • hooks into leanpub via GitHub service
    • every commit to this repo will trigger a build

3.2.5 Tool leanpub-book-site

• explain what it is and how it works
  • rules of engagement
  • folders and file structure
    • auto-generation of book.txt file
    • consolidation of images
• reason for doing it was : 1) solve problem of massive image folder (now each chapter is directly mapped to it’s images, which is ok as long as the image’s name are not repeated) 2) solve problem of having to maintain the Book.txt file 3) allow splitting of manuscript folder into separate repo

3.2.6 Atom, Markdown, Graphiz, DOT

Editing and diagram creation was done on Atom editor with the markdown-preview-enhanced plugin

Text was written in markdown

Diagrams where created using DOT Language , rendered by GraphWiz and Viz.js

This is what the IDE looks like:

References:

• GraphWiz and Dot:
  • Polygon-based Nodes
  • Node, Edge and Graph Attributes
  • Viz.js online demo

3.3 Appendix C: Security Tests Examples

• add multiple examples of security tests
  • in node/coffee script
    • HSTS header check
    • detecting attack surface changes
    • performance tests
  • in Javascript
    • emberjs safehtml issue
  • in java
    • random() lack of randomness
    • detecting methods calls
  • in .net
    • email regex issue
    • using reflection to check api usage
    • testing XSS on HTML Elements
  • android
    • query SQL Injection

3.4 Appendix D: Case Studies

3.4.1 File Upload

• public competition where external users where supposed to upload their work (this was aimed at University grads)
• lots of moving parts in original design
• better solution was to use GitHub for file submissions
• massive difference in the risk and complexity of each solution

3.4.2 HTML editing

• common request/feature in web-apps
• massive attack surface and security issues (equivalent to XSS) i
• prevents clients to protecting themselves (unless they can use CSP)
• good example of not answering the real business need
  • which tends to be ‘edit text, with images, some formatting (bold, italics), links and tables’
  • all these can be meet if using Markdown (which can be even better for the user, due to its easy of use, ease of diff and readability)
• lots of un-intended side-effects, for example with copy-and-paste
• trying to create ‘safe html’ is very dangerous due to the crazy stuff that HTML allows and the ‘cleverness of some browsers’ (which are able to fix broken HTML and Javascript)

3.5 Appendix E: GitHub Issue worklfow

3.5.1 GitHub Labels

• Below are some examples of the use Labels

Labels on book generation

Labels on complex software development

3.5.2 Reporting issues

• GitHub can also be used to create similar risk (and other) workflows
• Leanpub Issues
• Veracode Issues

3.6 Draft Notes

These are a mix bag of notes made on real notebooks which need to be normalized, converted into paragraphs and placed in the right location

There might be some repeated content which has already been covered in the book

3.6.1 Draft notes - AppSec Tools

• The false positive and false negative problem in tools and in code review
• when doing code reviews, for me the issue (and vulnerability) starts in a line of code or in a method (even if it not being used at the moment, or exposed to an malicious source)
• desired SAST workflow
  • scan every app and component in the company
  • for each app scan all their versions
    • find versions with vulnerabilities (which should be banned from being used)
  • mode where they sell concurrent scans (change more for speed, performance) not number of apps
    • apps get created and deleted with every scan
    • results are stored in git
  • for some apps it might make sense to have apps live longer (specially if the ui helps with remediation)
  • allow custom rules
  • expose internal objects and graphs
• personalize and timing of the delivery of SAST (or other) tools knowledge (i.e. findings)
  • this can be as important as the content since the recipient must be in a position to consume/understand that guidance
  • this will need to customized to the developer’s skills, appsec experience, status of mind and stage of SDL (planing, architecture, development, testing, qa, bug fixing)

3.6.2 Draft notes - Code Quality

• Bad code and lack of testing has to be measured, since it has to have side effects.
  • If code has not tests but always works, all releases are smooth and there are no major incidents, then it is OK not to have tests,
  • … now back in the real world … lack of tests and weak CI will always have side effects:
    • problematic releases
    • customers finding bugs
    • long time to create features requested by business
    • quality issues on deliverables
    • weak and ever-changing briefs

3.6.3 Draft notes - DevOps

Stages of AppSec automation

Start with static analysis which don’t need a live environment to deploy the application

1. Have a CI that builds code automatically for every commit of every branch
2. Ran ‘a’ tool after build (doesn’t really matter which one what matters is that it uses the materials created on step 1)
   • use for example FindBugs with FindSecBugs ⁷
3. Find tune scan targets (i.e. exactly what needs to be scanned)
4. Filter results, customize rules to reduce noise
5. Gather reports and scans and put them on git repo
6. create a consolidated dashboard (for management and business owners)
7. add more tools
8. loop from 5

after this is mature, add a step to deploy the app to a live environment (or simulator)

3.6.4 Draft notes - Developers

• developers can be disconnected from their future-self
  • why are they accept and make decisions that will hurt them in the future
  • idea of showing the real cost of technological & coding decisions
    • Positive & Negative impact
    • one-off vs recurring cost/benefit analysis
  • we need developers to act more inline with their (and the apps) long-term interests
    • usually business owners are not made accountable for their technological decisions (they get the bonus for ‘delivering’ and move on to another project or company )
  • developers need to start picking fights on what is better for the business in the long run.
    • when they overruled, that needs to be ‘on the record’, which is better done with the JIRA Risk Workflow
    • best strategy to do this is to align yourself with the values and direction of the organization or company you are working for
• We should be able to predict when a developers is about to create a security vulnerability
  • based on similar patterns/code-graphs of other developers/code-changes in the past
  • Google and StackOverflow queries
    • detect bad results (maybe with even security vulnerabilities) and provide better recommendations
    • detect what questions they will ask next and suggest better results (proactive code-complete like, AI technology will help)
• Detect when code is one degree away from and exploit/vulnerability
  • Defense concept: always be or three degrees of separation from an serious vulnerably/exploits

3.6.5 Draft notes - Government

• In the ‘open letter to president’ (find link) there is no mention of secure coding or secure apps
  • but all our technology runs on code and until we control it, we will not really be solving the root causes

3.6.6 Draft notes - PoCs

• related to ‘why exploits are important’
• example for demos to perform ‘management and C-level execs’
  • When XSS/JS-injection is possible
    • Replace website logo with their competitor’s logo , or
    • Add the competitor logo after an ‘… a division of company Y…’ tag (Y are their competitor)
  • capture session tokens via insecure cookies and use session tokens to perform visible action (change user details, make transactions, delete user assets)
  • logout the user (continuously) via CSRF, so that the legit user is not able to login
  • Clickjacking that performs action on target website
  • DoS/DDoS the site (for a little bit) using small amounts of traffic (discovered after profiling the app via integration tests, which identified a number of expensive calls)

3.6.7 Draft notes - Threat Models

• Do Threat Models in layers
  • identify each STRIDE issue per layer
  • each layer it built on top of the previous one
  • connect the threats
  • map the Urls and Data objects and connect them across layers
  • created ‘refactored’ and ‘collapsed’ views of the diagrams (specially when there are ‘web services that act like proxies’ in the middle)
• Introduce the concept of Sinks in Threat Modeling
  • how to connect the multiple threats (one for each layer) so that they are chained
    • map this concept with the concept of Attack Trees
• Idea to create a Book focused on “Threat Modeling examples and patterns”

3.7 Audio Transcriptions

These files are a first pass at a particular topic, done as audio files, recorded on my mobile, and transcribed verbatim.

Some say that you know the vulnerability where user and account ID was received from the outer world that was basically passed into the back end and user and he was able to just use easy to be data.

The root cause of those problems tend to be the fact that the controller is actually able to access and make those calls. So you need to open a risk for that. Then you also need to open another risk for the fact that your need to create an account ID from the front end from user data.

So there is already implication of that dangerous operation. And the third one is that actually step of actually using that you know the one that creates a vulnerability, passing the risk directly to the back end with the violation.

Now when you fix this a lot of times to fix is done at the control level so you add the method there to say, “hey this user has access to this, this user has access to this account”.

The problem that is the wrong fix, it is a hack so you need to create a risk to that. Because the real fix should be done at the back end servers the real fix should be you should pass for example the user token or the back end and then use that to make a decision whether the user can access that information or not.

So that is a good example situation though the fix was formed, you actually the hack and also need to review other cases where that occurred. So you need to create a new ticket to be accepted to saying hey although we solved the problem it is actually not, we didn’t fix the root cause of the problem and then add that as code references.

How to deal with teams that say they are already secure and don’t have time for security.

So, every now and then you will find a team that has a lot of power to live with the level of applications.

And it is able to push back even at the very senior levels of the company to say we don’t need these security stuff, we don’t need these threat models, we don’t need all these security tasks, all these activities that you guys are asking us to do.

Assuming that the security isn’t really in attack space reading dragged by a team that is actually trying to do the right thing and is trying to push good practices and is actually trying to add value then the way to deal with these guys is to basically call their bluff.

The way to deal with that is to basically play them at their own game and to basically say well if you don’t do that it means that you are better reproducing really high secure code, you better have no security vulnerabilities, you better have no security exploits, you better fully understand your attack surface, you better have no side effects, very clean deployments all that jazz because those are all side effects of bad security practices.

So, basically what we then need to do is to document them, make them accept all those parts of risks, and then wait for Murphy’s Law to come along and to do that.

The other thing that is very important is that you also need to challenge the assumptions. So for example if they have pen tests, make sure they understand pen tests aren’t worth unless they are full white box.

The solution is to make them click on that accept to be supported. What is important to understand is that that isn’t the moment that they will accept the risk that is the moment where they will actually read it.

So that is very interesting long term gain that you play where it is all about changing the culture, it is all about finding ways to create a positive work flow.

So, sometimes you have to be aggressive, sometimes you have to be harsh and make teams accountable, because remember this game has already been played the question is whether you control instruments over the rules of the game or not.

And the thing to understand is that most people will only actually read it, and only actually pay attention the moment they have to be accountable.

So take that into account, don’t be frustrated when you have had [inaudible 00:01:13] about a topic and then only at that moment they actually realize what you are saying, realize that everybody is very busy, everybody has parts of stuff to do so actually the fact they aren’t paying attention isn’t really bad than what you were before.

As the insurance industry matures, and there are more and more companies that will choose to insure their risks instead of actually writing secure code and fix the applications is key that those values and those insurance premiums are made public. Because, those contracts will be a great measurement of how secure a company is.

The problem is we need ways to measure company’s security. We need ways to measure what they are doing so that we reward the client, so that the entity making a purchasing decision can choose A, B, and C.

So, we can’t have a situation where the only measuring sticks are features, reputation, cost and maybe some performance. We need security to be in there, we need companies to have in a way to pay a cost for not doing security.

Now, if the customer chooses to go with a particular vendor that clearly doesn’t have application security practices, clear is not doing the right thing that is fine that is okay that is market economics at work.

But my experience is that won’t happen, my experience is these days more and more the clients are getting way more savvy. And if they can make sure the multiple products that they are consuming, then they will put pressure and in a way they will vote with their decisions which is really what you want to see happening.

And a great way this will occur is as the insurance company gets along which will push a much better validation of the issues which will bring a huge amount of rationality and data points and data analysis into what is at risk, how you can measure, how you name it, how you define it, we need to make sure that those mappings and that information is all public.

And this is something that can easily not happen, but I feel that because the industry is still immature and is so young, we can actually point it in the right direction from now on.

Reducing risk to a number, a very interesting idea given to me by a CISO friend is to reduce all of AppSec and InfoSec activities into one number, which gets presented to the board, which then can be used to measure what is the current risk profile.

In a way the number is a collation of multiple numbers which are then easy to understand and easy to map. And this actually is very tightly connected to the idea of maturity models where you use different…you measure the maturity of the multiple teams or multiple systems or multiple products and then understand better who is doing what.

What I really like about the maturity models is that it allows the temperature to be defined in the actual model, so it allows the temperature much more objective and much more pragmatic way of actually looking at the problem and looking at the issues and that is basically a really nice way of controlling the temperature and applying pressure on the right places.

And also knowing where to act because when you look at the multiple patterns and the multiple activities in the maturity model, you see which activities are working or it isn’t working or is being done or not being done and that is a great way of analyzing the organization.

In fact even the individual items of the maturity model needs sometimes maturity models because when you say you have a security champion, the whole security champion world has in a way its own maturity model where in the beginning it is the binary, do you have one or not?

Then you start to look at how effective they are, how knowledgeable they are, how actually able they are to perform their duties same thing with code reviews, same thing with path chain, same thing with management of dependencies, same thing with threat analysis, all sorts of stuff.

All of those are basically things that you should measure the maturity model eventually leading to a higher one which eventually leads to a number.

So it is quite a nice work flow to do across the enterprise, it also scales really, really well.

As the majority of the AppSec world grows up, it is very key to make very clear to business that a typical black box pen-test i.e. test of website from the outside world who know inner knowledge is at most a waste of money, at best just a check to see if a kid or a non specific attacker has a go at your application whether you will find the job or the pen-test is to find the blind spot.

In fact the job of the security assessment is to find the blind spot. Is to take everything you know about the application or their threat models or the assumptions, in fact all those ticket items that your security champs have raised as being the problem we don’t have enough evidence and then find it.

So, in a way the job of security assessment is to improve your evidence, improve your evidence of the problem, or improve your evidence of how secure you are.

Which basically means that they need to have access to everything. And also it means that you need to capture everything they do, you need to capture in a way even the tests that they do which actually don’t produce the findings they are very important because they in a way are your regression tests.

And if you can capture that it means that the pen-tests or security assessment has added a lot more value to your system.

It also means you scale more, it means that they are something that you can introduce more often because it is something that you actually get a lot of business return versus this issue that you are going throw over the fence that you do it.

So this is also very good for pen-testing teams because they can move up the food chain, they can actually be a lot more valuable to the company versus being some disposable entity and you are going to have this race to the bottom to see who can do it cheaper and who can do it in a way that looks good, doesn’t necessarily mean that it is actually good.

And as attacks become much more real, as things become much more serious, you actually want to start to have the assurance of the pen-testing, companies need to move into a world where they provide assurance, where they provide proof of what they just found.

And that is why they need to be able to provide evidence on what they performed and that is why they need to have access to everything that exists on the application.

Then once you have the findings you can make a risk based decisions whether you think your current level of attackers are actually able to discover the vulnerabilities and which is more important are able to actually perform those actions in a way they can detect.

Because if you can detect how certain vulnerabilities that you have are exploited, and you can very quickly mitigate and prevent that for the damage then that is okay, then you can argue that you can actually leave them on because it is actually not a big deal.

An interesting thread that I have been having recently with security champions is that they really need to provide evidence for the tickets they open.

So it is quite nice when you have this work flow with the security champion opening up issues and they have been pract about it. In a way the next step is to start to provide very strong evidence about it and if they can that is an issue in itself.

And a good exercise for example for hackathons or for get togethers is actually come to the table. So for the security champions to come to the table and say, “hey, I think there is a problem, I think there is an issue can we prove it?”

Because that is the key of the game, you have to build a proof, you have to be able to provide evidence for the findings you are actually opening up. And you need to basically be able to allow the person who will actually accept the risk or make a decision to really understand what is going on.

Which is why the exploits these are critical, see the power of exploits there because that is so critical, you really need to have evidence and sometimes you need an exploit to really show this is how it could happen, this is how the problem is and sometimes you do in production, sometimes you do in QA it depends on the impact. The key of all this is evidence and proof.

Once you get high degree of code coverage, a really powerful technique that you can start to use is to start to run specific suite of tests and specific slices of the application and see what gets covered and what doesn’t get covered.

It is very important especially on things like APIs where you are able to understand what actually is tabled from the outside world and now you argue that especially from an API that is maintained as exposed, you want to make sure that you have no functionality in the application that actually doesn’t exist.

And this could be a problem with some of the code coverage techniques, code coverage practices where it is easier to leave code there that doesn’t get used any more but because it is tested, it is still in the application.

Basically you want to have almost zero tolerance for code that is currently not being used, any code that instead has no code paths or not being evoked should be eliminated from the application because you really want to make sure that the app is really in sync with its current capability and expectations and extra code be maintained, be reviewed and get rid if you don’t want it there.

The problem with users having all access all the time. One legacy that we still have from our days of running everything with admin and everything with a couple of users is this lack of segmentation of user access and especially user data access and in this case it can be a database access from a real application is what we really want is a situation where whenever somebody needs a particular asset, needs a particular resource, they ask for it and then they get unit token which are given use to access the resource.

The power of this is not only it enforces some of those access more explicit, it allows you place to control abuses.

So at the moment, the reason why there are sometimes so many data leaks and data dump is because it is possible for an application to dump all the data in one go or to just keep asking for it and nobody will probably notice it.

Where if that particular application or user had to generate tones or hundreds of tones or millions of tokens to access that that will be easy to monitor.

Because this is key, you cannot generate good visualization or good understanding of what is going on at the edges because there is too much there that is not easy.

What you can do is to visualize the patterns in a way the web services call the sequence of events or specific things which then will allow you to much better understand what is going on.

So if we did that for assets and accessing assets, we would be in a much better situation to understand when abuses occur.

The key challenges of becoming a developer are understanding concepts that are intrinsic to the development work flow. And in a way I would say version controls is probably one of the most important ones. And one area that I find a huge weakness in students and IT professionals is version control where because they have a lack of development experience, they have not experienced the power of version control.

And this is something that it is hard to explain until you experience this. It is one of those things that only when you do it, you really understand the power of a particular technology. In this case, of how to manage content in a distributive way.

The power of git and the reason it has been so successful for data version controls is that it radically changes the way you think, it radically changes the way you collaborate with creating the original source code but it could be anything. In fact there is amazing case all over the world of using git to manage laws to manage all sorts of things.

So it is already spreading but in a way not fast enough. So I think how to recommend somebody who is stuck in a loop of for example managing a word document or a spreadsheet to use git as a version control system. So even if in the beginning all they are doing is using git to manage the binaries either the files itself, they will already be exposed to central repositories, pushing data, git flows, git flow or git lap flow, pull requests, merge requests, etc.

Now, eventually what you want to see is a move into native and tech space data storage. So even for example somebody has excel spreadsheet, you would want to see a move to storing the actual data points into a format that can easily be read.

And it is important because lots of things I read, I mean easily be stored in a text format that is humanly read. And the reason this is important is because diffing and versionning is key when you have a team collaborating a real challenge is not in the first review, a real challenge is the second, third, fourth, fifth reviews where you don’t want to read the whole thing again you just want to see what changed, especially when you asked for changes to be made, at that time you are just verifying that did the person that made the change understand your request and did they make the change where you expect them to be.

At that moment in time reading the whole thing doesn’t work. And the same thing happens with code changes, if you think about that technology which is basically quite successful in track changes, you can track changes to really get how that works that is great but that is also like one or two levels you need a lot more than that.

And it also doesn’t have one of the peers of git which is push it the person making the request of the change to actually clean up and really make it easy for the person that is going to read and change through the work.

So when you store the data natively, or when you store the data in a text based format, you can then store the digitization part into spreadsheet. Which the logic will be that your visualization hasn’t really changed most of the time, what will change is the data points and recapturing so again that will scale.

And then eventually what you really want to do is move into more web based again code based kind of format where instead of using excel, using another web inter phase, using a website, using some technology like D3 or Vis, .js, and in a way that is a great part of the program where you are just taking a spreadsheet and codifying it on a git repo or on a website is already a great programming task.

So I highly recommend students and info-sec practitioners who want to get to coding to start there.

Notes