Tuesday, 25 October 2016

When Failed Tests are Good

When you make a code change, it is fundamental that every change you make breaks a test, or breaks something. You are testing for that behaviour; you are testing for the particular action that you are changing.

This means you should be happy when you make a change and the test fails, because you can draw confidence from knowing the side effects of the change. If you make a test change in one place and a couple of tests break, and you make another test change in a different place and fifty tests break, you get a far better sense of the impact of the changes that you made.

Security makes you a Better Developer

When you look at the development world from a security angle, you learn very quickly that you need to look deeper than a developer normally does. You need to understand how things occur, how the black magic works, and how things happen 'under the hood'. This makes you a better developer.

Studying in detail allows you to learn in an accelerated way. In a way, your brain does not learn well when it observes behaviour, but not cause. If you are only dealing with behaviour, you don't learn why something is happening, or the root causes of certain choices that were made in the app or the framework.

Monday, 24 October 2016

Chained threat models

When you create threat models per feature or per component, a key element is to start to chain them, i.e. create the connections between them. If you chain them in a sequence you will get a much better understanding of reality. You will be able to identify uber-vulnerabilities, or uber-threats, that are created by paths that exist from threat model, A to threat model B, to threat model C.

For example, I have seen threat models where one will say, "Oh, we get data from that over there. We trust their system, and they are supposed to have DOS protections, and they rate limit their requests".

Code Confidence Index

Most teams don't have confidence in their own code, in the code that they use, in the third parties, or the soup of dependencies that they have on the application. This is a problem, because the less confidence you have in your code, the less likely you are to want to make changes to that code. The more you hesitate to touch it, the slower your changes, your re-factoring, and your securing of the code will be.

To address this issue, we need to find ways to measure the confidence of code, in a kind of Code Confidence Index (COI).

Every Bug is an Opportunity

The power of having a very high degree of code coverage (97%+) is that you have a system where making changes is easy.

The tests are easy to fix, and you don't have an asymmetric code fixing problem, where a small change of code gives you a nightmare of test changes, or vice versa.

Instead, you get a very interesting flow where every bug, every security issue, or every code change is an opportunity to check the validity of your tests. Every time you make a code change, you want the tests to break. In fact you should worry if the tests don't break when you make code changes.

Sunday, 23 October 2016

Developers should be able to fire their managers

Many problems developer teams deal with arise from the inverted power structure of their working environment. The idea persists that the person managing the developers is the one who is ultimately in charge, responsible, and accountable.

That idea is wrong, because sometimes the person best-equipped to make the key technological decisions, and the difficult decisions, is the developer, who works hands-on, writing and reading the code to make sure that everything is correct.

A benefit of the 'Accept Risk' workflow, is that it pushes the responsibility to the ones that really matter. I've seen cases when upper-layers of management realise that they are not the ones that should be accepting that particular risk, since they are not the ones deciding on it. In theses cases usualy the decision comes down to the developers, who should use the opportunity to gain a bigger mandate to make the best decisions for the project.

Sometimes, a perverse situation occurs where the managers are no longer coding. They may have been promoted in the past because they used to be great programmers, or for other reasons, but now they are out of touch with programming and they no longer understand how it works.

Developer Teams Need Budgets

Business needs to trust developer teams.

Business needs to trust that developers want to do their best for their projects, and for their company.

If business doesn't learn to trust its developer teams, problems will emerge, productivity will be affected and quality/security will suffer.

A great way to show trust is to give the developer team a budget, and with it the power to spend money on things that will benefit the team.

Published "SecDevOps Risk Workflow" book (v0.60)

I just published version v0.60 of the SecDevOps Risk Workflow book.

You can get the book (for free) at https://leanpub.com/secdevops (when you become a reader you will get email alerts with every release)

The diff for this version (with v0.57) shows 138 changed files, 459 additions and 174 deletions.

Hyperlink everything you do

Whether you are a developer or a security person, it is crucial that you link everything you do, to a location where somebody can just click on a link and hit it. Make sure whatever you do is hyperlinkable.

This means that what you create is scalable, and it can be shared and found easily. It forms part of a workflow that is just about impossible, if you don't hyperlink your material.

An email is a black box, a dump of data which is a wasted opportunity because once an email is sent, it is difficult to find the information again. Yes, it is still possible to create a mess when you start to link things, connect things, and generate all sorts of data, but you are playing a better game. You are on a path that makes it much easier in the medium term for somebody to come in, click on the link, and make sure it is improved. It is a much better model.

Saturday, 22 October 2016

Getting Assurance and Trust from Application Security Tests

When you write an application security test, you ask a question. Sometimes the tests you do don't work, but the tests that fail are as important as the tests that succeed. Firstly, they tell you that something isn't there today so you can check it for the future. Secondly, they tell you the coverage of what you do.

These tests must pass, because they confirm that something is impossible. If you do a SQL injection test, in a particular page or field, or if you do an authorization test, and it doesn't work, you must capture that.

If you try something, and a particular vulnerability or exploit isn't working, the fact that it doesn't work is a feature. The fact that it isn't exploitable today is something that you want to keep testing. Your test confirms that you are not vulnerable to that and that is a powerful piece of information.

Sunday, 16 October 2016

Presenting "Surrogate dependencies" at London LSCC on 20th of October (Thursday)

Here is the description of this presentation:
Don't mock internal functions and methods, mock external dependencies. How to do that? This presentation will present a framework and practical example of creating Surrogate dependencies (think custom proxies, similar to WireMock). They are based on data collected from Integration tests to create environments where target applications can be executed offline and be subject to advanced security, quality and performance testing. All data is stored natively (JSON, XML) and Git is used for content versioning and simulation.
You can register at https://skillsmatter.com/meetups/8431-lscc-meetup (here is the meetup page)

This is going to be an variation of the Surrogate dependencies presentations delivered last month at the Owasp London Chapter

Published "SecDevOps Risk Workflow" Book (v0.57)

I just published version v0.57 of the (previously called) Jira Risk Workflow book.

You can get the book (for free) at https://leanpub.com/secdevops (when you become a reader you will get email alerts with every release)

As you probably noticed, there was a significant change in this release. The title of the book has been changed to 'SecDevOps Risk Workflow' (see here for the background story)

I hope you will agree that this change will represent better the direction of the book and the content I've been adding to it.

Saturday, 15 October 2016

AppSec memo from God

Having an Board level mandate is very important since it sends a strong message of AppSec importance.
The best way to provide a mandate to the existing AppSec team is to send a memo to the entire company, providing a vision for AppSec and re-enforcing its board-level visibility.
Sometimes called the 'Memo from God' the most famous one is Bill Gates 'Trustworthy computing' memo from January 2002 (responsible for making Microsoft turn the corner on AppSec)

Example of what it could look like

Here is a variation of a memo that I wrote for a CTO (in a project where I was leading the AppSec efforts) which contains the key points to make. 

Friday, 7 October 2016

Backlog Pit of Despair

In lots of development teams, especially in agile teams, the backlog has become a huge black hole of issues and features that get thrown into it and disappear. It is a mixed bag of things we might want to do in the future, so we store them in the backlog.

The job of the product backlog is to represent all the ideas that anyone in the application food chain has had about the product, the customer, and the sales team. The fact that these ideas are in the backlog means they aren’t priority tasks, but are still important enough that they are captured. Moving something into the backlog in this way, and identifying it as a future task, is a business decision.

Thursday, 6 October 2016

Who is actually making decisions?

One of the interesting situations that occurs when we play the risk acceptance game at large organisations, is how we are able to find out exactly who is making business and technical decisions.
Initially, when a ‘Risk Accepted’ request is made, what tends to happen is that the risk is pushed up the management chain, where each player pushes to their boss to accept risk. After all, what is at stake is who will take responsibility for that risk, and in some cases, who might be fired for it.
Eventually there is a moment when a senior director (or even the CTO) resists accepting the risk and pushes it down. What he is saying at that moment in time, is that the decision to accept that particular risk, has to be made by someone else, who has been delegated (officially or implicitly) that responsibility.

Risk Dashboards and emails

 It is critical that you create a suite of management dashboards that map the existing security metrics and the status of RISK tickets:
  • Open, In Progress
  • Awaiting Risk Acceptance, Risk Accepted
  • Risk Approved, Risk not Approved, Risk Expired
  • Allocated for Fix, Fixing, Test Fix
  • Fixed
Visualising this data makes it real and creates powerful dashboards. These need to be provided to all players in the software development lifecycle.

Email is not an Official Communication Medium

Emails are conversations, they are not official communication mediums. In companies, there is a huge amount of information and decisions that is only communicated using emails, namely:
  • risks,
  • to-dos
  • non-functional requirements
  • re-factoring needs
  • post-mortem analysis
This knowledge tends to only exist on an email thread or in a middle of document. That is not good enough. Email is mostly noise, and once something goes into an email, it is often very difficult to find it again.

"JIRA Risk Workflow" Book , alpha version published at Leanpub

To expand the ideas I presented with Using JIRA to manage RISKS,  I decided to create a smallish book focused on how that workflow works. 

A key objective is to document the workflow better and allow to teams to implement it using their own version of JIRA (I have done multiple presentations where one of the follow up questions is "Ok, we like it, now what")

You can get this book from Leanpub (with option to get it for free) at 

Please take a look and let me know what you think of the structure, font, layout, order, content, voice, idea, etc...

Tuesday, 4 October 2016

Survey about Security Champion program

Once you have a Security Champion (SC) program in place, you need to keep track of its effectiveness.

Here (see below) is a survey created by Vinod Anadan designed to learn from the current SCs and make it better

Any other questions we should be asking?

The AppSec team would like to conduct a survey about the Security Champion program.

Sunday, 2 October 2016

Use AppSec to visualise logs

Once you've got your logs, a typical challenge is how to process, and visualise them.
This usually happens when you try to address visualisation as a whole. But until good filters and multi-stage-analysis are in place, there is just too much data, too much information, and you will be left struggling with the sheer size of the data you are looking at.
The key is to create an environment where we are only querying an subset of the data, with fast queries and REPL like workflow.
One of the ways to manage this is to start with a small AppSec question (ideally codified as a Test).

Is your pentest delivering on AppSec?

Here is how to review a pentest and figure out if it is a network security assessment or an AppSec security review.

When you look at a pen test, you can tell very quickly if it was done by somebody who understands AppSec (somebody who can code), or by somebody who is approaching the problem from an network security point of view (usually running lots of tools).
The first main points to notice are if they asked for the source code, and if they performed an threat model on the target application. If they didn't, then it is most likely going to be a network security assessment.

Who is Paying for AppSec on open code?

When there isn't a commercial company behind an application or library, who is paying for:
  • secure development
  • secure coding standards,
  • threat models,
  • security reviews,
  • dependency management,
  • etc...
One of the interesting questions that arose when we talk about the need for open-source security coding technology, security coding centres, and everything we need to build secure code is: Who pays for it?

No server-site generation of dynamic web content

A very powerful design pattern that can provide a huge amount of security for web applications, is when there is no server-site generation of dynamic web content.
This avoids the pattern of:
  • starting with clean data objects on the server
  • merging code and data on the server
  • sending it over to the client as HTML
  • rebuilding it on the browser site for rendering and execution
The way to make this happen is to make all your web pages and content to be downloaded as static resources. This is done from a locked down server, ideally using git as the deploy mechanism. Data is provided to the UI via AJAX requests from dedicated WebServices.

Make sure your Security Champions are given time

It is very important that security champions are given the time, the focus, the mandate and the information required to do their jobs.

The good news is that now that you have security champions (at least one per team), their work will allow you to see the difference between the multiple teams and the parts of a company who are able to make it work, and those who are struggling make it happen.

Saturday, 1 October 2016

The currency of AppSec is provability and assurance

(from Software Quality book)

When we (AppSec) make a statement about a particular security issue, it always be clear and unambiguous.

We should never say "This might be a problem", or "That might be exploited". When it comes to problems or weaknesses in AppSec, we have to express ourselves with confidence.