Crossposts – Yvan Boily

Speeding Up Security Reviews

This post is cross-posted from the Mozilla Security blog.

At Mozilla we have a strong commitment to security; unfortunately due to the volume of work underway at Mozilla we sometimes have a bit of a backlog in getting security reviews done.

Want to speed up your security review request? You can dramatically increase the turn around time for your security review request by providing the information below. In addition to this, we are working to expand our overall security review process documentation; you can follow those efforts here.

1. Architecture Diagram

An architecture diagram illustrates how the various components of the service communicate with one another. This information allows the individual doing the security review to understand which services are required, how and where data is stored, and provides a general understanding of how the application or service works. Producing an architecture diagram is a good practice as it allows anyone to get a rapid view of how complex a system is, and can inform how much time it will take to work through a review of the system.

Examples

Note that these are just examples; the architecture diagram is intended to help the reviewer visualize what they are assessing. It doesn’t have to be a fancy diagram, and our team has worked from camera shots of whiteboards from meetings!

2. Detailed Application Diagram

A Detailed Application Diagram is essentially a Dataflow diagram; a data flow diagram enumerates each application or service that is a component of a system, and provides a list of the paths that data can flow through. A dataflow diagram helps the security reviewer to understand how data moves through the system, how different operations are performed, and if detailed enough, how different roles within the system access different operations.

While there are a number of different opinions on the “best way” to do a DFD, it is more helpful to have the information than it is to focus on presenting the information “the right way”. Examples

3. Dataflow Enumeration

An enumeration of data flows in the application explains how and what data moves between various components. Note that this doesn’t need to be a rigorous explanation of fields; in this case we want a general description of the message, the origin of the message (browser, third party, service, database, etc), the general contents (e.g. “description of the add-on”, “content to be shared”, etc), and a list of sensitive fields. The BrowserID Dataflow Enumeration is an excellent example.

4. Threat Analysis

The next step is reviewing all of this information to build out a list of the threats to an application. The important bit here is that you, as a developer or contributor, know how an application or system works. You know what a good set of the failure modes of the application are, and you understand the ‘business logic’ of the application. Many developers have a working knowledge of vulnerabilities, and can identify these types of issues. In order to properly perform a threat analysis a reviewer needs to understand how the various components of the system work, what threats exist, and be able to identify what mitigating controls have been put into place. Here is an example of what a threat analysis might look like (links below):

he threat analysis should contain, at a minimum the following information:

ID – a identifier for the threat
Title – a concise description of the threat
Threat – a description of the threat
Mitigations – a recommendation for a control that can be implemented
Threat Agent – a list of the potential actors considered that would exploit a vulnerability
Notes – Related comments that contribute to the analysis, but don’t belong in other columns
Rating – A qualitative scoring for a vulnerability in the context of this application
Impact – A qualitative score representing the impact should a vulnerability be exploited
Likelihood – A qualitative score representing the likelihood of a vulnerability being exploited

Additional information on how we assess and rate threats will be published as part of the documentation for our risk rating and security review process. Examples:

Help us help you!

Part of determining the scope of a security review is understanding how an application works and what the risks are; the documentation described in this post helps us to understand this and will ensure that we can complete a security review as quickly as possible. Beyond that, as teams understand how security reviews are performed it gives them the opportunity to take ownership of security and build it more effectively into their own processes.

As with other Mozilla teams we are actively pursuing better community engagement and always welcome feedback.

Mozilla Security AMA on Reddit

This post is cross-posted from the Mozilla Web Application Security blog.

Several members of the Mozilla Security team will participate in an AMA on reddit this morning. You can find it here!

Mozilla CTF Competition

This post is cross-posted from the Mozilla Web Application Security blog.

Announcing the Mozilla CTF hacking competition

Per the announcement during Frederik Braun’s presentation today, Mozilla will host a CTF event in January, 2012.

The Mozilla CTF will take place on January 25, 2012 (24 hours, PST). (Yes, this is a business day. Sorry)

The competition will be open for 24 hours to allow people from all over the world to participate.

Registration

This CTF is mainly aimed at the Mozilla Community and is designed to be accessible to people of all skill levels. There will be two registration phases, with the Community registration opening first. If you are a member of our community and are interested early registration, please reach out to your contacts within in Mozilla, or the CTF organizers (fbraun at mozilla.com, or yboily at mozilla.com)

Other players are still encouraged to join us, but please note that this CTF is mainly aimed at people who are new to Capture The Flags and the skill level has been adjusted accordingly.

Detailed information can be found at https://wiki.mozilla.org/Security/Events/CTF and you can watch this blog and follow @mozwebsec on twitter for updates.

Securing Browserid

This post is cross-posted from the Mozilla Web Application Security blog.

One of the important projects that Mozilla has been building in 2011 is BrowserID, a user-centric identity protocol and authentication service. Significant work has gone into building out and testing the infrastructure and protocol to make sure that it is a robust, open, and simple to adopt authentication scheme. If you want to learn more about BrowserID, here is a quick 12 minute video that explains what it is, and why we are doing this.

BrowserID continues to evolve as we build support for it across Mozilla properties and encourage adoption from 3rd parties. To date there almost 1000 different websites that rely on BrowserID, but we still have a long way to go to see large scale adoption!

Much of the effort we have put into reviewing the protocol and implementation of BrowserID is discussed in a recent presentation, a recording of which can be found below:

[Slides: html | html(zip) | pdf] (if the video doesn’t appear, click here!)

In addition to ongoing application and infrastructure security work in the next year, we are aiming for two significant milestones in 2012. First we will engage two third party security reviews of the Browserid.org site, and the cryptography used in BrowserID (including the protocol, the algorithms used, and the libraries we are relying on). Our objective in doing the third party review is to remain as transparent as possible in the development and review of the security aspects of BrowserID. This commitment to transparency includes:

opening up currently closed security bugs as the issues are resolved
publishing the results of the 3rd party review once high risk issues are addressed

As we proceed with this effort we will publish additional information on this blog, and we will work to keep the community up to date at each stage of progress.

Second, once we have completed the 3rd party review, resolved the issues identified, and published the results, BrowserID.org will become one of the properties fully covered by the Bug Bounty program (as always, exception bugs reported for none covered sites will be considered for bounty nomination).

Automating Test Cases

This post is cross-posted from the Mozilla Web Application Security blog.

Garmr is deprecated. This post is informational.

Earlier this year I wrote about some of the challenges of scaling security efforts in an organization, and I mentioned that we are working to adopt better tooling to assist us in this. We have been working towards improving security in the development lifecycle by making security tests a part of the quality assurance process. In order to accomplish this we worked with the QA team at Mozilla to create a simple tool called Garmr to integrate automated security test cases as part of our continuous integration (CI) processes.

When we started the discussion about which tools to use, our requirements were pretty straightforward; it needed to be fast, simple, and accurate. When we looked at Selenium and Mozmill the feedback we got from other teams is that although powerful, these frameworks were complex to configure properly, and had significant overhead to maintain as a project progresses. Since the focus was on testing web applications in a repeatable fashion, we identified a set of required functionality and David Burns from the very awesome Mozilla QA team put together the Garmr prototype and initial test cases.

Garmr has been public since its initial version, but the tool has now reached the point where we will start to leverage it in our testing and CI processes. The tool works by running a series of tests against target URLs and reporting back the results. The results are currently formatted as a JUnit style XML report that can be consumed by other tools such as Jenkins.

Garmr Test Cases

Garmr currently supports two types of test cases that can be authored, an Active Test and a Passive Test. ActiveTests are expected to make at least one HTTP request, and return the most relevant HTTP response after the check is completed. Here is an example of an ActiveTest that makes a simple HTTP GET request:

class WebTouch(ActiveTest):
    run_passives = True
    description = "Make a GET request to the specified URL, and check for a 200 response after resolving redirects."
    def do_test(self, url):
        response = requests.get(url)
        if response.status_code == 200:
            result = self.result("Pass", "The request returned an HTTP 200 response.", None)
        else:
            result = self.result("Fail", "The response code was %s" % response.status_code, None)
        return (result, response)

The test case makes an HTTP Get request, and passes or fails dependent on the response code. This very basic check does serve a purpose; it grabs an HTTP response to run passive tests against. For an example of a more complex check, look at the StsUpgradeCheck implemented in the corechecks module.

PassiveTest instances receive a copy of an HTTP response and analyze it to determine if the test passes or fails. In most circumstances, Garmr will run all of the configured PassiveTests against the result of each ActiveTest. The goal of a passive test is to inspect each the response of a query for properties that are expected from a secure web application.

This is an example of a passive test that checks an HTTP response for the presence of X-Frame-Options headers:

<code data-enlighter-language="python" class="EnlighterJSRAW">class XFrameOptionsPresent(PassiveTest):
    description = "Check if X-Frame-Options header is present."
    def analyze(self, response):
        xfoheader = "X-Frame-Options"
        xfo = xfoheader in response.headers
        if xfo == False:
            result = self.result("Fail", "X-Frame-Options header not found.", None)
        else:
            result = self.result("Pass", "X-Frame-Options header present.", response.headers[xfoheader])
        return result</code>

The goal is to allow developers to create test cases that are specific to their applications and gain the benefit of running pedantic checks for security related attributes such as cookies and headers each step of the way. Note that there is no practical limitation on what a passive check does, as long as the result object is returned properly.

Using Garmr

The current version of Garmr can be downloaded from the Garmr Github repository.

Getting, installing, and using Garmr is very easy:

git clone https://github.com/mozilla/Garmr.git
cd Garmr
sudo python setup.py install
garmr -u http://my.target.app

Limitations

Garmr is an alpha tool. The only part of it that we expect to remain stable at this point is the Xml Report format that it produces, since this is specified by the Jenkins tool chain that we are integrating with.

This tool is not intended to replace a typical dynamic or static analysis tool any more than the existence of Unit Tests replaces end to end functional testing, but it does provide a simple facility for QA, development, and security teams to collaborate on web application testing.

Currently the tool can be used to run a number of simple tests, and supports the ability to load modules; two examples of how a module would be written are included in djangochecks.py and webchecks.py

There is a list of features still to come:

less noisy CLI
proxy support (already supported in requests)
sessions (controlled; sequence for active tests, with a cookie jar that is propagated through the session)
detailed reporting, including the ability to record all HTTP requests and responses generated
the ability to filter which passive checks are run by check name or by check type (i.e. cookies, headers, content-type, etc)
support for additional protocols (websockets, spdy)
Implement instances of each test case for each target scanned to allow them to retain state as a set of tests progresses.

I am very excited to see how the tool will develop over the next few months as we have an intern who will be working to implement these and other features, so please test it out, give us feedback (patches and feature requests are welcome too!).

Scaling Security

This post is cross-posted from the Mozilla Web Application Security blog.

The AppSec space is an extremely challenging field to work in, largely due to asymmetry; when you play defence you have to work to stay on top of each emerging threat, vulnerability, and development that falls into your scope. Working to protect a system or application where there is fixed number of resources to spend on protecting a set of assets, choices have to be made about how to best spend those resources to prevent the attackers from winning. The best way to do that is by applying risk analysis techniques and focusing on the highest risk assets. Once those assets are identified, a decision has to be made about how to invest time and effort in design vs. implementation, static vs. dynamic analysis, and automated vs. manual testing. Regardless of the goal of continuous engagement within the SDLC, decisions are made based on the risk and the pool of limited resources must be split up to work towards a solid defence.

The biggest challenge is that we have a rapidly growing development community; while the security team is growing to meet our needs, we need to find better ways to scale testing and analysis to get the same results with better efficiency. Out of the gate, I am going to deal with one important issue by casually tossing it off to the side. Tooling is a really important part of the discussion, but the bottom line is that tools won’t make a difference in your organization if you don’t have the right people to use them. Good tools might help unskilled workers get good results, but skilled workers with suboptimal tools will still get great results. The adage “It’s a poor craftsman who blames his tools” sums it up neatly.

In order to scale up a team with limited resources (time, people, money), there are a number of things that can be done.

Threat Modeling/SDL activities are the best investment; you can fix many problems early, and eliminate extremely costly design weaknesses
Bug Bounties are a great way to reward the efforts of community contributors, but many contributions don’t come until the target is in production (i.e. the worst time to find bugs)
Automated tools for dynamic and static analysis allow you to trade time and money for results, but you still have to invest in the people to use them properly
Manual analysis consumes time and people; it takes a great deal of time and effort by skilled people to fix the problems
Training and Education requires investment of time, people, and money, and although valuable, is rarely as effective as working through threat modelling and SDL activities with your development team

Each of these types of activities are already in place in Mozilla, but there is still more we can do. We perform a great deal of manual testing because once we have reached that point in the development life cycle, it is the best way to find implementation or design issues that slipped through the cracks. One area we are investigating is how to make our manual testing and analysis repeatable and reusable.

Some things we plan to do to move in this direction include:

Building repeatable security test cases using tools like MozMill, Selenium, etc. where possible, and develop highly specific, but reusable guidance where it isn’t.
Run repeatable test cases as regression testing against apps in development and production
Identify high risk applications, frameworks, and components, and regularly review changes to them outside of releases and milestones
Investigate how to use static and dynamic analysis tools to supplement regression and manual testing to bring the best value

As we get these activities up and running, we will keep the community updated on how we are progressing.