Securing Browserid

This post is cross-posted from the Mozilla Web Application Security blog.

One of the important projects that Mozilla has been building in 2011 is BrowserID, a user-centric identity protocol and authentication service. Significant work has gone into building out and testing the infrastructure and protocol to make sure that it is a robust, open, and simple to adopt authentication scheme. If you want to learn more about BrowserID, here is a quick 12 minute video that explains what it is, and why we are doing this.

BrowserID continues to evolve as we build support for it across Mozilla properties and encourage adoption from 3rd parties. To date there almost 1000 different websites that rely on BrowserID, but we still have a long way to go to see large scale adoption!

Much of the effort we have put into reviewing the protocol and implementation of BrowserID is discussed in a recent presentation, a recording of which can be found below:

[Slides: html | html(zip) | pdf] (if the video doesn’t appear, click here!)

In addition to ongoing application and infrastructure security work in the next year, we are aiming for two significant milestones in 2012. First we will engage two third party security reviews of the Browserid.org site, and the cryptography used in BrowserID (including the protocol, the algorithms used, and the libraries we are relying on). Our objective in doing the third party review is to remain as transparent as possible in the development and review of the security aspects of BrowserID. This commitment to transparency includes:

  • opening up currently closed security bugs as the issues are resolved
  • publishing the results of the 3rd party review once high risk issues are addressed

As we proceed with this effort we will publish additional information on this blog, and we will work to keep the community up to date at each stage of progress.

Second, once we have completed the 3rd party review, resolved the issues identified, and published the results, BrowserID.org will become one of the properties fully covered by the Bug Bounty program (as always, exception bugs reported for none covered sites will be considered for bounty nomination).

Automating Test Cases

This post is cross-posted from the Mozilla Web Application Security blog.

Garmr is deprecated. This post is informational.

Earlier this year I wrote about some of the challenges of scaling security efforts in an organization, and I mentioned that we are working to adopt better tooling to assist us in this. We have been working towards improving security in the development lifecycle by making security tests a part of the quality assurance process. In order to accomplish this we worked with the QA team at Mozilla to create a simple tool called Garmr to integrate automated security test cases as part of our continuous integration (CI) processes.

When we started the discussion about which tools to use, our requirements were pretty straightforward; it needed to be fast, simple, and accurate. When we looked at Selenium and Mozmill the feedback we got from other teams is that although powerful, these frameworks were complex to configure properly, and had significant overhead to maintain as a project progresses. Since the focus was on testing web applications in a repeatable fashion, we identified a set of required functionality and David Burns from the very awesome Mozilla QA team put together the Garmr prototype and initial test cases.

Garmr has been public since its initial version, but the tool has now reached the point where we will start to leverage it in our testing and CI processes. The tool works by running a series of tests against target URLs and reporting back the results. The results are currently formatted as a JUnit style XML report that can be consumed by other tools such as Jenkins.

Garmr Test Cases

Garmr currently supports two types of test cases that can be authored, an Active Test and a Passive Test. ActiveTests are expected to make at least one HTTP request, and return the most relevant HTTP response after the check is completed. Here is an example of an ActiveTest that makes a simple HTTP GET request:

class WebTouch(ActiveTest):
    run_passives = True
    description = "Make a GET request to the specified URL, and check for a 200 response after resolving redirects."
    def do_test(self, url):
        response = requests.get(url)
        if response.status_code == 200:
            result = self.result("Pass", "The request returned an HTTP 200 response.", None)
        else:
            result = self.result("Fail", "The response code was %s" % response.status_code, None)
        return (result, response)

The test case makes an HTTP Get request, and passes or fails dependent on the response code. This very basic check does serve a purpose; it grabs an HTTP response to run passive tests against. For an example of a more complex check, look at the StsUpgradeCheck implemented in the corechecks module.

PassiveTest instances receive a copy of an HTTP response and analyze it to determine if the test passes or fails. In most circumstances, Garmr will run all of the configured PassiveTests against the result of each ActiveTest. The goal of a passive test is to inspect each the response of a query for properties that are expected from a secure web application.

This is an example of a passive test that checks an HTTP response for the presence of X-Frame-Options headers:

<code data-enlighter-language="python" class="EnlighterJSRAW">class XFrameOptionsPresent(PassiveTest):
    description = "Check if X-Frame-Options header is present."
    def analyze(self, response):
        xfoheader = "X-Frame-Options"
        xfo = xfoheader in response.headers
        if xfo == False:
            result = self.result("Fail", "X-Frame-Options header not found.", None)
        else:
            result = self.result("Pass", "X-Frame-Options header present.", response.headers[xfoheader])
        return result</code>

The goal is to allow developers to create test cases that are specific to their applications and gain the benefit of running pedantic checks for security related attributes such as cookies and headers each step of the way. Note that there is no practical limitation on what a passive check does, as long as the result object is returned properly.

Using Garmr

The current version of Garmr can be downloaded from the Garmr Github repository.

Getting, installing, and using Garmr is very easy:

git clone https://github.com/mozilla/Garmr.git
cd Garmr
sudo python setup.py install
garmr -u http://my.target.app

Limitations

Garmr is an alpha tool. The only part of it that we expect to remain stable at this point is the Xml Report format that it produces, since this is specified by the Jenkins tool chain that we are integrating with.

This tool is not intended to replace a typical dynamic or static analysis tool any more than the existence of Unit Tests replaces end to end functional testing, but it does provide a simple facility for QA, development, and security teams to collaborate on web application testing.

Currently the tool can be used to run a number of simple tests, and supports the ability to load modules; two examples of how a module would be written are included in djangochecks.py and webchecks.py

There is a list of features still to come:

  • less noisy CLI
  • proxy support (already supported in requests)
  • sessions (controlled; sequence for active tests, with a cookie jar that is propagated through the session)
  • detailed reporting, including the ability to record all HTTP requests and responses generated
  • the ability to filter which passive checks are run by check name or by check type (i.e. cookies, headers, content-type, etc)
  • support for additional protocols (websockets, spdy)
  • Implement instances of each test case for each target scanned to allow them to retain state as a set of tests progresses.

I am very excited to see how the tool will develop over the next few months as we have an intern who will be working to implement these and other features, so please test it out, give us feedback (patches and feature requests are welcome too!).

Scaling Security

This post is cross-posted from the Mozilla Web Application Security blog.

The AppSec space is an extremely challenging field to work in, largely due to asymmetry; when you play defence you have to work to stay on top of each emerging threat, vulnerability, and development that falls into your scope. Working to protect a system or application where there is fixed number of resources to spend on protecting a set of assets, choices have to be made about how to best spend those resources to prevent the attackers from winning. The best way to do that is by applying risk analysis techniques and focusing on the highest risk assets. Once those assets are identified, a decision has to be made about how to invest time and effort in design vs. implementation, static vs. dynamic analysis, and automated vs. manual testing. Regardless of the goal of continuous engagement within the SDLC, decisions are made based on the risk and the pool of limited resources must be split up to work towards a solid defence.

The biggest challenge is that we have a rapidly growing development community; while the security team is growing to meet our needs, we need to find better ways to scale testing and analysis to get the same results with better efficiency. Out of the gate, I am going to deal with one important issue by casually tossing it off to the side. Tooling is a really important part of the discussion, but the bottom line is that tools won’t make a difference in your organization if you don’t have the right people to use them. Good tools might help unskilled workers get good results, but skilled workers with suboptimal tools will still get great results. The adage “It’s a poor craftsman who blames his tools” sums it up neatly.

In order to scale up a team with limited resources (time, people, money), there are a number of things that can be done.

  • Threat Modeling/SDL activities are the best investment; you can fix many problems early, and eliminate extremely costly design weaknesses
  • Bug Bounties are a great way to reward the efforts of community contributors, but many contributions don’t come until the target is in production (i.e. the worst time to find bugs)
  • Automated tools for dynamic and static analysis allow you to trade time and money for results, but you still have to invest in the people to use them properly
  • Manual analysis consumes time and people; it takes a great deal of time and effort by skilled people to fix the problems
  • Training and Education requires investment of time, people, and money, and although valuable, is rarely as effective as working through threat modelling and SDL activities with your development team

Each of these types of activities are already in place in Mozilla, but there is still more we can do. We perform a great deal of manual testing because once we have reached that point in the development life cycle, it is the best way to find implementation or design issues that slipped through the cracks. One area we are investigating is how to make our manual testing and analysis repeatable and reusable.

Some things we plan to do to move in this direction include:

  • Building repeatable security test cases using tools like MozMill, Selenium, etc. where possible, and develop highly specific, but reusable guidance where it isn’t.
  • Run repeatable test cases as regression testing against apps in development and production
  • Identify high risk applications, frameworks, and components, and regularly review changes to them outside of releases and milestones
  • Investigate how to use static and dynamic analysis tools to supplement regression and manual testing to bring the best value

As we get these activities up and running, we will keep the community updated on how we are progressing.