Every day, countless Mozillians spend numerous hours testing Firefox to ensure that Firefox users get a stable and secure product. However, no product is bug free and, despite all of our testing efforts, browsers still crash sometimes. When we investigate our crash reports, some of them even look like lingering security issues (e.g. use-after-free or other memory corruptions) but the data we have in these reports is often not sufficient for them to be actionable on their own (i.e. they do not provide enough information for a developer to be able to find and fix the problem). This is particularly true for use-after-free problems and some other types of memory corruptions where the actual crash happens a lot later than the memory violation itself.
In our automated integration and fuzz testing, we have been using AddressSanitizer (ASan), a compile-time instrumentation, very successfully for over 5 years. The information it provides about use-after-free is much more actionable than a simple crash stack: It not only tells you immediately when the violation happens, but also includes the location where the memory was free’d previously.
In order to leverage the combined power of Nightly testing and ASan we have joined them together to form the ASan Nightly Project. For this purpose we made a custom ASan Nightly build that is equipped with a special ASan reporter addon. This addon is capable of collecting and reporting ASan errors back to Mozilla, once they are detected. We launched this project to find errors in the wild and then leverage the ASan error report to identify and fix the problem, even though it might not be reproducible. So far, we made these builds for Linux only, but we are actively working on Windows and Mac builds.
Of course this approach comes with a drawback: While ASan’s performance can almost compete with the performance of a regular build, its already higher memory usage grows the longer you run the browser as ASan needs to retain freed memory for a while in order to detect use-after-free on it. Hence, running such a build requires you to have enough RAM (at least 16 GB is recommended) and to restart the browser once or twice a day to free memory.
However, if you are willing to browse the web using this new Firefox environment, you might be eligible to earn a bug bounty: We will treat the automated reporter submissions as if they were filed in Bugzilla (with no test case) which means that if the issue is 1) an eligible security problem and 2) can be fixed by our developers, you will receive a bug bounty for it. All rules of the Mozilla Bug Bounty Program apply. If you would like to participate, ensure that you read the Bug Bounty section carefully and set the right preference, so your report can be attributed to you.
This project can only succeed if enough people are using it. So if you meet the current requirements, we would be very happy if you joined the project.