An Historical Examination of Open Source Releases and Their Vulnerabilities
Edwards, Nigel; Chen, Liquin
Keyword(s): Security; Protection; Measurement; Static Analysis; Risk Analysis; Open Source Software
Abstract: This paper examines historical releases of Sendmail, Postfix, Apache httpd and OpenSSL by using static source code analysis and the entry-rate in the Common Vulnerabilities and Exposures dictionary (CVE) for a release, which we take as a measure of the rate of discovery of exploitable bugs. We show that the change in number and density of issues reported by the source code analyzer is indicative of the change in rate of discovery of exploitable bugs for new releases - formally we demonstrate a statistically significant correlation of moderate strength. The strength of the correlation is an artifact of other factors such as the degree of scrutiny: the number of security analysts investigating the software. This also demonstrates that static source code analysis can be used to make some assessment of risk even when constraints do not permit human review of the issues identified by the analysis. We find only a weak correlation between absolute values measured by the source code analyzer and rate of discovery of exploitable bugs, so in general it is unsafe to use absolute values of number of issues or issue densities to compare different applications or software. Our results demonstrate that software quality, as measured by the number of issues, issue density or number of exploitable bugs, does not always improve with each new release. However, generally the rate of discovery of exploitable bugs begins to drop three to five years after the initial release. Copyright ACM 2012. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in CCS'12, 19th ACM Conference on Computer and Communications Security, October 16 - 18, 2012, Raleigh, North Carolina, USA.
Additional Publication Information: To be published in CCS'12, 19th ACM Conference on Computer and Communications Security, October 16 - 18, 2012, Raleigh, North Carolina, USA.
External Posting Date: October 6, 2012 [Fulltext]. Approved for External Publication
Internal Posting Date: October 6, 2012 [Fulltext]