For the first time in the history of software engineering, we can both broadly and deeply analyze the behavior and dynamics of software development projects. This has become possible because of open source, which is publicly developed software. In this presentation, I will discuss our recent findings about open source software, its development process, and programmer behavior. I also discuss the challenges we encountered when quantitatively mining software repositories for such insights.
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Six Easy Pieces of Quantitatively Analyzing Open Source
1. Six Easy Pieces (of Quantitatively Analyzing Open Source Software) Dirk Riehle SAP Research, SAP Labs LLC dirk@riehle.org, www.riehle.org, twitter.com/driehle
2.
3. Talk Overview (Agenda) The Growth of Open Source Software Data Mining for Fun and Profit Efficiently Estimating Commit Sizes Developer Activity in Open Source Software Projects 1. 2. 3. 5. The Commit Size Distribution of Open Source 4. The Commenting Practice of Open Source 6. Team Size Evolution in Open Source Projects 7. Conclusions 8.
4. The Growth of Open Source Software Amit Deshpande, Dirk Riehle. “The Total Growth of Open Source.” In Proceedings of the Fourth Conference on Open Source Systems (OSS 2008). Springer Verlag, 2008. Page 197-209. http://www.riehle.org/2008/03/14/the-total-growth-of-open-source/
6. Model of Source Code Growth where, y: Total open source lines of code x: Time from Jan 1995 to Dec 2006 in months 0.964 y = 2E+06*e 0.0464x Lower bound 0.961 y = 784098*e 0.0555x Upper bound R-square value Model Approach
8. Model of Project Growth where, y: Total number of open source projects x: Time from Jan 1995 to Dec 2006 in months 0.956 y = 7.1511e 0.0499x R-square value Model
9.
10. Data Mining for Fun and Profit Oliver Arafat, Amit Deshpande, Philipp Hofmann, Dirk Riehle. http://www.riehle.org/publications/
14. Efficiently Estimating Commit Sizes Philipp Hofmann, Dirk Riehle. “Estimating Commit Sizes Efficiently.” In Proceedings of the 5th International Conference on Open Source Systems (OSS 2009). Springer Verlag, 2009. Forthcoming. http://www.riehle.org/2009/02/11/estimating-commit-sizes-efficiently/
15.
16. What Diff Does 4,5c4,6 < d < f --- > e > e > e 7a9 > j 9d10 < n a b c e e e g h j m a b c d f g h m n 01: 02: 03: 04: 05: 06: 07: 08: 09: 10: 11: diff a.txt b.txt b.txt a.txt
17.
18.
19. Garden Variety of Heuristics 5.44 0 Linear Estimation 7 40.35 -5.95 Ldiff 6 30.87 -3.06 GNU diff –d 5 19.55 -1.96 GNU diff 4 7.68 -0.27 Bounds Mean 3 6.39 -4.41 Upper Bound 2 16.64 3.86 Lower Bound 1 Error Standard Deviation Error Mean Approach
22. The Commit Size Distribution of Open Source Oliver Arafat, Dirk Riehle. “The Commit Size Distribution of Open Source Software.” In Proceedings of the 42nd Hawaiian International Conference on System Science (HICSS-42). IEEE Press: 2009. Page 1-8. http://www.riehle.org/2008/09/23/ the-commit-size-distribution-of-open-source-software/
26. Developer Activity in Open Source Software Projects Dirk Riehle, Oliver Arafat, Amit Deshpande. “Developer Activity in Open Source Software Projects.” In preparation. Amit Deshpande, Dirk Riehle. “Continuous Integration in Open Source Software Development.” In Proceedings of the Fourth Conference on Open Source Systems (OSS 2008). Springer Verlag, 2008. Page 273-280. http://www.riehle.org/2008/03/08/ continuous-integration-in-open-source-software-development/
30. The Commenting Practice of Open Source Oliver Arafat, Dirk Riehle. “The Comment Density of Open Source Software Code.” In Companion to Proceedings of the 31st International Conference on Software Engineering (ICSE 2009). IEEE Press, 2009: Forthcoming. http://www.riehle.org/2009/02/04/ the-comment-density-of-open-source-software-code/
31.
32. Comment Density by Programming Language 273 7% 10% Perl 6. 534 8% 11% Python 5. 276 9% 16% Javascript 4. 1621 8% 18% C/C++ 3. 559 12% 22% php 2. 1085 11% 26% Java 1. Population Size Stddev [%] Average [%] Language #