Temporal Patterns of Software Evolution Defects: A Comparative Analysis of Open Source and Closed Source Projects

Uzma Raja, Joanne Elaine Hale, David Peter Hale

.

DOI: 10.4236/jsea.2011.48058                

Abstract

This study examines temporal patterns of software systems defects using the Autoregressive Integrated Moving Average (ARIMA) approach. Defect reports from ten software application projects are analyzed; five of these projects are open source and five are closed source from two software vendors. Across all sampled projects, the ARIMA time series modeling technique provides accurate estimates of reported defects during software maintenance, with organizationally dependent parameterization. In contrast to causal models that require extraction of source-code level metrics, this approach is based on readily available defect report data and is less computation intensive. This approach can be used to improve software maintenance and evolution resource allocation decisions and to identify outlier projects—that is, to provide evidence of unexpected defect reporting patterns that may indicate troubled projects.

Keywords

Open Source Software, Software Defects, Software Maintenance, Time Series Analysis

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	P. Van Roy, “Self Management and the Future of Software Design,” Electronic Notes in Theoretical Computer Science, Vol. 182, No. 29, 2007, pp. 201-217. doi:10.1016/j.entcs.2006.12.043
[2]	E. Figueiredo, N. Cacho, C. Sant’Anna, M. Monteiro, U. Kulesza, A. Garcia, S. Soares, F. Ferrari, S. Khan, F. Castor Filhoand and F. Dantas, “Evolving Software Product Lines with Aspects: An Empirical Study on Design Stability,” Proceedings of the 30th international Conference on Software Engineering, Leipzig, 2008, pp. 261-270.
[3]	L. Wallace and M. Keil, “Software Project Risks and Their Effect on Outcomes,” Communications of the ACM, Vol. 47, No. 4, 2004, pp. 68-73. doi:10.1145/975817.975819
[4]	B. P. Lientz, E. B. Swanson and G. E. Tompkins, “Characteristics of Application Software Maintenance,” Communications of the ACM, Vol. 21, No. 6, 1978, pp. 466- 471. doi:10.1145/359511.359522
[5]	T. M. Pigoski, “Practical Software Maintenance: Best Practices for Managing Your Software Investment,” John Wiley & Sons, Inc., Hoboken, 1996.
[6]	ISO/IEC14764, “Software Engineering—Software Life Cycle Processes—Maintenance,” International Organization for Standardization, 2006.
[7]	G. Stark and P. Oman, “Software Maintenance Strategies: Observations from the Field,” Journal of Software Maintenance, Vol. 9, No. 6, 1997, pp. 365-378. doi:10.1002/(SICI)1096-908X(199711/12)9:6<365::AID-SMR160>3.3.CO;2-A
[8]	S. Chulani, B. Ray, P. Santhanam and R. Leszkowicz, “Metrics for Managing Customer View of Software Quality,” Proceedings. Ninth International Software Metrics Symposium, Sydney, 2003, pp. 189-198.
[9]	L. Pelayo and S. Dick, “Applying Novel Resampling Strategies to Software Defect Prediction,” Proceedings of the Meeting of the North American Fuzzy Information Processing Society, NAFIPS, San Diego, 2007, pp. 69-72.
[10]	J. Asundi and S. Sarkar, “Staffing Software Maintenance and Support Projects,” Proceedings of the 38th Annual Hawaii International Conference on System Sciences, HICSS’05, Big Island, 2005, p. 314b.
[11]	T. Chan, “Beyond Productivity in Software Maintenance: Factors Affecting Lead Time in Servicing Users’ Requests,” Proceedings of the International Conference on Software Maintenance, San Jose, 2000, pp. 228-237.
[12]	G. Antoniol, M. Di Penta and E. Merlo, “An Automatic Approach to Identify Class Evolution Discontinuities,” Proceedings of the 7th International Workshop on Principles of Software Evolution, Kyoto, 2004, pp. 31-40. doi:10.1109/IWPSE.2004.1334766
[13]	M. C. Ohlsson, A. Amschler Andrews and C. Wohlin, “Modelling Fault-Proneness Statistically over a Sequence of Releases: A Case Study,” Journal of Software Maintenance, Vol. 13, No. 3, 2001, pp. 167-199. doi:10.1002/smr.229
[14]	T. M. Khoshgoftaar and D. L. Lanning, “A Neural Network Approach for Early Detection of Program Modules Having High Risk in the Maintenance Phase,” Journal of Systems and Software, Vol. 29, No. 2, 1995, pp. 85-91. doi:10.1016/0164-1212(94)00130-F
[15]	K. El-Emam and O. Laitenberger, “Evaluating Capture- Recapture Models with Two Inspectors,” IEEE Transactions on Software Engineering, Vol. 27, No. 9, 2001, pp. 851-864. doi:10.1109/32.950319
[16]	T. M. Khoshgoftaar, B. Bhattacharyya and G. Richardson, “Predicting Software Errors, During Development, Using Nonlinear Regression Models: A Comparative Study,” IEEE Transactions on Reliability, Vol. 41, No. 3, 1992, pp. 390-395. doi:10.1109/24.159804
[17]	N. E. Fenton and M. Neil, “A Critique of Software Defect Prediction Models,” IEEE Transactions on Software Engineering, 25, No. 5, 1999, pp. 675-689. doi:10.1109/32.815326
[18]	E. N. Adams, “Optimizing Preventive Service of Software Products,” IBM Journal of Research and Development, Vol. 28, No. 1, 1984, pp. 2-14. doi:10.1147/rd.281.0002
[19]	M. S. Krishnan and M. I. Kellner, “Measuring Process Consistency: Implications for Reducing Software Defects,” IEEE Transactions on Software Engineering, Vol. 25, No. 6, 1999, pp. 800-815. doi:10.1109/32.824401
[20]	M. S. Krishnan, “The Role of Team Factors in Software Cost and Quality: An Empirical Analysis,” Information Technology & People, Vol. 11, No. 1, 1998, pp. 20-35. doi:10.1108/09593849810204512
[21]	M. Hecht and J. Handal, “A Discrete-Event Simulator for Predicting Outage Time and Costs as a Function of Maintenance Resources,” Reliability and Maintainability Symposium Proceedings. Annual, Seattle, 2002, pp. 612-617.
[22]	M. M. T. Thwin and T.-S. Quah, “Application of Neural Networks for Software Quality Prediction Using Object-Oriented Metrics,” Journal System and Software, Vol. 76, No. 2, 2005, pp. 147-156. doi:10.1016/j.jss.2004.05.001
[23]	A. Tsakonas and G. Dounias, “Predicting Defects in Software Using Grammar-Guided Genetic Programming,” Proceedings of the 5th Hellenic conference on Artificial Intelligence, SETN, Syros, 2008, pp. 413-418.
[24]	S. Dick and A. Sadia, “Fuzzy Clustering of Open-Source Software Quality Data: A Case Study of Mozilla,” IJCNN ’06. International Joint Conference on Neural Networks, Vancouver, 2006, pp. 4089-4096. doi:10.1109/IJCNN.2006.246954
[25]	T. M. Khoshgoftaar, E. B. Allen, W. D. Jones and J. I. Hudepohl, “Classification Tree Models of Software Quality over Multiple Releases,” Proceedings. 10th International Symposium in Software Reliability Engineering, IEEE Reliability Society, 1999, pp. 116-125.
[26]	N. Seliya, T. M. Khoshgoftaar and S. Zhong, “Analyzing Software Quality with Limited Fault-Proneness Defect Data,” Ninth IEEE International Symposium on High-Assurance Systems Engineering, HASE, Heidelberg, 2005, pp. 89-98.
[27]	H. Zhang, S. Jarzabek and B. Yang, “Quality Prediction and Assessment for Product Lines,” Proceedings of the 15th International Conference on Advanced Information Systems Engineering, Klagenfurt, 2003, pp. 681-695.
[28]	T. Menzies, J. Greenwald and A. Frank, “Data Mining Static Code Attributes to Learn Defect Predictors,” IEEE Transactions on Software Engineering, Vol. 32, No. 1, 2007, pp. 2-13. doi:10.1109/TSE.2007.256941
[29]	V. U. B. Challagulla, F. B. Bastani, I.-L. Yen and R. A. Paul, “Empirical Assessment of Machine Learning Based Software Defect Prediction Techniques,” IEEE Int’l Workshop on Object-oriented Real-time Dependable Systems (WORDS), Sedona, 2005, pp. 282-303.
[30]	Q. Song, M. Shepperd, M. Cartwright and C. Mair, “Software Defect Association Mining and Defect Correction Effort Prediction,” IEEE Transactions on Software Engineering, Vol. 32, No. 2, 2006, pp. 69-82. doi:10.1109/TSE.2006.1599417
[31]	S. H. Aljahdali, A. Sheta and D. Rine, “Prediction of Software Reliability: A Comparison between Regression and Neural Network Non-Parametric Models,” ACS/IEEE International Conference on Computer Systems and Applications, Beirut, 2001, pp. 470-473. doi:10.1109/AICCSA.2001.934046
[32]	W. S. Humphrey and N. D. Singpurwalla, “Predicting (Individual) Software Productivity,” IEEE Transactions on Software Engineering, Vol. 17, No. 2, 1991, pp. 196-207. doi:10.1109/32.67600
[33]	A. Mital, A. Desai, A. Subramanian and A. Mital, “Product Development: A Structured Approach to Consumer Product Development, Design, and Manufacture,” Butterworth-Heinemann, 2008.
[34]	P. H. Franses, “Periodicity and Stochastic Trends in Economic Time Series,” Oxford University Press, Oxford, 1996.
[35]	E. Ghysels and D. R. Osborn, “The Econometric Analysis of Seasonal Time Series,” Cambridge University Press, Cambridge, 2001.
[36]	U. Raja, D. P. Hale and J. E. Hale, “Modeling Software Evolution Defects: A Time Series Approach,” Journal of Software Maintenance and Evolution: Research and Practice, Vol. 21, No. 6, 2009, pp. 49-71. doi:10.1002/smr.398
[37]	C. F. Kemerer and S. A. Slaughter, “A Longitudinal Analysis of Software Maintenance Patterns,” ICIS International Conference on Information Systems Proceedings of the eighteenth ICIS, Atlanta, 1997, pp. 476-477.
[38]	B. Kenmei, G. Antoniol and M. Di Penta, “Trend Analysis and Issue Prediction in Large-Scale Open Source Systems,” 12th European Conference on Software Maintenance and Reengineering, CSMR. 2008, pp. 73-82.
[39]	C. Chatfield, “The Analysis of Time Series: An Introduction (6th Edition),” Chapman and Hall/CRC, London, 2003.
[40]	G. Box and G. Jenkins, “Time Series Analysis: Forecasting and Control,” Holden-Day, San Francisco, 1970.
[41]	R. R. Picard and R. D. Cook, “Cross-Validation of Regression Models,” Journal of the American Statistical Association, Vol. 79, No. 387, 1984, pp. 575-583. doi:10.2307/2288403
[42]	I. Herraiz, J. M. Gonzalez-Barahona, G. Robles and D. M. German, “On the Prediction of the Evolution of Libre Software Projects,” IEEE International Conference on Software Maintenance, (ICSM 2007), Paris, 2007, pp. 405-414. doi:10.1109/ICSM.2007.4362653
[43]	G. M. Ljung and G. E. P. Box, “On a Measure of Lack of fit in Time Series Models,” Biometrika Trust, Vol. 65, No. 2, 1978, pp. 297-303. doi:10.1093/biomet/65.2.297
[44]	K. Gregory, “Native-View Paradigms: Multiple Cultures and Culture Conflicts in Organizations,” Administrative Science Quarterly, Vol. 28, No. 3, 1983, pp. 359-376. doi:10.2307/2392247
[45]	D. Thomas, “Hacker Culture,” University of Minnesota Press, Minneapolis, 2003.
[46]	K. Crowston and B. Scozzi, “Open Source Software Projects as Virtual Organisations: Competency Rallying for Software Development,” IEE Proceedings Software, Vol. 149, 2002, pp. 3-17. doi:10.1049/ip-sen:20020197
[47]	T. T. Dinh-Trong and J. M. Bieman, “The FreeBSD Project: A Replication Case Study of Open Source Development,” IEEE Transactions on Software Engineering, Vol. 31, No. 6, 2005, pp. 481-494. doi:10.1109/TSE.2005.73
[48]	F. N. Kerlinger and H. B. Lee, “Foundations of Behavioral Research,” 4th Edition, Wadsworth, New York, 1999.