Test Suite Design Methodology Using Combinatorial Approach for Internet of Things Operating Systems

Abhinandan H. Patil¹, Neena Goveas¹, Krishnan Rangarajan²

¹Department of Computer Science and Information Systems, Birla Institute of Technology and Science, Goa, India.
²Department of Information Science, Dayanand Sagar College of Engineering, Bangalore, India.

DOI: 10.4236/jsea.2015.87031                

Abstract

In this paper we describe how the test design can be done by using the Combinatorial Testing approach for internet of things operating systems. Contiki operating system is taken as a case study but we discuss what can be the approach for RIOT and Tiny OS operating systems. We discuss how the combinatorial coverage measurement can be gathered in addition to the traditional metrics code coverage. The test design generated by using Advanced Combinatorial Testing for Software is analyzed for Contiki operating system. We elaborate the code coverage gathering technique for Contiki simulator which happens to be in Java. We explain the usage of Combinatorial Coverage Measurement tool. Although we have explained the test design methodology for internet of things operating systems, the approach explained can be followed for other open source software.

Keywords

CT, ACTS, CCM, Code Cover, Contiki, Cooja

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Patil, A.H., Goveas, N. and Rangarajan, K. (2015) Re-architecture of Contiki and Cooja Regression Test Suite Using Combinatorial Testing Approach. ACM SIGSOFT Software Engineering Notes, 40, 1-3.
[2]	Richard Kuhn, D., Kacker, R.N. and Lei, Y. (2013) Introduction to Combinatorial Testing. Text Book.
[3]	Richard Kuhn, D., Kacker, R.N. and Lei. Y. (2013) Practical Combinatorial Testing Manual. NIST Special Publications 800-142.
[4]	NIST. http://csrc.nist.gov/groups/SNS/acts/index.html
[5]	Nie, C., et al. (2011) A Survey of Combinatorial Testing. ACM Computing Surveys, Vol. 43, No. 2, Article 11.
[6]	Ammann, P. and Offutt, J. (2008) Introduction to Software Testing. Cambridge University Press, New York.http://dx.doi.org/10.1017/CBO9780511809163
[7]	Beizer, B. (1990) Software Testing Techniques. 2nd Edition, Van Nostrand Reinhold, New York.
[8]	ASTQB (2014) Introducing Combinatorial Testing in Large Organizations.
[9]	Bach, J. and Shroeder, P. (2004) Pairwise Testing—A Best Practice That Isn’t. Proceedings of 22nd Pacific Northwest Software Quality Conference, Portland, 180-196.
[10]	Baresi, L. and Young, M. (2001) Test Oracles. Department of Computer and Information Science, University of Oregon, Eugene. http://www.cs.uoregon.edu/michal/pubs/oracles.html
[11]	NourozBorazjany, M., Yu, L., Lei, Y., Kacker, R.N. and Kuhn, D.R. (2012) Combinatorial Testing of ACTS: A Case Study. 2012 IEEE 5th International Conference on Software Testing, Verification and Validation, 17-21 April 2012, Montreal, 971.
[12]	Combinatorial Coverage Measurement NASA IV&V Workshop, 11-13 September 2012.
[13]	Ammann, P. and Black, P.E. (1999) Abstracting Formal Specifications to Generate Software Tests via Model Checking. Proceedings of 18th Digital Avionics Systems Conference, 2, 10.A.6.1-10.A.6.10. http://dx.doi.org/10.1109/dasc.1999.822091
[14]	Kuhn, D.R., Higdon, J.M., Lawrence, J.F., Kacker, R.N. and Lei, Y. (2012) Combinatorial Methods for Event Sequence Testing. 5th International Workshop on Combinatorial Testing, Montreal, 17-21 April 2012, 601-609.
[15]	Lei, Y. (2007) IPOG—A General Strategy for t-Way Software Testing. 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, Tucson, 26-29 March 2007, 549-556.
[16]	Price, C., Kuhn, R., et al. (2013) Evaluating the t-Way Technique for Determining the Thoroughness of a Test Suite. NASA IV&V Workshop, Fairmont, September 2013.