Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Code-aware combinatorial interaction testing

  • Author(s): Bestoun S. Ahmed 1, 2 ; Angelo Gargantini 3 ; Kamal Z. Zamli 4 ; Cemal Yilmaz 5 ; Miroslav Bures 2 ; Marek Szeles 2
    • View affiliations
  • Source: Volume 13, Issue 6, December 2019, p. 600 – 609
    DOI:  10.1049/iet-sen.2018.5315 , Print ISSN 1751-8806, Online ISSN 1751-8814
© The Institution of Engineering and Technology
Received 21/08/2018, Accepted 19/07/2019, Revised 04/07/2019, Published 22/07/2019

Combinatorial interaction testing (CIT) is a useful testing technique to address the interaction of input parameters in software systems. CIT has been used as a systematic technique to sample the enormous test possibilities. Most of the research activities focused on the generation of CIT test suites as a computationally complex problem. Less effort has been paid for the application of CIT. To apply CIT, practitioners must identify the input parameters for the Software-under-test (SUT), feed these parameters to the CIT test generation tool, and then run those tests on the application with some pass and fail criteria for verification. Using this approach, CIT is used as a black-box testing technique without knowing the effect of the internal code. Although useful, practically, not all the parameters having the same impact on the SUT. This paper introduces a different approach to use the CIT as a gray-box testing technique by considering the internal code structure of the SUT to know the impact of each input parameter and thus use this impact in the test generation stage. The case studies results showed that this approach would help to detect new faults as compared to the equal impact parameter approach.

Inspec keywords: program testing; software fault tolerance; program verification

Other keywords: test generation stage; SUT; internal code structure; impact parameter approach; computationally complex problem; testing technique; software systems; systematic sampling technique; black-box testing technique; code-aware combinatorial interaction testing; grey-box testing technique; CIT tool; CIT test suites

Subjects: Software engineering techniques; Diagnostic, testing, debugging and evaluating systems

References

    1. 1)
      • 9. Colbourn, C.J., Syrotiuk, V.R.: ‘On a combinatorial framework for fault characterization’, Math. Comput. Sci., 2018, 12, (4), pp. 429–451.
    2. 2)
      • 1. Kuhn, R., Kacker, R., Lei, Y., et al: ‘Combinatorial software testing’, Computer, 2009, 42, (8), pp. 94–96.
    3. 3)
      • 12. Bures, M., Ahmed, B.S.: ‘On the effectiveness of combinatorial interaction testing: A case study’. 2017 IEEE Int. Conf. on Software Quality, Reliability and Security Companion (QRS-C), Prague, Czech Republic, July 2017, pp. 69–76.
    4. 4)
      • 24. Shi, J., Cohen, M.B., Dwyer, M.B.: ‘Integration testing of software product lines using compositional symbolic execution’. Proc. 15th Int. Conf. on Fundamental Approaches to Software Engineering, ser. FASE'12, 2012, pp. 270–284.
    5. 5)
      • 13. Tao, J., Li, Y., Wotawa, F., et al: ‘On the industrial application of combinatorial testing for autonomous driving functions’. 2019 IEEE Int. Conf. on Software Testing, Verification and Validation Workshops (ICSTW), Xi'an, China, April 2019, pp. 234–240.
    6. 6)
      • 8. Nasser, A.B., Zamli, K.Z., Alsewari, A.A., et al: ‘An elitist-flower pollination-based strategy for constructing sequence and sequence-less t-way test suite’, Int. J. Bio-Inspired Comput., 2018, 12, (2), pp. 115–127.
    7. 7)
      • 5. Ahmed, B.S., Abdulsamad, T.S., Potrus, M.Y.: ‘Achievement of minimized combinatorial test suite for configuration-aware software functional testing using the cuckoo search algorithm’, Inf. Softw. Technol., 2015, 66, pp. 13–29.
    8. 8)
      • 7. Ahmed, B.S., Zamli, K.Z.: ‘A variable strength interaction test suites generation strategy using particle swarm optimization’, J. Syst. Softw., 2011, 84, (12), pp. 2171–2185.
    9. 9)
      • 20. Ahmed, B.S., Sahib, M.A., Gambardella, L.M., et al: ‘Optimum design of PID controller for an automatic voltage regulator system using combinatorial test design’, Plos One, 2016, 11, (11), pp. 1–20.
    10. 10)
      • 3. Tzoref-Brill, R.: ‘Chapter two – advances in combinatorial testing’, in Memon, A.M. (Ed.): ‘Ser. Advances in computers’, vol. 112 (Elsevier, Amsterdam, Netherlands, 2019), pp. 79–134.
    11. 11)
      • 22. Deacon, D.C., Happe, C.L., Chen, C., et al: ‘Combinatorial interactions of genetic variants in human cardiomyopathy’, Nat. Biomed. Eng., 2019, 3, (2), pp. 147–157.
    12. 12)
      • 10. Ahmed, B.S., Pahim, A., Junior, C.R.R., et al: ‘Towards an automated unified framework to run applications for combinatorial interaction testing’. Proc. Evaluation and Assessment on Software Engineering, ser. EASE ‘19, Copenhagen, Denmark, 2019, pp. 252–258.
    13. 13)
      • 11. Yuan, X., Cohen, M.B., Memon, A.M.: ‘GUI interaction testing: incorporating event context’, IEEE Trans. Softw. Eng., 2011, 37, (4), pp. 559–574.
    14. 14)
      • 4. Simos, D.E., Zivanovic, J., Leithner, M.: ‘Automated combinatorial testing for detecting SQL vulnerabilities in web applications’. Proc. 14th Int. Workshop on Automation of Software Test, ser. AST ‘19, Montreal, Canada, 2019, pp. 55–61.
    15. 15)
      • 16. Nie, C., Leung, H.: ‘A survey of combinatorial testing’, ACM Comput. Surv., 2011, 43, (2), pp. 11:1–11:29.
    16. 16)
      • 19. Sulaiman, D.R., Ahmed, B.S.: ‘Using the combinatorial optimization approach for DVS in high performance processors’. 2013 The Int. Conf. on Technological Advances in Electrical, Electronics, and Computer Engineering (TAEECE), Konya, Turkey, May 2013, pp. 105–109.
    17. 17)
      • 6. Hartman, A.: ‘Software and hardware testing using combinatorial covering suites’ (Springer, Boston, MA, USA, 2005), pp. 237–266.
    18. 18)
      • 15. Ahmed, B.S., Zamli, K.Z., Afzal, W., et al: ‘Constrained interaction testing: A systematic literature study’, IEEE. Access., 2017, 5, pp. 25706–25730.
    19. 19)
      • 2. Yilmaz, C.: ‘Test case-aware combinatorial interaction testing’, IEEE Trans. Softw. Eng., 2013, 39, (5), pp. 684–706.
    20. 20)
      • 23. Demiroz, G., Yilmaz, C.: ‘Using simulated annealing for computing cost-aware covering arrays’, Appl. Soft Comput., 2016, 49, pp. 1129–1144.
    21. 21)
      • 21. Shasha, D.E., Kouranov, A.Y., Lejay, L.V., et al: ‘Using combinatorial design to study regulation by multiple input signals. a tool for parsimony in the post-genomics era’, Plant Physiol., 2001, 127, (4), pp. 1590–1594.
    22. 22)
      • 14. Kuhn, D.R., Kacker, R.N., Lei, Y.: ‘Introduction to combinatorial testing’ (Chapman & Hall/CRC, Boca Raton, FL, USA, 2013, 1st edn.).
    23. 23)
      • 18. Schubert, U.S.: ‘Experimental design for combinatorial and high throughput materials development. Edited by James N. Cawse’, Angew. Chem., Int. Ed., 2004, 43, (32), pp. 4123–4123.
    24. 24)
      • 17. Borodai, S.Y., Grunskii, I.S.: ‘Recursive generation of locally complete tests’, Cybern. Syst. Anal., 1992, 28, (4), pp. 504–508.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-sen.2018.5315
Loading

Related content

content/journals/10.1049/iet-sen.2018.5315
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address