Your browser does not support JavaScript!

Code-aware combinatorial interaction testing

Code-aware combinatorial interaction testing

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Software — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Combinatorial interaction testing (CIT) is a useful testing technique to address the interaction of input parameters in software systems. CIT has been used as a systematic technique to sample the enormous test possibilities. Most of the research activities focused on the generation of CIT test suites as a computationally complex problem. Less effort has been paid for the application of CIT. To apply CIT, practitioners must identify the input parameters for the Software-under-test (SUT), feed these parameters to the CIT test generation tool, and then run those tests on the application with some pass and fail criteria for verification. Using this approach, CIT is used as a black-box testing technique without knowing the effect of the internal code. Although useful, practically, not all the parameters having the same impact on the SUT. This paper introduces a different approach to use the CIT as a gray-box testing technique by considering the internal code structure of the SUT to know the impact of each input parameter and thus use this impact in the test generation stage. The case studies results showed that this approach would help to detect new faults as compared to the equal impact parameter approach.


    1. 1)
      • 9. Colbourn, C.J., Syrotiuk, V.R.: ‘On a combinatorial framework for fault characterization’, Math. Comput. Sci., 2018, 12, (4), pp. 429451.
    2. 2)
      • 1. Kuhn, R., Kacker, R., Lei, Y., et al: ‘Combinatorial software testing’, Computer, 2009, 42, (8), pp. 9496.
    3. 3)
      • 12. Bures, M., Ahmed, B.S.: ‘On the effectiveness of combinatorial interaction testing: A case study’. 2017 IEEE Int. Conf. on Software Quality, Reliability and Security Companion (QRS-C), Prague, Czech Republic, July 2017, pp. 6976.
    4. 4)
      • 24. Shi, J., Cohen, M.B., Dwyer, M.B.: ‘Integration testing of software product lines using compositional symbolic execution’. Proc. 15th Int. Conf. on Fundamental Approaches to Software Engineering, ser. FASE'12, 2012, pp. 270284.
    5. 5)
      • 13. Tao, J., Li, Y., Wotawa, F., et al: ‘On the industrial application of combinatorial testing for autonomous driving functions’. 2019 IEEE Int. Conf. on Software Testing, Verification and Validation Workshops (ICSTW), Xi'an, China, April 2019, pp. 234240.
    6. 6)
      • 8. Nasser, A.B., Zamli, K.Z., Alsewari, A.A., et al: ‘An elitist-flower pollination-based strategy for constructing sequence and sequence-less t-way test suite’, Int. J. Bio-Inspired Comput., 2018, 12, (2), pp. 115127.
    7. 7)
      • 5. Ahmed, B.S., Abdulsamad, T.S., Potrus, M.Y.: ‘Achievement of minimized combinatorial test suite for configuration-aware software functional testing using the cuckoo search algorithm’, Inf. Softw. Technol., 2015, 66, pp. 1329.
    8. 8)
      • 7. Ahmed, B.S., Zamli, K.Z.: ‘A variable strength interaction test suites generation strategy using particle swarm optimization’, J. Syst. Softw., 2011, 84, (12), pp. 21712185.
    9. 9)
      • 20. Ahmed, B.S., Sahib, M.A., Gambardella, L.M., et al: ‘Optimum design of PID controller for an automatic voltage regulator system using combinatorial test design’, Plos One, 2016, 11, (11), pp. 120.
    10. 10)
      • 3. Tzoref-Brill, R.: ‘Chapter two – advances in combinatorial testing’, in Memon, A.M. (Ed.): ‘Ser. Advances in computers’, vol. 112 (Elsevier, Amsterdam, Netherlands, 2019), pp. 79134.
    11. 11)
      • 22. Deacon, D.C., Happe, C.L., Chen, C., et al: ‘Combinatorial interactions of genetic variants in human cardiomyopathy’, Nat. Biomed. Eng., 2019, 3, (2), pp. 147157.
    12. 12)
      • 10. Ahmed, B.S., Pahim, A., Junior, C.R.R., et al: ‘Towards an automated unified framework to run applications for combinatorial interaction testing’. Proc. Evaluation and Assessment on Software Engineering, ser. EASE ‘19, Copenhagen, Denmark, 2019, pp. 252258.
    13. 13)
      • 11. Yuan, X., Cohen, M.B., Memon, A.M.: ‘GUI interaction testing: incorporating event context’, IEEE Trans. Softw. Eng., 2011, 37, (4), pp. 559574.
    14. 14)
      • 4. Simos, D.E., Zivanovic, J., Leithner, M.: ‘Automated combinatorial testing for detecting SQL vulnerabilities in web applications’. Proc. 14th Int. Workshop on Automation of Software Test, ser. AST ‘19, Montreal, Canada, 2019, pp. 5561.
    15. 15)
      • 16. Nie, C., Leung, H.: ‘A survey of combinatorial testing’, ACM Comput. Surv., 2011, 43, (2), pp. 11:111:29.
    16. 16)
      • 19. Sulaiman, D.R., Ahmed, B.S.: ‘Using the combinatorial optimization approach for DVS in high performance processors’. 2013 The Int. Conf. on Technological Advances in Electrical, Electronics, and Computer Engineering (TAEECE), Konya, Turkey, May 2013, pp. 105109.
    17. 17)
      • 6. Hartman, A.: ‘Software and hardware testing using combinatorial covering suites’ (Springer, Boston, MA, USA, 2005), pp. 237266.
    18. 18)
      • 15. Ahmed, B.S., Zamli, K.Z., Afzal, W., et al: ‘Constrained interaction testing: A systematic literature study’, IEEE. Access., 2017, 5, pp. 2570625730.
    19. 19)
      • 2. Yilmaz, C.: ‘Test case-aware combinatorial interaction testing’, IEEE Trans. Softw. Eng., 2013, 39, (5), pp. 684706.
    20. 20)
      • 23. Demiroz, G., Yilmaz, C.: ‘Using simulated annealing for computing cost-aware covering arrays’, Appl. Soft Comput., 2016, 49, pp. 11291144.
    21. 21)
      • 21. Shasha, D.E., Kouranov, A.Y., Lejay, L.V., et al: ‘Using combinatorial design to study regulation by multiple input signals. a tool for parsimony in the post-genomics era’, Plant Physiol., 2001, 127, (4), pp. 15901594.
    22. 22)
      • 14. Kuhn, D.R., Kacker, R.N., Lei, Y.: ‘Introduction to combinatorial testing’ (Chapman & Hall/CRC, Boca Raton, FL, USA, 2013, 1st edn.).
    23. 23)
      • 18. Schubert, U.S.: ‘Experimental design for combinatorial and high throughput materials development. Edited by James N. Cawse’, Angew. Chem., Int. Ed., 2004, 43, (32), pp. 41234123.
    24. 24)
      • 17. Borodai, S.Y., Grunskii, I.S.: ‘Recursive generation of locally complete tests’, Cybern. Syst. Anal., 1992, 28, (4), pp. 504508.

Related content

This is a required field
Please enter a valid email address