Your browser does not support JavaScript!

An acceptance testing approach for Internet of Things systems

An acceptance testing approach for Internet of Things systems

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Software — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Internet of things (IoT) systems are becoming ubiquitous and assuring their quality is fundamental. Unfortunately, a few proposals for testing these complex, and often safety-critical, systems are present in the literature. The authors propose an approach for acceptance testing of IoT systems adopting graphical user interfaces as a principal way of interaction. Acceptance testing is a type of black box testing based on test scenarios, i.e. sequences of steps/actions performed by the user or the system. In their approach, test scenarios are derived from a state machine that expresses the behaviour of the system under test, and test cases are derived from them by specifying the actual data and assertions and made executable by implementing the corresponding test scripts. As a case study, they selected a mobile health IoT system for diabetes management composed of local sensors/actuators, smartphones, and a remote cloud-based system. The effectiveness of the approach has been evaluated by measuring the capability of two test suites implemented using different localisation strategies (visual and structure-based) in detecting mutants of the original m-health system. Results show the effectiveness of the test suites implemented by following the proposed approach since 93% of the generated mutants have been detected.


    1. 1)
      • 26. Pageobject pattern. Available at
    2. 2)
      • 19. Segura, S., Fraser, G., Sanchez, A. B., et al: ‘A survey on metamorphic testing’, IEEE Trans. Softw. Eng., 2016, 42, (9), pp. 805824.
    3. 3)
      • 33. Jia, Y., Harman, M.: ‘An analysis and survey of the development of mutation testing’, IEEE Trans. Softw. Eng., 2011, 37, (5), pp. 649678.
    4. 4)
      • 20. Node-RED. Available at
    5. 5)
      • 22. Android emulator. Available at
    6. 6)
      • 6. Parasoft. End-to-end testing for IoT integrity. Technical report. Available at
    7. 7)
      • 12. Islam, S. R., Kwak, D., Kabir, M. H., et al: ‘The internet of things for health care: a comprehensive survey’, IEEE Access, 2015, 3, pp. 678708.
    8. 8)
      • 15. Chen, T. Y., Ho, J.W., Liu, H., et al: ‘An innovative approach for testing bioinformatics programs using metamorphic testing’, BMC Bioinf., 2009, 10, (1), p. 24.
    9. 9)
      • 10. Stryker.Available at
    10. 10)
      • 14. Functional testing for IoT. Available at
    11. 11)
      • 31. Kochhar, P. S., Thung, F., Lo, D.: ‘Code coverage and test suite effectiveness: empirical study with real bugs in large systems’. Proc. 22nd Int. Conf. on Software Analysis, Evolution and Reengineering, IEEE, 2015, pp. 560564.
    12. 12)
      • 28. Leotta, M., Clerissi, D., Ricca, F., et al: ‘Capture-replay vs. programmable web testing: an empirical assessment during test case evolution’. Proc. 20th Working Conf. on Reverse Engineering, WCRE 2013, IEEE, 2013, pp. 272281.
    13. 13)
      • 27. Stocco, A., Leotta, M., Ricca, F., et al: ‘APOGEN: automatic page object generator for web testing’, Softw. Qual. J., 2017, 25, (3), pp. 10071039.
    14. 14)
      • 17. Utting, M., Legeard, B.: ‘Practical model-based testing: a tools approach’ (Morgan Kaufmann, New York, NY, USA, 2010).
    15. 15)
      • 2. Kim, H., Ahmad, A., Hwang, J., et al: ‘IoT-TaaS: towards a prospective IoT testing framework’, IEEE Access, 2018, 6, pp. 1548015493.
    16. 16)
      • 7. Leotta, M., Ricca, F., Clerissi, D., et al: ‘Towards an acceptance testing approach for internet of things systems’, in Garrigos I., Wimmer M., (Eds.): Proceedings of 1st International Workshop on Engineering the Web of Things (EnWoT 2017), LNCS, vol. 10544 (Springer, 2018) pp. 125–138.
    17. 17)
      • 1. The 3Vs. Available at
    18. 18)
      • 16. Leotta, M., Clerissi, D., Ricca, F., et al: ‘Approaches and tools for automated end-to-end web testing’, Adv. Comput., 2016, 101, pp. 193237.
    19. 19)
      • 23. Beizer, B.: ‘Software testing techniques’ (John Wiley & Sons, Inc., USA, 1990).
    20. 20)
      • 3. Rosenkranz, P., Wählisch, M., Baccelli, E., et al: ‘A distributed test system architecture for open-source IoT software’. Proc. 1st Workshop on IoT Challenges in Mobile and Industrial Systems, IoT-Sys 2015, ACM, 2015, pp. 4348.
    21. 21)
      • 25. Leotta, M., Stocco, A., Ricca, F., et al: ‘PESTO: automated migration of DOM-based web tests towards the visual approach’, J. Softw., Test. Verif. Reliab., 2018, p. e1665.
    22. 22)
      • 29. Cloc.Available at
    23. 23)
      • 4. Arrieta, A., Sagardui, G., Etxeberria, L., et al: ‘Automatic generation of test system instances for configurable cyber-physical systems’, Softw. Qual. J., 2017, 25, (3), pp. 10411083.
    24. 24)
      • 11. Klonoff, D. C.: ‘The current status of mHealth for diabetes: will it be the next big thing?’, J. Diabet. Sci. Technol., 2013, 7, (3), pp. 749758.
    25. 25)
      • 9. Sikuli.Available at
    26. 26)
      • 21. Bluemix. Available at
    27. 27)
      • 5. Silva, L. C., Perkusich, M., Bublitz, F. M., et al: ‘A model-based architecture for testing medical cyber-physical systems’. Proc. 29th Symp. on Applied Computing (SAC 2014). ACM, 2014, pp. 2530.
    28. 28)
      • 8. Appium.Available at
    29. 29)
      • 13. Istepanian, R., Hu, S., Philip, N., et al: ‘The potential of internet of m-health things ‘m-IoT’ for non-invasive glucose level sensing’. 33rd Int. Conf. of the IEEE Engineering in Medicine and Biology Society, EMBC 2011, Boston, MA, USA, 2011, pp. 52645266.
    30. 30)
      • 30. Offutt, A. J., Untch, R. H.: ‘Mutation 2000: uniting the orthogonal’, in Wong, E. (Ed.): ‘Mutation testing for the new century’ (Springer, Berlin, Heidelberg, 2001), pp. 3444.
    31. 31)
      • 18. Shalev-Shwartz, S.: ‘Online learning and online convex optimization’, Found. Trends Mach. Learn., 2012, 4, (2), pp. 107194.
    32. 32)
      • 24. Leotta, M., Stocco, A., Ricca, F., et al: ‘ROBULA+: an algorithm for generating robust XPath locators for web testing’, J. Softw.: Evol. Process., 2016, 28, (3), pp. 177204.
    33. 33)
      • 32. Grün, B. J. M., Schuler, D., Zeller, A.: ‘The impact of equivalent mutants’. Proc. Int. Conf. on Software Testing, Verification, and Validation Workshops, IEEE, ICSTW 2009, USA, 2009, pp. 192199.

Related content

This is a required field
Please enter a valid email address