An acceptance testing approach for Internet of Things systems

An acceptance testing approach for Internet of Things systems

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Software — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Internet of things (IoT) systems are becoming ubiquitous and assuring their quality is fundamental. Unfortunately, a few proposals for testing these complex, and often safety-critical, systems are present in the literature. The authors propose an approach for acceptance testing of IoT systems adopting graphical user interfaces as a principal way of interaction. Acceptance testing is a type of black box testing based on test scenarios, i.e. sequences of steps/actions performed by the user or the system. In their approach, test scenarios are derived from a state machine that expresses the behaviour of the system under test, and test cases are derived from them by specifying the actual data and assertions and made executable by implementing the corresponding test scripts. As a case study, they selected a mobile health IoT system for diabetes management composed of local sensors/actuators, smartphones, and a remote cloud-based system. The effectiveness of the approach has been evaluated by measuring the capability of two test suites implemented using different localisation strategies (visual and structure-based) in detecting mutants of the original m-health system. Results show the effectiveness of the test suites implemented by following the proposed approach since 93% of the generated mutants have been detected.


    1. 1)
      • 1. The 3Vs. Available at
    2. 2)
      • 2. Kim, H., Ahmad, A., Hwang, J., et al: ‘IoT-TaaS: towards a prospective IoT testing framework’, IEEE Access, 2018, 6, pp. 1548015493.
    3. 3)
      • 3. Rosenkranz, P., Wählisch, M., Baccelli, E., et al: ‘A distributed test system architecture for open-source IoT software’. Proc. 1st Workshop on IoT Challenges in Mobile and Industrial Systems, IoT-Sys 2015, ACM, 2015, pp. 4348.
    4. 4)
      • 4. Arrieta, A., Sagardui, G., Etxeberria, L., et al: ‘Automatic generation of test system instances for configurable cyber-physical systems’, Softw. Qual. J., 2017, 25, (3), pp. 10411083.
    5. 5)
      • 5. Silva, L. C., Perkusich, M., Bublitz, F. M., et al: ‘A model-based architecture for testing medical cyber-physical systems’. Proc. 29th Symp. on Applied Computing (SAC 2014). ACM, 2014, pp. 2530.
    6. 6)
      • 6. Parasoft. End-to-end testing for IoT integrity. Technical report. Available at
    7. 7)
      • 7. Leotta, M., Ricca, F., Clerissi, D., et al: ‘Towards an acceptance testing approach for internet of things systems’, in Garrigos I., Wimmer M., (Eds.): Proceedings of 1st International Workshop on Engineering the Web of Things (EnWoT 2017), LNCS, vol. 10544 (Springer, 2018) pp. 125–138.
    8. 8)
      • 8. Appium.Available at
    9. 9)
      • 9. Sikuli.Available at
    10. 10)
      • 10. Stryker.Available at
    11. 11)
      • 11. Klonoff, D. C.: ‘The current status of mHealth for diabetes: will it be the next big thing?’, J. Diabet. Sci. Technol., 2013, 7, (3), pp. 749758.
    12. 12)
      • 12. Islam, S. R., Kwak, D., Kabir, M. H., et al: ‘The internet of things for health care: a comprehensive survey’, IEEE Access, 2015, 3, pp. 678708.
    13. 13)
      • 13. Istepanian, R., Hu, S., Philip, N., et al: ‘The potential of internet of m-health things ‘m-IoT’ for non-invasive glucose level sensing’. 33rd Int. Conf. of the IEEE Engineering in Medicine and Biology Society, EMBC 2011, Boston, MA, USA, 2011, pp. 52645266.
    14. 14)
      • 14. Functional testing for IoT. Available at
    15. 15)
      • 15. Chen, T. Y., Ho, J.W., Liu, H., et al: ‘An innovative approach for testing bioinformatics programs using metamorphic testing’, BMC Bioinf., 2009, 10, (1), p. 24.
    16. 16)
      • 16. Leotta, M., Clerissi, D., Ricca, F., et al: ‘Approaches and tools for automated end-to-end web testing’, Adv. Comput., 2016, 101, pp. 193237.
    17. 17)
      • 17. Utting, M., Legeard, B.: ‘Practical model-based testing: a tools approach’ (Morgan Kaufmann, New York, NY, USA, 2010).
    18. 18)
      • 18. Shalev-Shwartz, S.: ‘Online learning and online convex optimization’, Found. Trends Mach. Learn., 2012, 4, (2), pp. 107194.
    19. 19)
      • 19. Segura, S., Fraser, G., Sanchez, A. B., et al: ‘A survey on metamorphic testing’, IEEE Trans. Softw. Eng., 2016, 42, (9), pp. 805824.
    20. 20)
      • 20. Node-RED. Available at
    21. 21)
      • 21. Bluemix. Available at
    22. 22)
      • 22. Android emulator. Available at
    23. 23)
      • 23. Beizer, B.: ‘Software testing techniques’ (John Wiley & Sons, Inc., USA, 1990).
    24. 24)
      • 24. Leotta, M., Stocco, A., Ricca, F., et al: ‘ROBULA+: an algorithm for generating robust XPath locators for web testing’, J. Softw.: Evol. Process., 2016, 28, (3), pp. 177204.
    25. 25)
      • 25. Leotta, M., Stocco, A., Ricca, F., et al: ‘PESTO: automated migration of DOM-based web tests towards the visual approach’, J. Softw., Test. Verif. Reliab., 2018, p. e1665.
    26. 26)
      • 26. Pageobject pattern. Available at
    27. 27)
      • 27. Stocco, A., Leotta, M., Ricca, F., et al: ‘APOGEN: automatic page object generator for web testing’, Softw. Qual. J., 2017, 25, (3), pp. 10071039.
    28. 28)
      • 28. Leotta, M., Clerissi, D., Ricca, F., et al: ‘Capture-replay vs. programmable web testing: an empirical assessment during test case evolution’. Proc. 20th Working Conf. on Reverse Engineering, WCRE 2013, IEEE, 2013, pp. 272281.
    29. 29)
      • 29. Cloc.Available at
    30. 30)
      • 30. Offutt, A. J., Untch, R. H.: ‘Mutation 2000: uniting the orthogonal’, in Wong, E. (Ed.): ‘Mutation testing for the new century’ (Springer, Berlin, Heidelberg, 2001), pp. 3444.
    31. 31)
      • 31. Kochhar, P. S., Thung, F., Lo, D.: ‘Code coverage and test suite effectiveness: empirical study with real bugs in large systems’. Proc. 22nd Int. Conf. on Software Analysis, Evolution and Reengineering, IEEE, 2015, pp. 560564.
    32. 32)
      • 32. Grün, B. J. M., Schuler, D., Zeller, A.: ‘The impact of equivalent mutants’. Proc. Int. Conf. on Software Testing, Verification, and Validation Workshops, IEEE, ICSTW 2009, USA, 2009, pp. 192199.
    33. 33)
      • 33. Jia, Y., Harman, M.: ‘An analysis and survey of the development of mutation testing’, IEEE Trans. Softw. Eng., 2011, 37, (5), pp. 649678.

Related content

This is a required field
Please enter a valid email address