Calculating return on investment of training using process variation

Author(s): S. Matalonga and T. San Feliu
DOI: 10.1049/iet-sen.2011.0024

For access to this article, please select a purchase option:

Buy article PDF

Buy Knowledge Pack

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership

Recommend Title Publication to library

IET Software — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Author(s): S. Matalonga ¹ and T. San Feliu ²
- Affiliations: 1: Facultad de Ingeniería, Universidad ORT Uruguay, Montevideo, Uruguay
  2: Facultad de Informática, Universidad Politécnica de Madrid, Boadilla del Monte, Spain
Source: Volume 6, Issue 2, April 2012, p. 140 – 147
DOI: 10.1049/iet-sen.2011.0024 , Print ISSN 1751-8806, Online ISSN 1751-8814

Organisations have relied on training to increase the performance of their workforce. Also, software process improvement models suggest that training is an effective tool for institutionalising a development process. Training evaluation becomes important for understanding the improvements resulting from the investments in training. Like other production process, the software development process is subject to natural and special causes of variation, and process improvement models recommend its statistical management. Return on investment (ROI) has already been proposed as an effective measure to evaluate training interventions. Nevertheless, when applying ROI in production environments, practitioners have not taken into consideration the effects of variation in production processes. This study presents a method for calculating ROI that considers process variation; the authors argue that ROI results should be understood in accordance to statistical management guidance. The proposed method has been piloted at a software factory. The results of the case study are reported. These results show how to calculate ROI by taking into account the variation in a production process.

References

1. 1)
  - M. Diaz , J. Sligo . How software process improvement helped Motorola. IEEE Softw. , 5 , 75 - 81
2. 2)
  - J. Herbsleb , A. Carleton , J. Rozum , J. Siegel , D. Zubrow . (1994) Benefits of CMM-based software process improvement: initial results.
3. 3)
  - W.I. Zangwill , P.B. Kantor . Toward a theory of continuous improvement and the learning curve. Manage. Sci. , 7 , 910 - 920
4. 4)
  - E.F. Holton . The flawed 4-level evaluation model. Human Resour. Dev. Quart. , 1 , 5 - 21
5. 5)
  - P.S. Adler , K.B. Clark . Behind the learning curve: a sketch of the learning process. Manage. Sci. , 3 , 267 - 281
6. 6)
  - G. Glass , R.M. Jager . (1997) Interrupted time series quasi-experiments: complementary methods for research in education.
7. 7)
  - S. Shelton , G. Alliger . Who's afraid of level 4 evaluation? A practical approach. Train. Dev. , 6 , 43 - 46
8. 8)
  - Goldenson, D.: `The state of software measurement practice', Proc. Europe SEPG 08, 12–14 June 2008, Munich, Germany.
9. 9)
  - D.L. Kirkpatrick , J.D. Kirkpatrick . (2006) Evaluating training programs: the four levels.
10. 10)
  - G.G. Wang , Z. Dou , N. Li . A systems approach to measuring return on investment for HRD interventions. Human Resour. Dev. Quart. , 2 , 203 - 224
11. 11)
  - A. Field , G. Hole . (2003) How to design and report experiments. (7).
12. 12)
  - V.R. Basili . (1992) Software modeling and measurement: the goal/question/metric paradigm.
13. 13)
  - W.S. Humphrey . (2001) Winning with software: an executive strategy.
14. 14)
  - G.M. Alliger , S.I. Tannenbaum , W. Bennett , H. Traver , A. Shotland . A meta-analysis of the relations among training criteria. Pers. Psychol. , 2 , 341 - 358
15. 15)
  - D.J. Wheeler . (1993) Understanding variation: the key to managing chaos.
16. 16)
  - R. Van Solingen . Measuring the ROI of software process improvement. IEEE Softw. , 3 , 32 - 38
17. 17)
  - R.A. Swanson , E.F. Holton . (2009) Foundations of human resource development.
18. 18)
  - T. Hodges . (2002) Linking learning and performance.
19. 19)
  - Ilian, H.: `Levels of levels: making Kirkpatrick fit the facts and the facts fit Kirkpatrick', Proc. 6th Annual Human Services Training, 2004.
20. 20)
  - J.J. Phillips . (2003) Return on investment in training and performance improvement programs: improving human performance.
21. 21)
  - D.T. Campbell , J.C. Stanley . (1963) Experimental and quasi-experimental designs for research on teaching.
22. 22)
  - D. Russ-Eft , H. Preskill . In search of the Holy Grail: return on investment evaluation in human resource development. Adv. Dev. Human Resour. , 1
23. 23)
  - (2000) Quality management systems – requirements.
24. 24)
  - N.C. David , D. Kevin , D. Glyn . Making statistics part of decision making in an engineering organization. IEEE Softw. , 3 , 37 - 47.
25. 25)
  - Matalonga, S., Feliu, T.S.: `Using defect data to drive organizational training efforts', Proc. Int. Conf. on Software Engineering Theory and Practice, 2008, p. 61–68.
26. 26)
  - T. Pyzdek . (2003) The six sigma handbook: the complete guide for greenbelts, blackbelts, and managers at all levels.
27. 27)
  - G.G. Wang , D.R. Spitzer . Human resource development measurement and evaluation: looking back and moving forward. Adv. Dev. Human Resour. , 1 , 5 - 15
28. 28)
  - B. Boehm . Value-based software engineering: reinventing. SIGSOFT Softw. Engng. Notes , 2
29. 29)
  - Radice, R.: `Survey results of baselines and models used by level 4 and 5 organizations', Proc. SEPG, 2010.
30. 30)
  - M.B. Chrissis , M. Konrad , S. Shrum . (2007) CMMI: guidelines for process integration and product improvement.
31. 31)
  - Card, D.: `Building a business case for process improvement', Proc. Europe SEPG, 2008.
32. 32)
  - W.A. Shewhart . (1980) Economic control of quality of manufactured product.
33. 33)
  - L.B.S. Raccoon . A learning curve primer for software engineers. ACM SIGSOFT Softw. Engng. Notes , 1
34. 34)
  - G.G. Wang , D. Wilcox . Training evaluation: knowing more than is practiced. Adv. Dev. Human Resour. , 4 , 528 - 539
35. 35)
  - E.A. Ruona , M. Leimbach , F. Holton Iii , R. Bates . The relationship between learner utility reactions and predicted learning transfer among trainees. Int. J. Train. Dev. , 4 , 218 - 228
36. 36)
  - Murugappan, M., Keeni, G.: `Quality improvement – the Six sigma way', Proc. First AsiaPacific Conf. Quality Software, 2000, p. 248–257.
37. 37)
  - Mizukami, D.: `Analyzing defects can tell a LOT about a company', Proc. SEPG Conf., 26–29 March 2007.
38. 38)
  - Matalonga, S., San Feliu, T.: `Defect driven organizational training', Proc. Europe SEPG, 2008.
39. 39)
  - W.S. Humphrey . (1999) Introduction to the team software process.
40. 40)
  - C. Wohlin , P. Runeson , M. Höst , M.C. Ohlsson , B. Regnell , A. Wesslén . (2000) Experimentation in software engineering: an introduction.
41. 41)
  - D. Kirkpatrick . Four-level training evaluation model. US Train. Dev. J. , 9 - 21
42. 42)
  - W.A. Florac , R.E. Park , A.D. Carleton . (1997) Practical software measurement: measuring for process management and improvement.
43. 43)
  - W.E. Deming . (2000) Out of the crisis.
44. 44)
  - R.W. Glover . (1999) Return-on-investment (ROI) analysis of education and training in the construction industry.
45. 45)
  - K. El Emam , I. Garro . Estimating the extent of standards use: the case of ISO/IEC 15504. J. Syst. Softw. , 2 , 137 - 143
46. 46)
  - Matalonga, S., SanFeliu, T.: `Linking return on training investment with defects causal analysis', Proc. 20th Conf. on Software Engineering Theory and Practice, 2008, p. 42–47.
47. 47)
  - (2004) Information technology process assessment. Part 1: concepts and vocabulary.
48. 48)
  - W. Dettmer . (2007) The logical thinking process: a systems approach to complex problem solving.
49. 49)
  - G.M. Alliger , E.A. Janak . Kirckpatrick's levels of training: thirty years later. Pers. Psychol. , 2 , 331 - 342.

Calculating return on investment of training using process variation

Calculating return on investment of training using process variation

Buy article PDF

Buy Knowledge Pack

Thank you

References

Related content