Evaluation of traceability recovery in context: a taxonomy for information retrieval tools
Evaluation of traceability recovery in context: a taxonomy for information retrieval tools
- Author(s): M. Borg ; P. Runeson ; L. Broden
- DOI: 10.1049/ic.2012.0014
For access to this article, please select a purchase option:
Buy conference paper PDF
Buy Knowledge Pack
IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.
16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012) — Recommend this title to your library
Thank you
Your recommendation has been sent to your librarian.
- Author(s): M. Borg ; P. Runeson ; L. Broden Source: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012), 2012 p. 111 – 120
- Conference: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012)
- DOI: 10.1049/ic.2012.0014
- ISBN: 978-1-84919-541-6
- Location: Ciudad Real, Spain
- Conference date: 14-15 May 2012
- Format: PDF
Background: Development of complex, software intensive systems generates large amounts of information. Several researchers have developed tools implementing information retrieval (IR) approaches to suggest traceability links among artifacts. Aim: We explore the consequences of the fact that a majority of the evaluations of such tools have been focused on benchmarking of mere tool output. Method: To illustrate this issue, we have adapted a framework of general IR evaluations to a context taxonomy specifically for IR-based traceability recovery. Furthermore, we evaluate a previously proposed experimental framework by conducting a study using two publicly available tools on two datasets originating from development of embedded software systems. Results: Our study shows that even though both datasets contain software artifacts from embedded development, the characteristics of the two datasets differ considerably, and consequently the traceability outcomes. Conclusions: To enable replications and secondary studies, we suggest that datasets should be thoroughly characterized in future studies on traceability recovery, especially when they can not be disclosed. Also, while we conclude that the experimental framework provides useful support, we argue that our proposed context taxonomy is a useful complement. Finally, we discuss how empirical evidence of the feasibility of IR-based traceability recovery can be strengthened in future research.
Inspec keywords: system recovery; program diagnostics; information retrieval; embedded systems; software engineering
Subjects: Software engineering techniques; Information retrieval techniques; Diagnostic, testing, debugging and evaluating systems
Related content
content/conferences/10.1049/ic.2012.0014
pub_keyword,iet_inspecKeyword,pub_concept
6
6