The authors present a comprehensive overview of image processing and analysis work done to support research into the model flowering plant Arabidopsis thaliana. Beside the plant's importance in biological research, using image analysis to obtain experimental measurements of it is an interesting vision problem in its own right, involving the segmentation and analysis of sequences of images of objects whose shape varies between individual specimens and also changes over time. While useful measurements can be obtained by segmenting a whole plant from the background, they suggest that the increased range and precision of measurements made available by leaf-level segmentation makes this a problem well worth solving. A variety of approaches have been tried by biologists as well as computer vision researchers. This is an interdisciplinary area and the computer vision community has an important contribution to make. They suggest that there is a need for publicly available datasets with ground truth annotations to enable the evaluation of new approaches and to support the building of training data for modern data-driven computer vision approaches, which are those most likely to result in the kind of fully automated systems that will be of use to biologists.

References

1. 1)
  - 66. Ren, M., Zemel, R.S.: ‘End-to-end instance segmentation and counting with recurrent attention’, 2016. ArXiv preprint ArXiv:1605.09410v1.
2. 2)
  - 41. Qiangqiang, Z., Zhicheng, W., Weidong, Z., et al: ‘Contour-based plant leaf image segmentation using visual saliency’. Image and Graphics, 2015 (LNCS, 9218), pp. 48–59.
3. 3)
  - 52. Jansen, M., Gilmer, F., Biskup, B., et al: ‘Simultaneous phenotyping of leaf growth and chlorophyll fluorescence via GROWSCREEN FLUORO allows detection of stress tolerance in Arabidopsis thaliana and other rosette plants’, Funct. Plant Biol., 2009, 36, (11), pp. 902–914.
4. 4)
  - 33. Lobet, G., Draye, X., Périlleux, C.: ‘An online database for plant image analysis software tools’, Plant Methods, 2013, 9, (1), p. 38+. The database itself is at http://plant-image-analysis.org (accessed 13/4/2016).
5. 5)
  - 64. Vukadinovic, D., Polder, G.: ‘Watershed and supervised classification based fully automated method for separate leaf segmentation’. The Netherlands Conf. on Computer Vision, 2015.
6. 6)
  - 44. Bours, R., Muthuraman, M., Bouwmeester, H., et al: ‘OSCILLATOR: a system for analysis of diurnal leaf growth using infrared photography combined with wavelet transformation’, Plant Methods, 2012, 8, (1), p.29+.
7. 7)
  - 55. Edwards, K.D., Millar, A.J.: ‘Analysis of circadian leaf movement rhythms in Arabidopsis thaliana’, in Rosato, E. (Ed.): ‘Circadian rhythms: methods and protocols’ (Humana Press, Totowa, NJ, 2007), pp. 103–113.
8. 8)
  - 40. Everingham, M., Eslami, S.M.A., Van Gool, L., et al: ‘The pascal visual object classes challenge: a retrospective’, Int. J. Comput. Vis., 2015, 111, (1), pp. 98–136.
9. 9)
  - 27. Walter, A., Scharr, H., Gilmer, F., et al: ‘Dynamics of seedling growth acclimation towards altered light conditions can be quantified via GROWSCREEN: a setup and procedure designed for rapid optical phenotyping of different plant species’, New Phytol., 2007, 174, (2), pp. 447–455.
10. 10)
  - 71. Aksoy, E.E., Abramov, A., Wörgötter, F., et al: ‘Modeling leaf growth of rosette plants using infrared stereo image sequences’, Comput. Electron. Agriculture, 2015, 110, pp. 78–90.
11. 11)
  - 62. Tessmer, O.L., Jiao, Y., Cruz, J.A., et al: ‘Functional approach to high-throughput plant growth analysis’, BMC Syst. Biol., 2013, 7, (Suppl 6), pp. S17–S29.
12. 12)
  - 31. Dhondt, S., Vanhaeren, H., Van Loo, D., et al: ‘Plant structure visualization by high-resolution x-ray computed tomography’, Trends Plant Sci., 2010, 15, (8), pp. 419–422.
13. 13)
  - 4. Leister, D., Varotto, C., Pesaresi, P., et al: ‘Large-scale evaluation of plant growth in Arabidopsis thaliana by non-invasive image analysis’, Plant Physiol. Biochem., 1999, 37, (9), pp. 671–678.
14. 14)
  - 68. Simek, K., Barnard, K.: ‘Gaussian process shape models for Bayesian segmentation of plant leaves’. Proc. of the Computer Vision Problems in Plant Phenotyping (CVPPP), September 2015, pp. 4.1–4.11.
15. 15)
  - 11. Dornbusch, T., Lorrain, S., Kuznetsov, D., et al: ‘Measuring the diurnal pattern of leaf hyponasty and growth in Arabidopsis – a novel phenotyping approach using laser scanning’, Funct. Plant Biol., 2012, 39, (11), pp. 860–869.
16. 16)
  - 50. Nagel, K.A., Putz, A., Gilmer, F., et al: ‘GROWSCREEN-Rhizo is a novel phenotyping robot enabling simultaneous measurements of root and shoot growth for plants grown in soil-filled rhizotrons’, Funct. Plant Biol., 2012, 39, (11), pp. 891–904.
17. 17)
  - 37. Dice, L.R.: ‘Measures of the amount of ecologic association between species’, Ecology, 1945, 26, (3), pp. 297–302.
18. 18)
  - 36. Apelt, F., Breuer, D., Nikoloski, Z., et al: ‘Phytotyping4D: a light-field imaging system for non-invasive and accurate monitoring of spatiotemporal plant growth’, Plant J., 2015, 82, (4), pp. 693–706.
19. 19)
  - 53. Augustin, M., Haxhimusa, Y., Busch, W., et al: ‘A framework for the extraction of quantitative traits from 2D images of mature Arabidopsis thaliana’, Mach. Vis. Appl., 2016, 27, 5, pp. 647–661.
20. 20)
  - 48. Moore, C.R., Johnson, L.S., Kwak, I.-Y., et al: ‘High-throughput computer vision introduces the time axis to a quantitative trait map of a plant growth response’, Genetics, 2013, 195, (3), pp. 1077–1086.
21. 21)
  - 2. Furbank, R.T., Tester, M.: ‘Phenomics – technologies to relieve the phenotyping bottleneck’, Trends Plant Sci., 2011, 16, (12), pp. 635–644.
22. 22)
  - 20. Backhaus, A., Kuwabara, A., Bauch, M., et al: ‘LEAFPROCESSOR: a new leaf phenotyping tool using contour bending energy and shape cluster analysis’, New Phytol., 2010, 187, (1), pp. 251–261.
23. 23)
  - 73. De Vylder, J., Ochoa, D., Philips, W., et al: ‘Leaf segmentation and tracking using probabilistic parametric active contours’. Computer Vision/Computer Graphics Collaboration Techniques, 2011 (LNCS, 6930), pp. 75–85.
24. 24)
  - 13. De Vylder, J., Vandenbussche, F., Hu, Y., et al: ‘Rosette tracker: an open source image analysis tool for automatic quantification of genotype effects’, Plant Physiol., 2012, 160, (3), pp. 1149–1159.
25. 25)
  - 14. Camargo, A., Papadopoulou, D., Spyropoulou, Z., et al: ‘Objective definition of rosette shape variation using a combined computer vision and data mining approach’, PLoS ONE, 2014, 9, (5), p. e96889.
26. 26)
  - 43. Pantin, F., Simonneau, T., Rolland, G., et al: ‘Control of leaf expansion: a developmental switch from metabolics to hydraulics’, Plant Physiol., 2011, 156, (2), pp. 803–815.
27. 27)
  - 24. Kaminuma, E., Heida, N., Tsumoto, Y., et al: ‘Automatic quantification of morphological traits via three-dimensional measurement of Arabidopsis’, Plant J., 2004, 38, (2), pp. 358–365.
28. 28)
  - 15. Boyes, D.C., Zayed, A.M., Ascenzi, R., et al: ‘Growth stage-based phenotypic analysis of Arabidopsis: a model for high throughput functional genomics in plants’, The Plant Cell, 2001, 13, (7), pp. 1499–1510.
29. 29)
  - 74. Dellen, B., Scharr, H., Torras, C.: ‘Growth signatures of rosette plants from time-lapse video’, IEEE/ACM Trans. Comput. Biol. Bioinform., 2015, 12, (6), pp. 1470–1478.
30. 30)
  - 22. Green, J.M., Appel, H., Rehrig, E.M., et al: ‘PhenoPhyte: a flexible affordable method to quantify 2d phenotypes from imagery’, Plant Methods, 2012, 8, (1), p. 45+.
31. 31)
  - 25. Wiese, A., Christ, M.M., Virnich, O., et al: ‘Spatio-temporal leaf growth patterns of Arabidopsis thaliana and evidence for sugar control of the diel leaf growth cycle’, New Phytol., 2007, 174, (4), pp. 752–761.
32. 32)
  - 47. Miller, N.D., Durham Brooks, T.L., Assadi, A.H., et al: ‘Detection of a gravitropism phenotype in glutamate receptor-like 3.3 mutants of Arabidopsis thaliana using machine vision and computation’, Genetics, 2010, 186, (2), pp. 585–593.
33. 33)
  - 34. Minervini, M., Fischbach, A., Scharr, H., et al: ‘Finely-grained annotated datasets for image-based plant phenotyping’, Pattern Recognit. Lett., 2016, 81, 80–89, doi: 10.1016/j.patrec.2015.10.013.
34. 34)
  - 26. Yin, X., Liu, X., Chen, J., et al: ‘Multi-leaf tracking from fluorescence plant videos’. 2014 IEEE Int. Conf. on Image Processing (ICIP), October 2014, pp. 408–412.
35. 35)
  - 72. Rousseau, D., Van de Zedde, H.J.: ‘Counting leaves without ‘finger-counting’ by supervised multiscale frequency analysis of depth images from top view’. Proc. of the Computer Vision Problems in Plant Phenotyping (CVPPP), September 2015, pp. 2.1–2.9.
36. 36)
  - 16. Tsaftaris, S.A., Noutsos, C.: ‘Plant phenotyping with low cost digital cameras and image analytics’. Information Technologies in Environmental Engineering, Environmental Science and Engineering, Berlin, Heidelberg, 2009, pp. 238–251.
37. 37)
  - 1. The Arabidopsis Genome Intiative: ‘Analysis of the genome sequence of the flowering plant Arabidopsis thaliana’, Nature, 2000, 408, (6814), pp. 796–836.
38. 38)
  - 6. Engelmann, W., Simon, K., Phen, C.J.: ‘Leaf movement rhythm in Arabidopsis thaliana’, Z. Naturforchung C, 1992, 47, (11-12), pp. 925–928.
39. 39)
  - 30. Wuyts, N., Palauqui, J.-C.C., Conejero, G., et al: ‘High-contrast three-dimensional imaging of the Arabidopsis leaf enables the analysis of cell dimensions in the epidermis and mesophyll’, Plant Methods, 2010, 6, (1), p. 17+.
40. 40)
  - 5. Spalding, E.P., Miller, N.D.: ‘Image analysis is driving a renaissance in growth measurement’, Curr. Opin. Plant Biol., 2013, 16, (1), 100–104.
41. 41)
  - 56. Giuffrida, M.V., Minervini, M., Tsaftaris, S.: ‘Learning to count leaves in rosette plants’. Proc. of the Computer Vision Problems in Plant Phenotyping (CVPPP), September 2015, pp. 1.1–1.13.
42. 42)
  - 54. Zhang, X., Hause, R.J., Borevitz, J.O.: ‘Natural genetic variation for growth and development revealed by high-throughput phenotyping in Arabidopsis thaliana’, G3: Genes, Genomes, Genetics, 2012, 2, (1), pp. 29–34.
43. 43)
  - 45. Subramanian, R., Spalding, E.P., Ferrier, N.J.: ‘A high throughput robot system for machine vision based plant phenotype studies’, Mach. Vis. Appl., 2013, 24, (3), pp. 619–636.
44. 44)
  - 9. Minervini, M., Scharr, H., Tsaftaris, S.A.: ‘Image analysis: the new bottleneck in plant phenotyping [applications corner]’, IEEE Signal Process. Mag., 2015, 32, (4), pp. 126–131.
45. 45)
  - 70. Janssens, O., De Vylder, J., Aelterman, J., et al: ‘Leaf segmentation and parallel phenotyping for the analysis of gene networks in plants’. Proc. of the 21st European Signal Processing Conf. (EUSIPCO), 2013, 2013, pp. 1–5.
46. 46)
  - 29. Dhondt, S., Van Haerenborgh, D., Van Cauwenbergh, C., et al: ‘Quantitative analysis of venation patterns of Arabidopsis leaves by supervised image analysis’, Plant J., 2012, 69, (3), pp. 553–563.
47. 47)
  - 49. Wang, L., Uilecan, I.V.V., Assadi, A.H., et al: ‘HYPOTrace: image analysis software for measuring hypocotyl growth and shape demonstrated on Arabidopsis seedlings undergoing photomorphogenesis’, Plant Physiol., 2009, 149, (4), pp. 1632–1637.
48. 48)
  - 61. Yanikoglu, B., Aptoula, E., Tirkaz, C.: ‘Automatic plant identification from photographs’, Mach. Vis. Appl., 2014, 25, (6), pp. 1369–1383.
49. 49)
  - 17. Yin, X., Liu, X., Chen, J., et al: ‘Multi-leaf alignment from fluorescence plant images’. 2014 IEEE Winter Conf. on Applications of Computer Vision (WACV), March 2014, pp. 437–444.
50. 50)
  - 51. Klukas, C., Chen, D., Pape, J.-M.: ‘Integrated analysis platform: an open-source information system for high-throughput plant phenotyping’, Plant Physiol., 2014, 165, (2), pp. 506–518.
51. 51)
  - 7. Granier, C., Aguirrezabal, L., Chenu, K., et al: ‘PHENOPSIS, an automated platform for reproducible phenotyping of plant responses to soil water deficit in Arabidopsis thaliana permitted the identification of an accession with low sensitivity to soil water deficit’, New Phytol., 2006, 169, (3), pp. 623–635.
52. 52)
  - 38. Pont-Tuset, J., Marques, F.: ‘Supervised evaluation of image segmentation and object proposal techniques’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 38, (7), p. 1465–1478, doi: 10.1109/TPAMI.2015.2481406.
53. 53)
  - 10. Brown, T.B., Cheng, R., Sirault, X.R.R., et al: ‘Trait-capture: genomic and environment modelling of plant phenomic data’, Curr. Opin. Plant Biol., 2014, 18, pp. 73–79.
54. 54)
  - 8. Humplík, J.F., Lazár, D., Husičková, A., et al: ‘Automated phenotyping of plant shoots using imaging methods for analysis of plant stress responses – a review’, Plant Methods, 2015, 11, (1), pp. 1–10.
55. 55)
  - 60. Cerutti, G., Tougne, L., Mille, J., et al: ‘Understanding leaves in natural images – a model-based approach for tree species identification’, Comput. Vis. Image Underst., 2013, 10, (117), pp. 1482–1501.
56. 56)
  - 69. Wu, X.: ‘Color quantization by dynamic programming and principal analysis’, ACM Trans. Graph., 1992, 11, (4), pp. 348–372.
57. 57)
  - 63. Pape, J.-M., Klukas, C.: ‘3-D histogram-based segmentation and leaf detection for rosette plants’. Computer Vision – ECCV 2014 Workshops, 2015 (LNCS, 8928), pp. 61–74.
58. 58)
  - 3. Dhondt, S., Gonzalez, N., Blomme, J., et al: ‘High resolution time-resolved imaging of in vitro Arabidopsis rosette growth’, Plant J, 2014, 80, (1), pp. 172–184.
59. 59)
  - 67. Romera-Paredes, B., Torr, P.H.S.: ‘Recurrent instance segmentation’. European Conf. on Computer Vision (ECCV) 2016, 2016. ArXiv preprint ArXiv:1511.08250v2.
60. 60)
  - 39. Cruz, J.A., Yin, X., Liu, X., et al: ‘Multi-modality imagery database for plant phenotyping’, Mach. Vis. Appl., 2016, 27, 5 pp. 735–749.
61. 61)
  - 35. Scharr, H., Minervini, M., Fischbach, A., et al: ‘Annotated image datasets of rosette plants’. Technical Report -, Institute of Bio- and Geosciences: Plant Sciences (IBG-2), Forschungszentrum Jülich GmbH, Jülich, Germany, 2014.
62. 62)
  - 57. Scharr, H., Minervini, M., French, A.P., et al: ‘Leaf segmentation in plant phenotyping: a collation study’, Mach. Vis. Appl., 2016, 27, (4), pp. 585–606.
63. 63)
  - 23. Minervini, M., Giuffrida, M.V., Tsaftaris, S.: ‘An interactive tool for semi-automated leaf annotation’. Proc. of the Computer Vision Problems in Plant Phenotyping (CVPPP), September 2015, pp. 6.1–6.13.
64. 64)
  - 32. Lee, K., Avondo, J., Morrison, H., et al: ‘Visualizing plant development and gene expression in three dimensions using optical projection tomography’, Plant Cell, 2006, 18, (9), pp. 2145–2156.
65. 65)
  - 42. Ispiryan, R., Grigoriev, I., zu Castell, W., et al: ‘A segmentation procedure using colour features applied to images of Arabidopsis thaliana’, Funct. Plant Biol., 2013, 40, (10), pp. 1065–1075.
66. 66)
  - 19. Gorbe, E., Calatayud, A.: ‘Applications of chlorophyll fluorescence imaging technique in horticultural research: a review’, Sci. Horticulturae, 2012, 138, pp. 24–35.
67. 67)
  - 21. Arvidsson, S., Pérez-Rodríguez, P., Mueller-Roeber, B.: ‘A growth phenotyping pipeline for Arabidopsis thaliana integrating image analysis and rosette area modeling for robust quantification of genotype effects’, New Phytol., 2011, 191, (3), pp. 895–907.
68. 68)
  - 12. Gifford, R.M., Thorne, J.H., Hitz, W.D., et al: ‘Crop productivity and photoassimilate partitioning’, Science, 1984, 225, (4664), pp. 801–808.
69. 69)
  - 58. Bylesjö, M., Segura, V., Soolanayakanahally, R.Y., et al: ‘LAMINA: a tool for rapid quantification of leaf size and shape parameters’, BMC Plant Biol., 2008, 8, (1), pp. 1–9.
70. 70)
  - 28. Minervini, M., Abdelsamea, M.M., Tsaftaris, S.A.: ‘Image-based plant phenotyping with incremental learning and active contours’, Ecol. Inform., 2014, 23, pp. 35–48.
71. 71)
  - 65. Pape, J.-M., Klukas, C.: ‘Utilizing machine learning approaches to improve the prediction of leaf counts and individual leaf segmentation of rosette plant images’. Proc. of the Computer Vision Problems in Plant Phenotyping (CVPPP), September 2015, pp. 3.1–3.12.
72. 72)
  - 59. Weight, C., Parnham, D., Waites, R.: ‘LeafAnalyser: a computational method for rapid and large-scale analyses of leaf shape variation’, Plant J., 2008, 53, (3), pp. 578–586.
73. 73)
  - 46. Perrin, R.M., Young, L.S., Murthy, U.M.N., et al: ‘Gravity signal transduction in primary roots’, Ann. Botany, 2005, 96, (5), pp. 737–743.
74. 74)
  - 18. Nedbal, L., Whitmarsh, J.: ‘Chlorophyll fluorescence imaging of leaves and plants’, in Papageorgiou, G.C., Govindjee, (Eds.): ‘Chlorophyll a fluorescence: a signature of photosynthesis’ (Kluwer academic publishers, 2004), pp. 389–407.

Watching plants grow – a position paper on computer vision and Arabidopsis thaliana

References

Related content