Pixelwise instance segmentation of leaves in dense foliage - Institut Agro Dijon Access content directly
Journal Articles Computers and Electronics in Agriculture Year : 2022

Pixelwise instance segmentation of leaves in dense foliage

Jehan-Antoine Vayssade
Gawain Jones
Christelle Gée
Jean-Noël Paoli


Detecting and identifying plants using image analysis is a key step for many applications in precision agriculture (from phenotyping to site specific weed management). Instance segmentation is usually carried on to detect entire plants. However, the shape of the detected objects changes between individuals and growth stages. A relevant approach to reduce these variations is to narrow the detection on the leaf. Nevertheless, segmenting leaves is a difficult task, when images contain mixes of plant species, and when individuals overlap, particularly in an uncontrolled outdoor environment. To leverage this issue, this study based on recent Convolutional Neural Network mechanisms, proposes a pixelwise instance segmentation to detect leaves in dense foliage environment. It combines “deep contour aware” (to separate the inner of big leaves from its edges), “Leaf Segmentation trough classification of edges” (to separate instances with a specific inner edges) and “Pyramid CNN for Dense Leaves” (to consider edges at different scales). But the segmentation output is also refined using a Watershed and a method to compute optimized vegetation indices (DeepIndices). The method is compared to others running the leaf segmentation challenge (provided by the International Network on Plant Phenotyping) and applied on an external dataset of Komatsuna plants. In addition, a new multispectral dataset of 300 images of bean plants is introduced (with dense foliage, individuals overlapping, mixes of species and natural lighting conditions). The ground truth (e.g. the leaves boundaries) is defined by labelled polygons and can be used to train and assess the performance of various algorithms dedicated to leaf detection or crop/weed classification. On the usual datasets, the performances of the proposed method are similar to those of the usual methods involved in the leaf segmentation challenges. On the new dataset, their results are strongly better than those of the usual RCNN method. Remaining errors are bad fusion between neighboring areas and over segmentation of multi-foliate leaves. Structural analysis methods could be studied in order to overcome these deficiencies.
No file

Dates and versions

hal-03641427 , version 1 (14-04-2022)



Jehan-Antoine Vayssade, Gawain Jones, Christelle Gée, Jean-Noël Paoli. Pixelwise instance segmentation of leaves in dense foliage. Computers and Electronics in Agriculture, 2022, 195, pp.106797. ⟨10.1016/j.compag.2022.106797⟩. ⟨hal-03641427⟩
67 View
0 Download



Gmail Facebook Twitter LinkedIn More