Indices Matter: Learning to Index for Deep Image Matting

Lu, Hao; Dai, Yutong; Shen, Chunhua; Xu, Songcen

doi:10.1109/iccv.2019.00336

Cited by 182 publications

(247 citation statements)

References 43 publications

Supporting

Mentioning

228

Contrasting

Order By: Relevance

“…The idea of counting by regression is further amplified by Lempitsky and Zisserman ( 2010 ) who introduce the concept of the density map. The density map is generated from dotted annotations with Gaussian smoothing such that each pixel is assigned with a value that corresponds to the object density, which transforms counting into a dense prediction problem (Lu et al, 2019 , 2020 ). It has become the basic building block for many object counting models (Chen et al, 2013 ; Arteta et al, 2014 ) including recent deep counting networks (Zhang et al, 2015 , 2016 ; Sindagi and Patel, 2017 ; Li et al, 2018 ; Liu et al, 2020 ; Ma et al, 2019 ; Xiong et al, 2019b ).…”

Section: Introductionmentioning

confidence: 99%

TasselNetV2+: A Fast Implementation for High-Throughput Plant Counting From High-Resolution RGB Imagery

Lü

2020

Front. Plant Sci.

Self Cite

View full text Add to dashboard Cite

Plant counting runs through almost every stage of agricultural production from seed breeding, germination, cultivation, fertilization, pollination to yield estimation, and harvesting. With the prevalence of digital cameras, graphics processing units and deep learning-based computer vision technology, plant counting has gradually shifted from traditional manual observation to vision-based automated solutions. One of popular solutions is a state-of-the-art object detection technique called Faster R-CNN where plant counts can be estimated from the number of bounding boxes detected. It has become a standard configuration for many plant counting systems in plant phenotyping. Faster R-CNN, however, is expensive in computation, particularly when dealing with high-resolution images. Unfortunately high-resolution imagery is frequently used in modern plant phenotyping platforms such as unmanned aerial vehicles, engendering inefficient image analysis. Such inefficiency largely limits the throughput of a phenotyping system. The goal of this work hence is to provide an effective and efficient tool for high-throughput plant counting from high-resolution RGB imagery. In contrast to conventional object detection, we encourage another promising paradigm termed object counting where plant counts are directly regressed from images, without detecting bounding boxes. In this work, by profiling the computational bottleneck, we implement a fast version of a state-of-the-art plant counting model TasselNetV2 with several minor yet effective modifications. We also provide insights why these modifications make sense. This fast version, TasselNetV2+, runs an order of magnitude faster than TasselNetV2, achieving around 30 fps on image resolution of 1980 × 1080, while it still retains the same level of counting accuracy. We validate its effectiveness on three plant counting tasks, including wheat ears counting, maize tassels counting, and sorghum heads counting. To encourage the use of this tool, our implementation has been made available online at https://tinyurl.com/TasselNetV2plus.

show abstract

Section: Introductionmentioning

confidence: 99%

TasselNetV2+: A Fast Implementation for High-Throughput Plant Counting From High-Resolution RGB Imagery

Lü

2020

Front. Plant Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is also the foundation of promoting the end‐to‐end network to be more efficient and light‐weight. In particular, built upon a recent state‐of‐the‐art light‐weight matting network [LDSX19, LDSX20], we investigate three alternative architectures to generate prior information from a segmentation decoder following the shared encoder.…”

Section: Introductionmentioning

confidence: 99%

“…), computational complexity (GFLOPs), Sum of Absolute Differences (SAD), Gradient (Grad) and Connectivity (Conn) errors of different models on the Portrait‐2k test set. DeepLabV3+ [CZP*18] and IndexNet [LDSX19] are currently state‐of‐the‐art segmentation and matting networks, respectively. ‘DeepLabV3+ w. IndexNet’ is the cascaded structure implementing portrait matting without trimap input (prior‐free).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Light‐Weight Portrait Matting via Parameter Sharing

Dai

Shen

2020

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Traditional portrait matting methods typically consist of a trimap estimation network and a matting network. Here, we propose a new light‐weight portrait matting approach, termed parameter‐sharing portrait matting (PSPM). Different from conventional portrait matting models where the encoder and decoder networks in two tasks are often separately designed, here a single encoder is employed for the two tasks in PSPM, while each task still has its task‐specific decoder. Thus, the role of the encoder is to extract semantic features and two decoders function as a bridge between low‐resolution feature maps generated by the encoder and high‐resolution feature maps for pixel‐wise classification/regression. In particular, three variants capable of implementing the parameter‐sharing portrait matting network are proposed and investigated, respectively. As demonstrated in our experiments, model capacity and computation costs can be reduced significantly, by up to 57.8% and 40.5%, respectively, with PSPM, whereas the matting accuracy only slightly deteriorates. In addition, qualitative and quantitative evaluations show that sharing the encoder is an effective way to achieve portrait matting with limited computational budgets, indicating a promising direction for applications of real‐time portrait matting on mobile devices.

show abstract

“…Targeting in-field maize plants, a representative agricultural crop, the goal of this work is to present a comprehensive evaluation of state-of-the-art object detection and object counting methods on the task of maize tassels counting. Object detection is a typical dense prediction problem [29,30]. In recent years, there appear many advanced object detection approaches, such as R-CNN [21], Fast R-CNN [31], Faster R-CNN [22], SSD [32], YOLO9000 [33], RetinaNet [34], etc.…”

mentioning

confidence: 99%

Maize tassels detection: a benchmark of the state of the art

Zou

et al. 2020

Plant Methods

Self Cite

View full text Add to dashboard Cite

Background: The population of plants is a crucial indicator in plant phenotyping and agricultural production, such as growth status monitoring, yield estimation, and grain depot management. To enhance the production efficiency and liberate labor force, many automated counting methods have been proposed, in which computer vision-based approaches show great potentials due to the feasibility of high-throughput processing and low cost. In particular, with the success of deep learning, more and more deeper learning-based approaches are introduced to deal with agriculture automation. Since different detection-and regression-based counting models have distinct characteristics, how to choose an appropriate model given the target task at hand remains unexplored and is important for practitioners. Results: Targeting in-field maize tassels as a representative case study, the goal of this work is to present a comprehensive benchmark of state-of-the-art object detection and object counting methods, including Faster R-CNN, YOLOv3, FaceBoxes, RetinaNet, and the leading counting model of maize tassels-TasselNet. We create a Maize Tassel Detection Counting (MTDC) dataset by supplementing bounding box annotations to the Maize Tassels Counting (MTC) dataset to allow the training of detection models. We investigate key factors effecting the practical applications of the models, such as convergence behavior, scale robustness, speed-accuracy trade-off, as well as parameter sensitivity. Based on our benchmark, we summarise the advantages and limitations of each method and suggest several possible directions to improve current detection-and regression-based counting approaches to benefit nextgeneration intelligent agriculture. Conclusions: Current state-of-the-art detection-and regression-based counting approaches can all achieve a relatively high degree of accuracy when dealing with in-field maize tassels, with at least 0.85 R 2 values and 28.2% rRMSE error. While detection-based methods are more robust than regression-based methods in scale variations and can infer extra information (e.g., object positions and sizes), the latter ones have significantly faster convergence behaviors and inference speed. To choose an appropriate in-filed plant counting method, accuracy, robustness, speed and some other algorithm-specific factors should be taken into account with the same priority. This work sheds light on different aspects of existing detection and counting approaches and provides guidance on how to tackle in-field plant counting. The MTDC dataset is made available at https ://git.io/MTDC

show abstract

Indices Matter: Learning to Index for Deep Image Matting

Cited by 182 publications

References 43 publications

TasselNetV2+: A Fast Implementation for High-Throughput Plant Counting From High-Resolution RGB Imagery

TasselNetV2+: A Fast Implementation for High-Throughput Plant Counting From High-Resolution RGB Imagery

Towards Light‐Weight Portrait Matting via Parameter Sharing

Maize tassels detection: a benchmark of the state of the art

Contact Info

Product

Resources

About