Collaborative Video Object Segmentation by Foreground-Background Integration

Yang, Zongxin; Wei, Yunchao; Yang, Yi

doi:10.1007/978-3-030-58558-7_20

Cited by 167 publications

(195 citation statements)

References 37 publications

Supporting

Mentioning

192

Contrasting

Order By: Relevance

“…Since the propagation is conducted in a short-time interval, the methods often exploit the temporal smoothness constraint but are not robust to occlusion. The matching-based methods [41,13,52,43,14,50] predict a foreground mask in the current frame based on matching with previously predicted or given mask. Recently, STM [33] introduced a memorybased method for offline-learning VOS and demonstrated a significantly improved performance while achieving a fast run-time.…”

Section: Related Workmentioning

confidence: 99%

“…Temporal smoothness is one of the strong constraints that we can assume for the VOS task. Previous VOS methods without memory often applied a local matching [43,50] or local refinement [34,16,32,49,13,53] between two adjacent frames for temporal smoothness. However, in the memory-based method [33], the non-local matching completely ignores the constraint and it raises the risk of false matches (e.g., when multiple similar instances exist, see Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Kernelized Memory Network for Video Object Segmentation

Seong

Hyun

Kim

2020

Computer Vision – ECCV 2020

136

165

View full text Add to dashboard Cite

We present Hierarchical Memory Matching Network (HMMN) for semi-supervised video object segmentation. Based on a recent memory-based method [33], we propose two advanced memory read modules that enable us to perform memory reading in multiple scales while exploiting temporal smoothness. We first propose a kernel guided memory matching module that replaces the nonlocal dense memory read, commonly adopted in previous memory-based methods. The module imposes the temporal smoothness constraint in the memory read, leading to accurate memory retrieval. More importantly, we introduce a hierarchical memory matching scheme and propose a top-k guided memory matching module in which memory read on a fine-scale is guided by that on a coarsescale. With the module, we perform memory read in multiple scales efficiently and leverage both high-level semantic and low-level fine-grained memory features to predict detailed object masks. Our network achieves state-of-theart performance on the validation sets of

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Kernelized Memory Network for Video Object Segmentation

Seong

Hyun

Kim

2020

Computer Vision – ECCV 2020

136

165

View full text Add to dashboard Cite

show abstract

“…Finally, we can place papers D and E in context to the stateoftheart, figure 6.1 shows the improvement over time, of mean J and F score improvement. The methods were evaluated on the DAVIS 2017 validation split, and includes both the methods mentioned before, i.e OSVOS [6], RGMP [26], STM [27] and LWL [2], as well as the more recent method CFBI [41]. Both STM and CFBI were considered stateoftheart at the time of writing.…”

Section: Performance Evaluationmentioning

confidence: 99%

Discriminative correlation filters in robot vision

Robinson

2021

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

This was not the plan. But here I am, writing the last paragraphs of my disser tation. These years have been a journey, not just for the places I had a chance to visit but for both the challenges, rewards and personal growth.First and foremost, I am grateful to my supervisor Michael Felsberg for the trust he placed in me, and for being so patient and positive throughout this process, especially when things did not go according to plan. I would also like to thank my cosupervisors, PerErik Forssén for many interesting discussions and Fahad Khan for introducing me to the art of publishing sci entific papers.Science is not a solitary pursuit. I am very grateful for all the collabora tions within the lab, that actually made this thing possible in the first place: To Mikael Persson, both for our working together in various endeavors, both at CVL and outside, and helping out during stressful times. To Martin Danell jan with whom I shared an office, both for the collaborations on two papers, the discussions we had, and him taking the time to explain math much bet ter than any textbook. To Felix Järemo Lawin, for all the discussions and the sometimes stressful but ultimately fun work together. To Bertil Grelsson, for our work on a rather unusal and interesting project. To Abdelrahman Eldes okey, for diving into a new topic and joining me in that mad dash to get the last paper finished.I would like to thank everyone at CVL, past and present, both labmates and seniors, for the great and friendly work environment, interesting discussions and board games. Also many thanks to Mikael, Felix, Abdo, Gustav Häger and Emil Brissman, for proofreading and providing feedback on the manuscript.Finally, thank you to my family; to my brother for saying I could do this, and to mom and dad for their endless support, and just being there.

show abstract

“…Target classification in the test frame is then performed by applying the correlation filters. The feature matching approach in [52,112] was further extended in [121], by including background features and target attention modulation vectors into the target model. Finally, the STM approach [86] performs feature matching within a space-time memory network.…”

Section: Background and Previous Workmentioning

confidence: 99%

“…Intuitively, the parameters θ and ϕ are learned for target agnostic segmentation and generic VOS properties, while τ encodes the target-specific appearance. If we compare this framework to the feature matching approaches in [121,52,112,86], the target state τ corresponds to the feature embedding generated from the previous frames and masks. Contrary to these approaches, our target model T τ is composed by a parametric operator, designed to predict a target score representation s = T τ (x) given the image features x = F (I) from the backbone network.…”

Section: Vos With Optimization-based Target Modelsmentioning

confidence: 99%

Learning Representations for Segmentation and Registration

Lawin

2021

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

I could not have made it through my Ph.D. studies without the help of my colleagues at CVL. First and foremost I would like to thank my supervisor Per-Erik Forssén for his friendliness and all the valuable guidance. Throughout, he has always been encouraging and open to discuss ideas. I have also enjoyed our supervision meetings, and often certain information has come to light during our discussions. Next, I want to thank my co-supervisor Michael Felsberg who first took me in as a Ph.D. student, or should I say, tricked me into it. Either way, I am glad that he gave me the chance at perusing a Ph.D. degree. I am very thankful for his support and for learning from his knowledge and experience. To both my supervisors I want to express my gratitude for giving me the freedom and the encouragement to explore different topics and ideas. I also want to thank all my collaborators and co-authors for sharing their brilliant ideas and for our interesting discussions. Martin Danelljan for his notation, for inviting me into exciting research projects and for pushing me into taking time-consuming math courses. Fahad Shahbaz Khan for his encouraging attitude and spreading of positive energy during the darkest deadline hours. Andreas Robinson for sharing his programming skills and for patiently helping me with software installations and computer issues. Goutam Bhat for his awesome figures, ability to improve the results and our discussions on football. I am very proud of what we achieved in our joint works. Moreover, I want to thank all of CVL, including past and current members, for the friendly atmosphere at the lab and all the interesting discussions during coffee breaks. I further want to thank Zahra for arranging barbecues and Mikael for hosting board gaming sessions. Hannes, Abdo and Emanuel for their nice company while sharing offices with me. I am also grateful to everyone who joined me for late lunch, even if the only acceptable option was the bad house. Special thanks go to Per-Erik, Gustav, Abdo, and Andreas for helping out with the proofreading of the thesis manuscript. Lastly, I want to thank my family and all my friends for all the support and good times.

show abstract

Collaborative Video Object Segmentation by Foreground-Background Integration

Cited by 167 publications

References 37 publications

Kernelized Memory Network for Video Object Segmentation

Kernelized Memory Network for Video Object Segmentation

Discriminative correlation filters in robot vision

Learning Representations for Segmentation and Registration

Contact Info

Product

Resources

About