Searching for people using semantic soft biometric descriptions

Denman, Simon; Halstead, Michael; Fookes, Clinton; Sridharan, Sridha

doi:10.1016/j.patrec.2015.06.015

Cited by 32 publications

(47 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…red, blue, black, etc.) serving as categorical labels [4], [10][11][12][13][14][15]. We rather use computer vision features to categorize clothing colors according to a more generic and higher level annotation describing the Color-scheme as 'Warm' like orange, and 'Neutral' like gray, so it is less error-prone in case of illumination variation.…”

Section: A Context and Contributionsmentioning

confidence: 99%

“…For identification/re-identification proposes, our approach utilizes more fine-grained soft clothing attributes rather than coarse-grained attributes as in [13]. Besides, we directly use either an image or a verbal description as a query to retrieve a subject-of-interest rather than transforming semantic attributes into a searchable avatar to retrieve any matching representations as in [11]. In this research, two different groups of soft clothing biometrics are employed for the purpose of person identification and retrieval.…”

Section: A Context and Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

From Clothing to Identity: Manual and Automatic Soft Biometrics

Jaha

Nixon

2016

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

Abstract-Soft biometrics have increasingly attracted research interest and are often considered as major cues for identity, especially in the absence of valid traditional biometrics, as in surveillance. In everyday life, several incidents and forensic scenarios highlight the usefulness and capability of identity information that can be deduced from clothing. Semantic clothing attributes have recently been introduced as a new form of soft biometrics. Although clothing traits can be naturally described and compared by humans for operable and successful use, it is desirable to exploit computer-vision to enrich clothing descriptions with more objective and discriminative information. This allows automatic extraction and semantic description and comparison of visually detectable clothing traits in a manner similar to recognition by eyewitness statements. This study proposes a novel set of soft clothing attributes, described using small groups of high-level semantic labels, and automatically extracted using computer-vision techniques. In this way we can explore the capability of human attributes vis-a-vis those which are inferred automatically by computer-vision. Categorical and comparative soft clothing traits are derived and used for identification/re-identification either to supplement soft body traits or to be used alone. The automatically-and manuallyderived soft clothing biometrics are employed in challenging invariant person retrieval. The experimental results highlight promising potential for use in various applications.

show abstract

Section: A Context and Contributionsmentioning

confidence: 99%

Section: A Context and Contributionsmentioning

confidence: 99%

From Clothing to Identity: Manual and Automatic Soft Biometrics

Jaha

Nixon

2016

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

show abstract

“…Combining several soft biometrics modalities, especially clothing, has proven important in improving subject recognition rates [3,14] and can be estimated for surveillance tracking and search [9,44].…”

Section: Pedestrian Re-identificationmentioning

confidence: 99%

Analysing comparative soft biometrics from crowdsourced annotations

2016

View full text Add to dashboard Cite

Soft biometrics enable human description and identification from low quality surveillance footage. This paper premises the design, collection and analysis of a novel crowdsourced dataset of comparative soft biometric body annotations, obtained from a richly diverse set of human annotators. We annotate 100 subject images to provide a coherent, in-depth appraisal of the collected annotations and inferred relative labels. The dataset includes gender as a comparative trait and we find that comparative labels characteristically contain additional discriminative information over traditional categorical annotations. Using our pragmatic dataset, we perform semantic recognition by inferring relative biometric signatures using a RankSVM algorithm. This demonstrates a practical scenario, reproducing responses from a video surveillance operator searching for an individual. The approach can reliably return the correct match in the the top 7% of results with 10 comparisons, or top 13% of results using just 5 sets of subject comparisons.

show abstract

“…As closed-circuit television (CCTV) coverage becomes more prevalent, research into its effective use has increased to include computer vision techniques such as event detection [54], person re-detection [2] and more recently person search [39]. Nevertheless these common security and surveillance tasks are still primarily undertaken by operators and staff on the ground, which often prove labour intensive, time consuming, and ineffective.…”

Section: Motivationmentioning

confidence: 99%

“…Soft biometrics have been utilised in a diverse range of applications: including fashion parsing [153], content based image retrieval [73,92], surveillance [39], shopping applications [96], simplistic replication of human verbal descriptions of images in [79], and recently as an image labeling tool for surveillance with a deep neural network (DNN) [118].…”

Section: Figure 22mentioning

confidence: 99%

Locating People in Video Surveillance from Semantic Descriptions

Halstead¹

View full text Add to dashboard Cite

AcknowledgmentsThe completion of my PhD research programme has been an enjoyable and at times difficult and complex journey that has involved a number of people that have assisted me in so many ways throughout the process. The people that I thank here is by no means a comprehensive list as there isn't enough paper for the names I have to mention.I would firstly like to thank my supervisory team, Clinton Fookes, Sridha Sridharan, and Simon Denman for providing valuable guidance throughout the course of my study. I would like to thank Clinton and Sridha for providing me with the opportunity to study my PhD, and also for providing the Vision and Signal Processing (VSP) laboratory environment that allowed me to research at a high level, and interact with other PhD students to assist with my PhD. I would like to make special mention of Simon for his day to day support and being consistently available to help me with my endless questions, and without his help, guidance, and encouragement I would not have made it to the end of my PhD. Another member of the of the VSP discipline that I would like to thank is David Dean, as a member of VSP he assisted me multiple times throughout the course of my PhD.I would also like to thank the other students and past and present staff within the VSP laboratory, thank you for the sharing of knowledge but mostly thank you all for the entertainment, and enjoyable moments that we shared throughout this time. AbstractIn surveillance and security situations it is often necessary to search and locate a subject of interest from a verbally or textually supplied subject description. At present, searches using these forms of queries are still predominantly performed manually and often ineffectively, via physical searching of a premises or personally watching countless hours of video footage, looking for the specific subject of interest. Surveillance networks are typically monitored by few people viewing several monitors displaying single and multiple camera feeds. In these situations, the ability to translate an externally supplied description into one of computer vision relevance would be of great assistance.To date, state-of-the-art soft biometric based search techniques primarily include some form of pre-enrollment component, where the target subject is enrolled from one camera and searched in other cameras for a match, or alternatively enrolled in an earlier instance of the same camera, and searched for in later footage. These re-identification techniques exclude however, instances where the subject has not previously been located and enrolled from the footage.Soft biometric based semantic search is one technique that provides a solution to this problem. The target query no longer requires scene interrogation and instead can be generated from a textually supplied query. While soft biometrics have obvious limitations, non-permanence and not individually identifiable, they are generally simplistic for a human operator to visually extract and relay to another human operator. They also have th...

show abstract

Searching for people using semantic soft biometric descriptions

Cited by 32 publications

References 2 publications

From Clothing to Identity: Manual and Automatic Soft Biometrics

From Clothing to Identity: Manual and Automatic Soft Biometrics

Analysing comparative soft biometrics from crowdsourced annotations

Locating People in Video Surveillance from Semantic Descriptions

Contact Info

Product

Resources

About