Learning an event classifier is challenging when the scenes are semantically different but visually similar. However, as humans, we typically handle such tasks painlessly by adding our background semantic knowledge. Motivated by this observation, we aim to provide an empirical study about how additional information such as semantic keywords can boost up the discrimination of such events. To demonstrate the validity of this study, we first construct a novel Malicious Crowd Dataset containing crowd images with two events, benign and malicious, which look visually similar. Note that the primary focus of this paper is not to provide the state-ofthe-art performance on this dataset but to show the beneficial aspects of using semantically-driven keyword information. By leveraging crowd-sourcing platforms, such as Amazon Mechanical Turk, we collect semantic keywords associated with images and then subsequently identify a subset of keywords (e.g. police, fire, etc.) unique to specific events. We first show that by using recently introduced attention models, a naïve CNN-based event classifier actually learns to primarily focus on local attributes associated with the discriminant semantic keywords identified by the Turks. We further show that incorporating the keyword-driven information into earlyand late-fusion approaches can significantly enhance malicious event classification.
The US Army Research Laboratory (ARL) and the Air Force Research Laboratory (AFRL) have established a collaborative research enterprise referred to as the Situational Understanding Research Institute (SURI). The goal is to develop an information processing framework to help the military obtain real-time situational awareness of physical events by harnessing the combined power of multiple sensing sources to obtain insights about events and their evolution. It is envisioned that one could use such information to predict behaviors of groups, be they local transient groups (e.g., protests) or widespread, networked groups, and thus enable proactive prevention of nefarious activities. This paper presents a vision of how social media sources can be exploited in the above context to obtain insights about events, groups, and their evolution.
Sensors and Electron Devices Directorate, ARLApproved for public release; distribution unlimited. Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.
ii
REPORT DOCUMENTATION PAGE
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.