<title>Synthetic vision system for improving unmanned aerial vehicle operator situation awareness</title>

Abstract

The Air Force Research Laboratory's Human Effectiveness Directorate (AFRL/HE) supports research addressing human factors associated with Unmanned Aerial Vehicle (UAV) operator control stations. Recent research, in collaboration with Rapid Imaging Software, Inc., has focused on determining the value of combining synthetic vision data with live camera video presented on a UAV control station display. Information is constructed from databases (e.g., terrain, cultural features, pre-mission plan, etc.), as well as numerous information updates via networked communication with other sources (e.g., weather, intel). This information is overlaid conformal, in real time, onto the dynamic camera video image display presented to operators. Synthetic vision overlay technology is expected to improve operator situation awareness by highlighting key spatial information elements of interest directly onto the video image, such as threat locations, expected locations of targets, landmarks, emergency airfields, etc. Also, it may help maintain an operator's situation awareness during periods of video datalink degradation/dropout and when operating in conditions of poor visibility. Additionally, this technology may serve as an intuitive means of distributed communications between geographically separated users. This paper discusses the tailoring of synthetic overlay technology for several UAV applications. Pertinent human factors issues are detailed, as well as the usability, simulation, and flight test evaluations required to determine how best to combine synthetic visual data with live camera video presented on a ground control station display and validate that a synthetic vision system is beneficial for UAV applications. OVERVIEWUnmanned Aerial Vehicles (UAVs) are aircraft without the onboard presence of a pilot or crew. Though the physical separation of the crew from the aircraft offers many promising benefits, it also presents challenges to the effective design of the UAV control station. Numerous human factors issues such as system time delays, poor crew coordination, high workload, and reduced situational awareness may negatively affect mission performance 1 . When onboard an aircraft, a pilot and crew receive a rich supply of multi-sensory information instantaneously regarding their surrounding environment. UAV operators, however, may be limited to a time-delayed, reduced stream of sensory feedback delivered almost exclusively through the visual channel.Of all the information displays within military UAV control stations, the video imagery from various cameras mounted on the UAV is particularly valuable. UAV pilots use imagery from the nose and gimbal cameras to verify clear path for taxi/runway operations, scan for other air traffic in the area, and identify navigational landmarks and potential obstructions. Additionally, sensor operators use imagery from a gimbal-mounted camera to conduct a wide variety of intelligence, surveillance and reconnaissance activities as well as to directly support combat operations. However, video...

Using a Mini-UAV to Support Wilderness Search and Rescue: Practices for Human-Robot Teaming

“…It is sometimes useful to include a third person to monitor the behavior of the pilot and sensor operator; this third person helps protect the pilot and sensor operator and helps provide greater situation awareness [2,6]. Complementing such search-specific work is general work on the human factors of UAVs [3,5,12] and on the number of people required to manage an unmanned vehicle [10,15].…”

Comparing the usefulness of video and map information in navigation tasks

“…Although the ways to combine maps and visualization tools have been studied in other domains such as aviation (see, for example [4,16]) this problem has not been well studied in human-robot operation with occupancy grid maps.…”

Supporting wilderness search and rescue using a camera‐equipped mini UAV

“…a probabilistic description of the person's likely location ͑Hansen, McLain & Goodrich, 2007͒. Additional literature includes operator interface work for both UAVs and traditional aviation displays ͑Calhoun, Draper, Abernathy, Patzek & Delgado, 2003;Prinzel, Glaab, Kramer & Arthur, 2004;Alexander & Wickens, 2005;Drury, Richer, Rackliffe & Goodrich, 2006;Wickens, Olmos, Chud & Davenport, 1995͒. The exact type of interaction between a human and onboard autonomy varies widely across UAV platforms.…”