Neuroimaging studies of biological motion perception have found a network of coordinated brain areas, the hub of which appears to be the human posterior superior temporal sulcus (STSp). Understanding the functional role of the STSp requires characterizing the response tuning of neuronal populations underlying the BOLD response. Thus far our understanding of these response properties comes from single-unit studies of the monkey anterior STS, which has individual neurons tuned to body actions, with a small population invariant to changes in viewpoint, position and size of the action being viewed. To measure for homologous functional properties on the human STS, we used fMR-adaptation to investigate action, position and size invariance. Observers viewed pairs of point-light animations depicting human actions that were either identical, differed in the action depicted, locally scrambled, or differed in the viewing perspective, the position or the size. While extrastriate hMT+ had neural signals indicative of viewpoint specificity, the human STS adapted for all of these changes, as compared to viewing two different actions. Similar findings were observed in more posterior brain areas also implicated in action recognition. Our findings are evidence for viewpoint invariance in the human STS and related brain areas, with the implication that actions are abstracted into object-centered representations during visual analysis.

Data are often viewed as a single set of values, but those values frequently must be compared with another set. The existing evaluations of designs that facilitate these comparisons tend to be based on intuitive reasoning, rather than quantifiable measures. We build on this work with a series of crowdsourced experiments that use low-level perceptual comparison tasks that arise frequently in comparisons within data visualizations (e.g., which value changes the most between the two sets of data?). Participants completed these tasks across a variety of layouts: overlaid, two arrangements of juxtaposed small multiples, mirror-symmetric small multiples, and animated transitions. A staircase procedure sought the difficulty level (e.g., value change delta) that led to equivalent accuracy for each layout. Confirming prior intuition, we observe high levels of performance for overlaid versus standard small multiples. However, we also find performance improvements for both mirror symmetric small multiples and animated transitions. While some results are incongruent with common wisdom in data visualization, they align with previous work in perceptual psychology, and thus have potentially strong implications for visual comparison designs.

Perceptual tasks in visualizations often involve comparisons. Of two sets of values depicted in two charts, which set had values that were the highest overall? Which had the widest range? Prior empirical work found that the performance on different visual comparison tasks (e.g., “biggest delta”, “biggest correlation”) varied widely across different combinations of marks and spatial arrangements. In this paper, we expand upon these combinations in an empirical evaluation of two new comparison tasks: the“biggest mean” and “biggest range” between two sets of values. We used a staircase procedure to titrate the difficulty of the data comparison to assess which arrangements produced the most precise comparisons for each task. We find visual comparisons of biggest mean and biggest range are supported by some chart arrangements more than others, and that this pattern is substantially different from the pattern for other tasks. To synthesize these dissonant findings, we argue that we must understand which features of a visualization are actually used by the human visual system to solve a given task. We call these perceptual proxies. For example, when comparing the means of two bar charts, the visual system might use a “Mean length” proxy that isolates the actual lengths of the bars and then constructs a true average across these lengths. Alternatively, it might use a “Hull Area” proxy that perceives an implied hull bounded by the bars of each chart and then compares the areas of these hulls. We propose a series of potential proxies across different tasks, marks, and spatial arrangements. Simple models of these proxies can be empirically evaluated for their explanatory power by matching their performance to human performance across these marks, arrangements, and tasks. We use this process to highlight candidates for perceptual proxies that might scale more broadly to explain performance in visual comparison.

Representing temporally continuous objects across change (e.g., in position) requires integration of newly sampled visual information with existing object representations. We asked what consequences representational updating has for visual search. In this dynamic visual search task, bars rotated around their central axis. Observers searched for a single episodic target state (oblique bar among vertical and horizontal bars). Search was efficient when the target display was presented as an isolated static display. Performance declined to near chance, however, when the same display was a single state of a dynamically changing scene (Experiment 1), as though temporal selection of the target display from the stream of stimulation failed entirely (Experiment 3). The deficit is due neither to masking (Experiment 2), nor to a lack of temporal marker for the target display (Experiment 4). The deficit was partially reduced by visually marking the target display with unique feature information (Experiment 5). We suggest that representational updating causes a loss of access to instantaneous state information in search. Similar to spatially crowded displays that are perceived as textures (Parkes, Lund, Angelucci, & Solomon, 2001), we propose a temporal version of the trees (instantaneous orientation information) being lost for the forest (rotating bars).

Analytic task: Compare means Analytic task: Compare ranges = = > Mean Length Extract ensembles of length, choose larger ensemble. > Hull Area extract ensembles of length choose larger ensemble Range compare all pairwise deltas, choose largest. Neighbor Deltas compare only neighbor deltas, choose largest. (Erroneous) Analytic goals and visualization guide selection of perceptual heuristic. Designers choose marks and arrangements to create visualizations. Visualizations contain numerous visual features. What is visually compared?Fig. 1: What's visual in visual comparisons, such as finding the larger mean value? We identify mark arrangements that allow for better performance across comparison tasks. Combining previous results with the results of two new tasks fails to produce a clean ranking of arrangement effectiveness across tasks. We argue that to explain these complex patterns of performance, we first need a perceptual explanation of how visual comparison actually unfolds. Viewers likely perform these mathematical comparison operations with perceptual proxies. We propose and evaluate a candidate set of proxies for two visual comparison tasks.

Previous work has demonstrated that the ability to keep track of moving objects is improved when the objects have unique visual features, such as color or shape. In the present study, we investigated how orientation information is used during the tracking of objects. Orientation is an interesting feature to explore in moving objects because it is directional and is often informative of the direction of motion. Most objects move forward, in the direction they are oriented. In the present experiments, participants tracked a subset of moving isosceles triangles whose orientations were constant, related, or unrelated to the direction of motion. In the standard multiple object tracking (MOT) task, tracking performance improved when orientations were unique and remained constant, but not when orientation and direction of motion were aligned. In the target recovery task, in which MOT was interrupted by a brief blanking of the display, performance did improve when orientation and direction were aligned. In the final experiment, results showed that orientation was not used before the blank to predict future target locations, but was instead used after the blank. We concluded that people use orientation to compare a stored representation to target position for recovery of lost targets.

Given a set of simple objects, visual working memory capacity drops from 3 to 4 units down to only 1 to 2 units when the display rotates. But real-world STEM experts somehow overcome these limits. Here, we study a potential domain-general mechanism that might help experts exceed these limits: compressing information based on redundant visual features. Participants briefly saw 4 colored shapes, either all distinct or with repetitions of color, shape, or paired Color + Shape (e.g., two green squares among a blue triangle and a yellow diamond), with a concurrent verbal suppression task. Participants reported potential swaps (change/no change) in a rotated view. In Experiments 1a through 1c, repeating features improved performance for color, shape, and paired Color + Shape. Critically, Experiments 2a and 2b found that the benefits of repetitions were most pronounced when the repeated objects shared both feature dimensions (i.e., two green squares). When color and shape repetitions were split across different objects (e.g., green square, green triangle, red triangle), the benefit was reduced to the level of a single redundant feature, suggesting that feature-based grouping underlies the redundancy benefit. Visual compression is an effective encoding strategy that can spatially tag features that repeat.

No abstract