Abstract. The Open-Source IR Reproducibility Challenge brought together developers of open-source search engines to provide reproducible baselines of their systems in a common environment on Amazon EC2. The product is a repository that contains all code necessary to generate competitive ad hoc retrieval baselines, such that with a single script, anyone with a copy of the collection can reproduce the submitted runs. Our vision is that these results would serve as widely accessible points of comparison in future IR research. This project represents an ongoing effort, but we describe the first phase of the challenge that was organized as part of a workshop at SIGIR 2015. We have succeeded modestly so far, achieving our main goals on the Gov2 collection with seven opensource search engines. In this paper, we describe our methodology, share experimental results, and discuss lessons learned as well as next steps.

Background Animal movement expressed through home ranges or space-use can offer insights into spatial and habitat requirements. However, different classes of estimation methods are currently instinctively applied to answer home range, space-use or movement-based research questions regardless of their widely varying outputs, directly impacting conclusions. Recent technological advances in animal tracking (GPS and satellite tags), have enabled new methods to quantify animal space-use and movement pathways, but so far have primarily targeted mammal and avian species. Methods Most reptile spatial ecology studies only make use of two older home range estimation methods: Minimum Convex Polygons (MCP) and Kernel Density Estimators (KDE), particularly with the Least Squares Cross Validation (LSCV) and reference (href) bandwidth selection algorithms. These methods are frequently applied to answer space-use and movement-based questions. Reptile movement patterns are unique (e.g., low movement frequency, long stop-over periods), prompting investigation into whether newer movement-based methods –such as dynamic Brownian Bridge Movement Models (dBBMMs)– apply to Very High Frequency (VHF) radio-telemetry tracking data. We simulated movement data for three archetypical reptile species: a highly mobile active hunter, an ambush predator with long-distance moves and long-term sheltering periods, and an ambush predator with short-distance moves and short-term sheltering periods. We compared traditionally used estimators, MCP and KDE, with dBBMMs, across eight feasible VHF field sampling regimes for reptiles, varying from one data point every four daylight hours, to once per month. Results Although originally designed for GPS tracking studies, dBBMMs outperformed MCPs and KDE href across all tracking regimes in accurately revealing movement pathways, with only KDE LSCV performing comparably at some higher frequency sampling regimes. However, the LSCV algorithm failed to converge with these high-frequency regimes due to high site fidelity, and was unstable across sampling regimes, making its use problematic for species exhibiting long-term sheltering behaviours. We found that dBBMMs minimized the effect of individual variation, maintained low error rates balanced between omission (false negative) and commission (false positive), and performed comparatively well even under low frequency sampling regimes (e.g., once a month). Conclusions We recommend dBBMMs as a valuable alternative to MCP and KDE methods for reptile VHF telemetry data, for research questions associated with space-use and movement behaviours within the study period: they work under contemporary tracking protocols and provide more stable estimates. We demonstrate for the first time that dBBMMs can be applied confidently to low-resolution tracking data, while improving comparisons across regimes, individuals, and species.

“Based on theoretical reasoning it has been suggested that the reliability of findings published in the scientific literature decreases with the popularity of a research field” (Pfeiffer and Hoffmann, 2009). As we know, deep learning is very popular and the ability to reproduce results is an important part of science. There is growing concern within the deep learning community about the reproducibility of results that are presented. In this paper we present a number of controllable, yet unreported, effects that can substantially change the effectiveness of a sample model, and thusly the reproducibility of those results. Through these environmental effects we show that the commonly held belief that distribution of source code is all that is needed for reproducibility is not enough. Source code without a reproducible environment does not mean anything at all. In addition the range of results produced from these effects can be larger than the majority of incremental improvement reported.

No abstract

Background: Studying animal movement provides insights into how animals react to land-use changes. As agriculture expands, we can use animal movement to examine how animals change their behaviour in response. Recent reviews show a tendency for mammalian species to reduce movements in response to increased human landscape modification, but reptile movements have not been as extensively studied. Methods: We examined movements of a large reptilian predator, the King Cobra (Ophiophagus hannah), in Northeast Thailand. We used a consistent regime of radio telemetry tracking to document movements across protected forest and adjacent agricultural areas. Using dynamic Brownian Bridge Movement Model derived motion variance, Integrated Step-Selection Functions, and metrics of site reuse, we examined how King Cobra movements changed in agricultural areas. Results: Motion variance values indicated that King Cobra movements increased in forested areas and tended to decrease in agricultural areas. Our Integrated Step-Selection Functions revealed that when moving in agricultural areas King Cobras restricted their movements to remain within vegetated semi-natural areas, often located along the banks of irrigation canals. Site reuse metrics of residency time and number of revisits appeared unaffected by distance to landscape features (forests, semi-natural areas, settlements, water bodies, and roads). Neither motion variance nor reuse metrics were consistently affected by the presence of threatening landscape features (e.g. roads, human settlements), suggesting that King Cobras will remain in close proximity to threats, provided habitat patches are available. Conclusions: Although King Cobras displayed individual heterogeneity in their response to agricultural landscapes, the overall trend suggested reduced movements when faced with fragmented habitat patches embedded in an otherwise inhospitable land-use matrix. Movement reductions are consistent with findings for mammals and forest specialist species.