We describe an automated method for detecting clusters of galaxies in imaging and redshift galaxy surveys. The adaptive matched Ðlter (AMF) method utilizes galaxy positions, magnitudes, andÈwhen availableÈphotometric or spectroscopic redshifts to Ðnd clusters and determine their redshift and richness. The AMF can be applied to most types of galaxy surveys, from two-dimensional (2D) imaging surveys, to multiband imaging surveys with photometric redshifts of any accuracy (2.5 dimensional to three-dimensional (3D) redshift surveys. The AMF can also be utilized in the selection of [21 2 D]), clusters in cosmological N-body simulations. The AMF identiÐes clusters by Ðnding the peaks in a cluster likelihood map generated by convolving a galaxy survey with a Ðlter based on a model of the cluster and Ðeld galaxy distributions. In tests on simulated 2D and data with a magnitude limit of 21 2 D r@ B 23.5, clusters are detected with an accuracy of *z B 0.02 in redshift and D10% in richness to z [ 0.5. Detecting clusters at higher redshifts is possible with deeper surveys. In this paper we present the theory behind the AMF and describe test results on synthetic galaxy catalogs.

Fusion of 3D laser radar (LIDAR) imagery and aerial optical imagery is an efficient method for constructing 3D virtual reality models. One difficult aspect of creating such models is registering the optical image with the LIDAR point cloud, which is characterized as a camera pose estimation problem. We propose a novel application of mutual information registration methods, which exploits the statistical dependency in urban scenes of optical apperance with measured LIDAR elevation. We utilize the well known downhill simplex optimization to infer camera pose parameters. We discuss three methods for measuring mutual information between LIDAR imagery and optical imagery. Utilization of OpenGL and graphics hardware in the optimization process yields registration times dramatically lower than previous methods. Using an initial registration comparable to GPS/INS accuracy, we demonstrate the utility of our algorithm with a collection of urban images and present 3D models created with the fused imagery.

This paper presents a new view of federated databases to address the growing need for managing information that spans multiple data models. This trend is fueled by the proliferation of storage engines and query languages based on the observation that "no one size fits all". To address this shift, we propose a polystore architecture; it is designed to unify querying over multiple data models. We consider the challenges and opportunities associated with polystores. Open questions in this space revolve around query optimization and the assignment of objects to storage engines. We introduce our approach to these topics and discuss our prototype in the context of the Intel Science and Technology Center for Big Data.

Interactive massively parallel computations are critical for machine learning and data analysis. These computations are a staple of the MIT Lincoln Laboratory Supercomputing Center (LLSC) and has required the LLSC to develop unique interactive supercomputing capabilities. Scaling interactive machine learning frameworks, such as TensorFlow, and data analysis environments, such as MATLAB/Octave, to tens of thousands of cores presents many technical challenges -in particular, rapidly dispatching many tasks through a scheduler, such as Slurm, and starting many instances of applications with thousands of dependencies. Careful tuning of launches and prepositioning of applications overcome these challenges and allow the launching of thousands of tasks in seconds on a 40,000-core supercomputer. Specifically, this work demonstrates launching 32,000 TensorFlow processes in 4 seconds and launching 262,000 Octave processes in 40 seconds. These capabilities allow researchers to rapidly explore novel machine learning architecture and data analysis algorithms.

No abstract

One of the largest uncertainties in understanding the effect of a background UV field on galaxy formation is the intensity and evolution of the radiation field with redshift. This work attempts to shed light on this issue by computing the quasi-hydrostatic equilibrium states of gas in spherically symmetric dark matter halos (roughly corresponding to dwarf galaxies) as a function of the amplitude of the background UV field. We integrate the full equations of radiative transfer, heating, cooling and non-equilibrium chemistry for nine species: H, H^+, H^-,H_2, H_2^+, He, He^+, He^{++}, and e^-. As the amplitude of the UV background is decreased the gas in the core of the dwarf goes through three stages characterized by the predominance of ionized (H^+), neutral (H) and molecular (H_2) hydrogen. Characterizing the gas state of a dwarf galaxy with the radiation field allows us to estimate its behavior for a variety of models of the background UV flux. Our results indicate that a typical radiation field can easily delay the collapse of gas in halos corresponding to 1-$\sigma$ CDM perturbations with circular velocities less than 30 km/s.Comment: 23 pages (including 8 figures). Figures 3 and 8 best viewed in colo

The GraphBLAS standard (GraphBlas.org) is being developed to bring the potential of matrix-based graph algorithms to the broadest possible audience. Mathematically, the GraphBLAS defines a core set of matrix-based graph operations that can be used to implement a wide class of graph algorithms in a wide range of programming environments. This paper provides an introduction to the mathematics of the GraphBLAS. Graphs represent connections between vertices with edges. Matrices can represent a wide range of graphs using adjacency matrices or incidence matrices. Adjacency matrices are often easier to analyze while incidence matrices are often better for representing data. Fortunately, the two are easily connected by matrix multiplication. A key feature of matrix mathematics is that a very small number of matrix operations can be used to manipulate a very wide range of graphs. This composability of a small number of operations is the foundation of the GraphBLAS. A standard such as the GraphBLAS can only be effective if it has low performance overhead. Performance measurements of prototype GraphBLAS implementations indicate that the overhead is low.

No abstract