We introduce Similarity Group Proposal Network (SGPN), a simple and intuitive deep learning framework for 3D object instance segmentation on point clouds. SGPN uses a single network to predict point grouping proposals and a corresponding semantic class for each proposal, from which we can directly extract instance segmentation results. Important to the effectiveness of SGPN is its novel representation of 3D instance segmentation results in the form of a similarity matrix that indicates the similarity between each pair of points in embedded feature space, thus producing an accurate grouping proposal for each point. Experimental results on various 3D scenes show the effectiveness of our method on 3D instance segmentation, and we also evaluate the capability of SGPN to improve 3D object detection and semantic segmentation results. We also demonstrate its flexibility by seamlessly incorporating 2D CNN features into the framework to boost performance.

Point clouds are an efficient data format for 3D data. However, existing 3D segmentation methods for point clouds either do not model local dependencies [22] or require added computations [15,24]. This work presents a novel 3D segmentation framework, RSNet 1 , to efficiently model local structures in point clouds. The key component of the RSNet is a lightweight local dependency module. It is a combination of a novel slice pooling layer, Recurrent Neural Network (RNN) layers, and a slice unpooling layer. The slice pooling layer is designed to project features of unordered points onto an ordered sequence of feature vectors so that traditional end-to-end learning algorithms (RNNs) can be applied. The performance of RSNet is validated by comprehensive experiments on the S3DIS[1], ScanNet [3], and ShapeNet [35] datasets. In its simplest form, RSNets surpass all previous state-of-the-art methods on these benchmarks. And comparisons against previous state-of-the-art methods [22,24] demonstrate the efficiency of RSNets.

Convolutional neural networks (CNN) are limited by the lack of capability to handle geometric information due to the fixed grid kernel structure. The availability of depth data enables progress in RGB-D semantic segmentation with CNNs. State-of-the-art methods either use depth as additional images or process spatial information in 3D volumes or point clouds. These methods suffer from high computation and memory cost. To address these issues, we present Depth-aware CNN by introducing two intuitive, flexible and effective operations: depth-aware convolution and depth-aware average pooling. By leveraging depth similarity between pixels in the process of information propagation, geometry is seamlessly incorporated into CNN. Without introducing any additional parameters, both operators can be easily integrated into existing CNNs. Extensive experiments and ablation studies on challenging RGB-D semantic segmentation benchmarks validate the effectiveness and flexibility of our approach. IntroductionRecent advances [1,2,3] in CNN have achieved significant success in scene understanding. With the help of range sensors (such as Kinect, LiDAR etc.), depth images are applicable along with RGB images. Taking advantages of the two complementary modalities with CNN is able to improve the performance of scene understanding. However, CNN is limited to model geometric variance due to the fixed grid computation structure. Incorporating the geometric information from depth images into CNN is important yet challenging.Extensive studies [4,5,6,7,8,9,10] have been carried out on this task. FCN [1] and its successors treat depth as another input image and construct two CNNs to process RGB and depth separately. This doubles the number of network parameters and computation cost. In addition, the two-stream network architecture still suffers from the fixed geometric structures of CNN. Even if the geometric relations of two pixels are given, this relation cannot be used in information propagation of CNN. An alternative is to leverage 3D networks [4,11,12] to handle geometry. Nevertheless, both volumetric CNNs [11] and 3D point cloud graph networks [4] are computationally more expensive than 2D CNN. Despite the encouraging results of these progresses, we need to seek a more flexible and efficient way to exploit 3D geometric information in 2D CNN.

Applications in virtual and augmented reality create a demand for rapid creation and easy access to large sets of 3D models. An effective way to address this demand is to edit or deform existing 3D models based on a reference, e.g., a 2D image which is very easy to acquire. Given such a source 3D model and a target which can be a 2D image, 3D model, or a point cloud acquired as a depth scan, we introduce 3DN, an end-to-end network that deforms the source model to resemble the target. Our method infers per-vertex offset displacements while keeping the mesh connectivity of the source model fixed. We present a training strategy which uses a novel differentiable operation, mesh sampling operator, to generalize our method across source and target models with varying mesh densities. Mesh sampling operator can be seamlessly integrated into the network to handle meshes with different topologies. Qualitative and quantitative results show that our method generates higher quality results compared to the state-of-the art learning-based methods for 3D shape generation. Code is available at github.com/laughtervv/3DN .

The facile and economical identification of pathogenic bacteria, especially their antibiotic-resistance, is crucial in the realm of human health and safety. The presence of Escherichia coli (E. coli) is considered as an indicator of water contamination and is closely related to human health. Herein, inspired by the biocatalysis of bacterial surfaces, we developed a simple and cost-effective colorimetric- and electrochemical-based bioassay that is capable of analyzing both the presence of E. coli and its relative level of antibiotic resistance. In this approach, p-benzoquinone is used as a redox mediator to monitor the bacterial concentration and specifically distinguish E. coli from four other common clinical bacteria, namely, Staphylococcus aureus (S. aureus), Enterococcus faecalis (E. faecalis), Salmonella pullorum (S. pullorum), and Streptococcus mutans (S. mutans). A visible color change, captured with a smartphone using a “light box”, without relying on any complex instruments, can reflect the concentration of bacteria. The accurate quantification of E. coli was investigated with an electrochemical system in the concentration ranges of 1.0 × 103 to 1.0 × 109 CFU/mL. We further demonstrated the capability of the presented biosensor in identifying drug-resistant bacteria with two artificially induced antibiotic-resistant bacteria. Therefore, the presented bioassay is not only capable of detecting E. coli with high sensitivity and specificity but also provides a rapid solution to evaluate E. coli antibiotic resistance.