A Deep Learning Approach for 3D Shape Understanding
3D shape understanding is fundamental for many visual computing tasks such as robotics, autonomous driving, augmented and virtual reality, etc. Deep learning has become a very powerful tool for many visual recognition tasks in 2D images. Its adaptation to 3D tasks such as 3D shape understanding however remains challenging. In this talk I will discuss our recent effort in applying deep learning for 3D shape understanding. First, I will introduce our recently proposed deep learning architecture-PointGrid for 3D visual recognition tasks such as 3D recognition and semantic segmentation. PointGrid is a hybrid representation of unorganized point clouds and volumetric grid that can better capture local geometric details while exhibits easy-to-learn regular structure. Experiments on widely used benchmark datasets show that PointGrid compares favorably over state-of-the-arts methods on both classification and segmentation with significantly smaller memory footprint. I will then describe our recent work of a multi-view recurrent neural network (MV-RNN) approach for 3D mesh segmentation. Our architecture combines the convolutional neural networks (CNN) and a two-layer long short-term memory (LSTM) to yield coherent segmentation of 3D shapes. The imaged-based CNN are useful for effectively generating the edge probability feature map while the LSTM correlates these edge maps across different views and output a well-defined per-view edge image. Evaluations on the Princeton Segmentation Benchmark dataset show that MV-RNN significantly outperforms other state-of-the-art 3D mesh segmentation methods.
Ye Duan received his B.S. in Mathematics from Peking University, and his M.S. and Ph.D. in Computer Science from the State University of New York at Stony Brook. He is currently an Associate Professor of Computer Science at University of Missouri at Columbia, where he is currently the Director of the Computer Graphics and Image Understanding Lab, the Director of the Urban Safety Center, and the Director of the Cognitive Internet of Things for Intelligent Community-Industry Sponsored Consortium. He was the Conference Chair of the IEEE International Conference on Shape Modeling (SMI) 2018, and is currently the Conference Chair of the IEEE International Conference on Shape Modeling (SMI) 2019. His research interests include Computer Graphics, Computer Vision, Machine Learning, Virtual Reality, as well as Biomedical Imaging. More information can be found at: engineers.missouri.edu/duanye/
Friday, February 15 at 2:30pm to 3:30pm
McAdams Hall, 114
821 McMillan Rd., Clemson, SC 29634, USA