Topology-aware non-rigid point cloud registration
In this project, we develop a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different. Standard warp field estimation algorithms, even under robust, discontinuity-preserving regularization, tend to produce erratic motion estimates on boundaries associated with 'close-to-open' topology changes. We overcome this limitation by exploiting backward motion: in the opposite motion direction, a 'close-to-open' event becomes 'open-to-close', which is by default handled correctly. At the core of our approach lies a general, topology-agnostic warp field estimation algorithm, similar to those employed in recently introduced dynamic reconstruction systems from RGB-D input. We improve motion estimation on boundaries associated with topology changes in an efficient post-processing phase. Based on both forward and (inverted) backward warp hypotheses, we explicitly detect regions of the deformed geometry that undergo topological changes by means of local deformation criteria and broadly classify them as 'contacts' or 'separations'. Subsequently, the two motion hypotheses are seamlessly blended on a local basis, according to the type and proximity of detected events. Our method achieves state-of-the-art motion estimation accuracy on the MPI Sintel dataset. Experiments on a custom dataset with topological event annotations demonstrate the effectiveness of our pipeline in estimating motion on event boundaries, as well as promising performance in explicit topological event detection.
References:
Topology-Aware Non-Rigid Point Cloud Registration
[pdf]
[bib]
Konstantinos Zampogiannis, Cornelia Fermüller, Yiannis Aloimonos,
PAMI 2019
We develop a pipeline for fitting approximate models to partial 3D scans of box-shaped container and appliance objects. Our approach relies on a general cuboid detection and fitting algorithm that, guided by a normal clustering preprocessing step, efficiently segments a scene into planar surface patches and appropriately refines and groups the latter into box candidates. By combining box detections from partial reconstructions of the object exterior and interior (obtained in a common coordinate frame) with prior part knowledge (e.g., existence and/or relative location of door and/or door handle), we automatically obtain a decomposition of the object into box-shaped parts. Based on the detected interior box, we obtain a rectified render of the object interior which we use to reconstruct the container's shelf structure (if any) by means of a hybrid 2D/3D approach. Our final output is an approximate object model built with box shape primitives.
In this project, we introduce an abstract representation for manipulation actions that is based on the evolution of the spatial relations between involved objects. Object tracking in RGBD streams enables straightforward and intuitive ways to model spatial relations in 3D space. Reasoning in 3D overcomes many of the limitations of similar previous approaches, while providing significant flexibility in the desired level of abstraction. At each frame of a manipulation video, we evaluate a number of spatial predicates for all object pairs and treat the resulting set of sequences (Predicate Vector Sequences, PVS) as an action descriptor. As part of our representation, we introduce a symmetric, time-normalized pairwise distance measure that relies on finding an optimal object correspondence between two actions. We experimentally evaluate the method on the classification of various manipulation actions in video, performed at different speeds and timings and involving different objects. The results demonstrate that the proposed representation is remarkably descriptive of the high-level manipulation semantics.
