With the rising need for reliable and real-time pose estimation in resource constrained environments such as smartphones, IoT devices, and head mounts, we need an efficient and compact pose estimation framework. To this end, we propose PoseFromGraph, a light-weight 3D pose estimation framework. The inputs to PoseFromGraph are: a graph obtained by skeletonizing the 3D meshes using the prairie-fire analogy and the RGB image, and the output is the 3D pose of the object. The introduction of 3D shapes to the architecture makes our model category-agnostic. Unlike computationally expensive multi-view geometry and point-cloud based representations to estimate pose, our approach uses a message passing network to incorporate local neighborhood information at the same time maintaining global shape property in a graph by optimizing a neighborhood preserving objective. PoseFromGraph surpasses the state-of-the-art pose estimation methods in terms of accuracy achieving, 84.43% on the Pascal3D dataset, and at the same time yields 4x reduction in the space and time complexity. The compact pose estimation models can then be used to facilitate on-device inference in applications in Augmented Reality and Robotics for 3D virtual model overlay.
Representing Shape
We use skeletonisation to reduce the point cloud of any given object to a simple graph. These skeletons provide a compact and expressive shape abstraction to 3D shapes and volume while reducing the memory and computational requirements. These graphs are invariant under transformations such as rotations and translations, and devoid of unwanted noise in the graph in the form of parallel edges, loops and short branches.
Pose Estimation
We use two different feature extractors: (i) Convolutional Network to extract features from the given RGB image and (ii) Graph Neural Network or Message Passing Network to extract shape feature from the given graph. These features are used to predict the 3-D pose of the.
Evaluation
We perform our evaluation on the Pascal3D+ dataset. We achieve state-of-the-art accuracy while reducing the memory and time requirements by approximately 4 times.
