Feature representation of RGB-D images using joint spatial-depth feature pooling

Hong Pan, Søren Ingvor Olsen, Yaping Zhu

Research output: Contribution to journalJournal articleResearchpeer-review

2 Citations (Scopus)

Abstract

Recent development in depth imaging technology makes acquisition of depth information easier. With the additional depth cue, RGB-D cameras can provide effective support for many RGB-D perception tasks beyond traditional RGB information. However, current feature representation based on RGB-D images utilizes depth information only to extract local features, without considering it to improve robustness and discriminability of the feature representation by merging depth cues into feature pooling. Spatial pyramid model (SPM) has become the standard protocol to split a 2D image plane into sub-regions for feature pooling of RGB-D images. We argue that SPM may not be the optimal pooling scheme for RGB-D images, as it only pools features spatially and completely discards their depth topological structures. Instead, we propose a novel joint spatial-depth pooling (JSDP) scheme which further partitions SPM using the depth cue and pools features simultaneously in 2D image plane and along the depth direction. By combining the JSDP with standard feature extraction and feature encoding modules, we outperform state-of-the-art methods on benchmarks for RGB-D object classification, detection and scene recognition.
Translated title of the contributionFeature extraction of RGB-D images using joint spatial-depth feature pooling
Original languageEnglish
JournalPattern Recognition Letters
Volume80
Issue number1
Pages (from-to)239-248
Number of pages10
ISSN0167-8655
DOIs
Publication statusPublished - 2016

Keywords

  • Faculty of Science

Cite this