Toward real-time grocery detection for the visually impaired

We present a study on grocery detection using our object detection system, Shelf Scanner, which seeks to allow a visually impaired user to shop at a grocery store without additional human assistance. Shelf Scanner allows online detection of items on a shopping list, in video streams in which some or all items could appear simultaneously. To deal with the scale of the object detection task, the system leverages the approximate planarity of grocery store shelves to build a mosaic in real time using an optical flow algorithm. The system is then free to use any object detection algorithm without incurring a loss of data due to processing time. For purposes of speed we use a multi class naive-Bayes classifier inspired by NIMBLE, which is trained on enhanced SURF descriptors extracted from images in the GroZi-120 dataset. It is then used to compute per-class probability distributions on video key points for final classification. Our results suggest Shelf Scanner could be useful in cases where high-quality training data is available.