Sensing Hand Interactions with Everyday Objects by Profiling Wrist Topography

Julius Cosmo Romeo Rudolph, David Holman, Bruno De Araujo, Ricardo Jota, Daniel Wigdor, Valkyrie Savage

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

5 Citations (Scopus)
26 Downloads (Pure)

Abstract

We demonstrate rich inferences about unaugmented everyday objects and hand object interactions by measuring minute skin surface deformations at the wrist using a sensing technique based on capacitance. The wristband prototype infers muscle and tendon tension, pose, and motion, which we then map to force (9 users, 13.66 +/- 9.84 N regression error on classes 0-49.1 N), grasp (9 users, 81 +/- 7 % classification accuracy on 6 grasps), and continuous interaction (10 users, 99 +/- 1 % discrimination accuracy between 6 interactions, 89-97 % accuracy on 3 states within each interaction) using basic machine learning models. We wrapped these sensing capabilities into a proof-of-concept end-to-end system, Ubiquitous Controls, that enables virtual range inputs by sensing continuous interactions with unaugmented objects. Eight users leveraged our system to control UI widgets (like sliders and dials) with object interactions (like "cutting with scissors'' and "squeezing a ball"). Finally, we discuss the implications and opportunities of using hands as a ubiquitous sensor of our surroundings.

Original languageEnglish
Title of host publicationTEI 2022 - Proceedings of the 16th International Conference on Tangible, Embedded, and Embodied Interaction
Number of pages14
PublisherAssociation for Computing Machinery, Inc
Publication date2022
Article number3501320
ISBN (Electronic)9781450391474
DOIs
Publication statusPublished - 2022
Event16th International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2022 - Virtual, Online, Korea, Republic of
Duration: 13 Feb 202216 Feb 2022

Conference

Conference16th International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2022
Country/TerritoryKorea, Republic of
CityVirtual, Online
Period13/02/202216/02/2022
SponsorACM SIGCHI
SeriesACM International Conference Proceeding Series

Bibliographical note

Publisher Copyright:
© 2022 ACM.

Keywords

  • affordances
  • capacitive sensing
  • everyday objects
  • wrist topography
  • wristband

Cite this