SpecTrans

Versatile Material Classification
for Interaction
with Textureless, Specular and Transparent Surfaces

Introduction

Surface and object recognition is of significant importance in ubiquitous and wearable computing. While various techniques exist to infer context from material properties and appearance, they are typically neither designed for real-time applications nor for optically complex surfaces that may be specular, textureless, and even transparent. These materials are, however, becoming increasingly relevant in HCI for transparent displays, interactive surfaces, and ubiquitous computing.

SpecTrans, New Sensing Technology

We present SpecTrans, a new sensing technology for surface classification of exotic materials, such as glass, transparent plastic, and metal. The proposed technique extracts optical features by employing laser and multi-directional, multi-spectral LED illumination that leverages the material’s optical properties.

The sensor hardware is small in size, and the proposed classification method requires significantly lower computational cost than conventional image-based methods, which use texture features or reflectance analysis, thereby providing real-time performance for ubiquitous computing.

DESIGN OF MULTI-SPECTRAL, MULTI- DIRECTIONAL MATERIAL CLASSIFICATION

SpecTrans combines a fast image sensor with multi-directional, multi-spectral imaging to capture materials’ varying optical properties under different lighting. This makes it possible to extract simple, yet efficient features for material classification that cannot be observed given a single image under a fixed illumination.

Additional 20 LEDs in 5 different wavelength and from 4 directions are arranged on a custom PCB. We use a high-speed laser optical mouse sensor with a small form factor lens. We extend these optical elements with four LED clusters placed at different positions, each containing LEDs at different peak wave lengths of blue (475 nm), green (525 nm), red (621 nm), and two infrared (850 and 940 nm) for a total of 20 LEDs.

Surface Sensing with Coherent and Incoherent Light

SpecTrans has 26 different lighting conditions with LEDs in five different wavelengths and four locations, which enables classification of highly specular or even transparent surfaces.

Our current implementation can capture four features at four different exposure setups with 26 lighting conditions (416 features in total) in merely 117 ms.

Interactions with Transparent Materials and Surfaces Using Optical Properties

Our evaluation of the sensing technique for nine different transparent materials, including air, shows a promising recognition rate of 99.0%.

We can encode imperceptible IDs with transparent surfaces. Interacting with transparent surfaces and displays has important applictions. By sensing imperceptible optical characteristics in the surfaces, the system maintains its transparent characteristics, while being able to respond to different user interactions.

Publications

Munehiko Sato^*, Shigeo Yoshida^*, Alex Olwal, Boxin Shi,
Atsushi Hiyama, Tomohiro Tanikawa Michitaka Hirose, and Ramesh Raskar.
“SpecTrans: Versatile Material Classification for Interaction with Textureless, Specular and Transparent Surfaces,” ACM CHI ’15, Seoul, Korea, April 2015.
(^* The first two authors contributed equally to this work.)
SpecTrans Paper Image
Download Paper PDF, ACM Digital Library