mmKey: Universal Virtual Keyboard using A Single Millimeter Wave Radio

Abstract

Keyboard acts as one of the most commonly used mediums for human-computer interaction. Today, massive Internet of Things (IoT) devices are designed without a physical keyboard as they go tiny, but are almost all equipped with a wireless module for networks. In this work, we aim to enable a universal virtual keyboard using wireless signals, which would allow a typing interface for tiny IoT devices or serve as a portable alternative to the unwieldy physical keyboards. To this end, we present mmKey, the first universal virtual keyboard system using a single millimeter-wave (mmWave) radio. By leveraging the unique advantages of mmWave signals, mmKey converts any flat surface, with a printed paper keyboard, into an effective typing medium. mmKey enables concurrent keystrokes and supports multiple keyboard layouts (e.g., computer keyboard, piano keyboard, or phone keypad). We design a novel signal processing pipeline to detect, segment and separate, and finally recognize keystrokes. mmKey does not need any training except for a minimal one-time effort of only three key-presses for keyboard calibration upon the initial setup. We prototype mmKey using a commodity 802.11ad/ay chipset, customized to support radar-like operations, and evaluate it with different keyboard layouts under various settings. Experimental results with 10 participants demonstrate a keystroke recognition accuracy of > 95% for single-key case and > 90% for multi-key scenario, which leads to a word recognition accuracy of > 97%.