Sixth Sense

While I was at Microsoft, I was fortunate to be able to work on a few personal research projects. This was one of my favorites — I started with these hypotheses:

Hypothesis #1:
We can leverage our brain function to filter out unwanted distractions without interruption.

Hypothesis #2:
Subtle haptic feedback over time can shift intent and awareness to a subconscious state.

Hypothesis #3:
Directionality (bearing), including an indication of center, can be derived from a stereo haptic signal oriented to the body.

I was able to build multiple working prototypes using off-the-shelf hardware (Arduino, Seed Studio, etc.), Node.js and a custom iOS application. The first prototype was tethered to my laptop, while the latter prototypes were untethered and connected to my iPhone via Bluetooth.

Below is the project proposal, followed by the results of the experiment.

Introduction

In the world of application notifications, interruption is a delicate subject. And it becomes far more complex when the notification is derived from user insight and algorithms — preempting the user to something deemed important enough to interrupt the inferred activity state.

The problem with notifications are that they require a level of interruption substantial enough to get the user’s attention, otherwise they become ineffective. The variability of an individual’s willingness to be interrupted makes this even more challenging.

Haptic Radar

The idea of a “haptic radar” has been around for a number of years - an array of actuators worn around the ankle, waist, neck or head, which provides a tactile indication of directional relevance. In one example [1] a haptic band was attached to the user’s ankle and continuously provided an indication of true north. Within a week, the user no longer felt the actuator pulses and internalized the directional awareness. The user stated that the directional persuasion felt intuitive rather than external [2].


Oriented Notification

Notification in its current state is primarily one dimensional – a notification can be tuned to describe “what”, but without screen confirmation, “where” is not implied. Taking the concept of the “haptic radar”, and applying it to a spatial notification model would add a sense of physical context to the alert.

Internalized Awareness

Based on the north-indicating haptic radar experiment mentioned, my hypothesis is that an actuator emitting specific frequencies, and placed in a particular location (e.g., behind the ear) could, over time, move from an external sensation to an internal cognitive impulse. If true, such a mechanism could present interest or relevance as a subconscious intuition or 6th sense, rather than an external notification. Subconscious decision making is highly optimized, so there would be far less risk of unwanted interruption. As in an un-augmented reality, we choose to either engage or ignore our impulses to explore.

Hardware


The hardware implementation could be realized in multiple forms such as: sunglasses, headphones, neck band, and behind-the-neck band. The sensor location could also be placed elsewhere on the body, but it’s unclear how effective it would be. Directional awareness would require a compass mapped to head or body position.

Software

Two primary services are needed: 1) A service aggregating relevant interests and social relationships, and 2) an interface with the mobile device to establish web connectivity, GPS data and query the data service.

Results


For the initial test I used piezoelectric speakers taped behind the ears and a low frequency pulse for notification. I spent some time trying to find the threshold of awareness, but found that either the signal was too weak, or tickled my skin uncomfortably.

I leveraged an insight from the Japanese Haptic Radar example above to inverse the notification technique — instead of pulsing for a notification, the pulsing would be constant and would stop when a notification occurred. I found that a consistent 1hz pulse quickly trained the brain to ignore the signal and a pause of more than 3 seconds would initiate an awareness sensation. My original hypothesis of directional awareness was less successful, but could vaguely distinguish left and right directionality.

I eventually got tired of taping speakers behind my ears, so tried to change to a low frequency audible chirp into standard earbuds. While this worked, the steady pulse was much more difficult to tune out and also interfered with other audible functions (conversations, phone calls, etc.).

Below are some photos of the hardware experiments.


Initial hardware components used (w/GPS and Spark Core connected to a MiFi)


Built unit


Prototyping


2nd prototype with headphones


3rd prototype with Bluetooth LE