Intuitive Multimodal Adjustments
- Activate contexts through gaze and modify values through gestures: While driving, Edwin gazes (Tobii) at the speedometer on his HUD and draws circles in the air counter clockwise with his right index finger(SKS) to decrease the cruise control speed. Then he quickly gazes at his infotainment system and starts drawing clockwise circles in the air with his right index finger to increase the volume. Similarly he can look at the temperature controllers and adjust the cabin temperature with a gesture of the index finger. As soon as he starts performing a gesture to change the values of a specific mode (speed, volume, temperature, etc.) he can gaze back at the road while continuing to make circle gestures to continue adjusting the values while looking at the road. After adjusting the volume of the infotainment system, the driver decides he wants to skip the song to go to the next one; he can do this by performing a swipe gesture to the right with his right index. As Edwin neither removed his hand from the interaction area nor gazed at another controllable spot, FANCI understands that the infotainment control is still the active context for his gestures.
The driver can interact with an icon on the IC or HUD as well as with the central infotainment display, but they may also interact with a physical object within the vehicle or perhaps even outside of the vehicle. After glancing at the infotainment display, finger rotations clockwise or counter clockwise will increase or decrease volume, while finger swipe gestures will change song/track/preset station. Similarly, if holding up an open palm a number of gaze-responsive context dependent icons may appear in the HUD. For instance a set of available radio channels may be shown and by closing the hand, the driver selects the one he is looking at or last viewed if he is already looking back at the road. If the active context is the phone, showing the open palm will visualize the top ten favourite contacts (most often called) and by closing the hand, the person/number looked at is selected.
Feedback (system reaction) of these changes can be implicit as in the case of volume (changes of volume can be perceived instantaneously) or displayed in an HUD. Alternatively, or in addition to this, an auditory feedback response can be provided to the driver confirming the level of the changes made for changes to speed, temperature, etc.