AudioUserInterface.org

Welcome to Audio User Interface Project Home Page!
An initiative to explore using audio media only in Human Computer Interaction (HCI).

Home
Technology
Demo Video
Links
About

What And Why?

How would you interact with mobile device when you are walking, running, driving, meeting... ?

The mindset of user interface must have visual display is wrong. Human ears receive mass information all the time often being neglected in user interface design. Today, the size of mobile device is mostly constrained by its LCD screen size. It should be much smaller to be practical wearable computer. Human has ability to interact with audio media and there are
obviously needs of using truly on-go device (not stop-look-go) if technology provides solution. Let's get rid of LCD and define a new user interface using audio only. That's the vision of this project.

This project defines the methods, system and infrastructure of human computer interaction completely in audio space. It tries to set the common standard of Audio User Interface (AUI). It focus on an innovative Audio UI design as well as net oriented feature. Here are some buzzwords to describe the system briefly:

  • A simple tactile control input with rich audio feedback to replace most functions on mobile device and computer. No visual display needed. User can operate device in Eyes-Free fashion. 
  • Audio Browser in AudioOS as Audio Net Terminal (ANT) to browse an Audio Space on Internet. The Audio Space is described by AudioXML and manipulated by JavaScript and PHP using existing Web/HTTP infrastructure. The Audio Space will replace the concept of Web page which was design for visual UI.
  • Computing on move! Eventually, computer in your ear!

More Backgrounds

The project establishes an innovative system and method of user interface with information processing device, such as mobile device, in-vehicle device, home entertaining system, telephony, Interactive Voice Response (IVR), industrial control system, medical equipment, desktop computer, etc. People use them for personal or work related purpose everyday. The acceptance and utility of the devices rely on not only processing power and storage but more important, on appropriate Input/Output (I/O) modalities and interaction techniques tailored to environment and user intention. Graphic User Interface (GUI), which dominates almost all user interface design, is cumbersome on small display or useless if no display. Audio, on the other hand, is not constrained at all in these cases. Audio perception is not generally less important. It just has a different function than visual perception. Humans embody the physiology needed to absorb information in the form of sound. Just as the eye can perceive many different variations of light: hue, brightness and contrast, the ear is capable of sensing a vast array of sounds through the alteration of timbre, loudness, and pitch. The mind can then associate these sounds with events, objects, or abstract ideas. But human computer interaction has not taken advantages of these abilities. By introducing audio into computer interaction, the aim was to provide more information while shifting additional cognitive load to a different modality.

Furthermore, considering a user engages in a task, user can't be distracted in many cases such as driving, running, walking, meeting or even soldier in battleground. Considering another case in dim light environment, sometimes it's preferred not to see the display at all. It's often live or death issue if visual get distracted like driving. That suggests the user interface should move to background. The user interaction should be in assisting mode rather than dominant mode. It is well know that hearing is passive process. User can easily listen to audio while being occupied with other tasks. Therefore, audio is the best candidate for background computing.

Today's computing or Internet device has thousands of applications in device or on Internet. People use them mostly for some very simple tasks, like check email, listen MP3, read news, check weathers, etc. But most such computers and devices are so powerful and so fancy that overloads user. AUI/ANT addresses this issue by simplifying user interface and user experience. It is so simple that it doesn't need to have visual display. Note that hearing is passive process and seeing requires active attention. By using audio media, user wouldn't feel overloaded. Thus, this simple device is designed to do the tasks suppose to be simple in the first place.


People would always think of speech recognition as promising solution of above issues. It is to some extend. But in many cases they are not enough and in some cases they are wholly inappropriate. Such as: 1) noisy environments, especially in crowds, where even the best speech recognition may never work. 2) No security, cases where a topic under discussion is confidential or private. 3) Irritating, cases where in public area and social environment. 4) Not error-free and unreliable. Even the best speech recognition wouldn't claim error-free. 5) Language dependency. Anyways, speech recognition is still very useful under certain constrained cases such as simple command input or to pick a person from a name list.


Applications

The core technology and system design do not target any single application but sets of applications. It only defines the methods and standards of user interface and provides framework and API for both standalone and net applications. Here is the list of possible applications: MP3 player; cell phone, telephony; in-vehicle computing; net radio; net information audio render like weather report, RSS news feeds, email; speech book; audio GPS; audio game; Interactive Voice Response (IVR) System, home appliance, medical equipment, etc.



Disclaimer: This is US patent pending 
technology and is not open source project at this time.
Last update: 06/02/2009, Silicon Valley, CA