Space-centered computation embedded in ordinary environments defines
intelligent spaces populated by cameras, microphones, displays, sound
output systems, radar systems, wireless networks, and controls for physical
entities such as curtains, lighting, door locks, soda dispensers, toll gates,
and automobiles. People interact in intelligent spaces naturally, using
speech, gesture, drawing, and movement, without necessarily being aware that
computation is present.
Human-centered computation supports social interaction in its familiar context.
People communicate with computation easily, as they do with each other, using
shared knowledge and intelligence. Freely-available computation makes it easy
for people to monitor and control their environment. Physical objects become
immensely more useful and usable, and people and agents in other environments
become immensely easier to reach.
Environmental devices, together called an E21, provide a local-area
computational and communication back-plane for an intelligent space. E21s are
connected to nearby sensors, actuators, and appliances, suitably encapsulated
in physical objects. They communicate with each other and with nearby handheld
devices (H21s) through dynamically configured networks
(N21s). E21s provide sufficient computational power
throughout the environment
to communicate with people using natural perceptual resources, such as speech
to support Oxygen's user technologies wherever people may be, and
to monitor and control their environment.
E21s, as well as H21s, are universal communication and computation appliances.
E21s leverage the same hardware components as the H21s so that the same
software can run on both devices. E21s differ from H21s mainly in
their connections to the physical world,
the computational power they provide, and
the policies adopted by the software that runs on the devices.
E21s connect directly to a greater number and wider variety of sensors,
actuators, and appliances than do H21s. These connections enable applications
built with Oxygen's perceptual and user technologies to monitor and control the
An E21 might control an array of microphones, which Oxygen's perceptual
resources use to improve communication with speakers by filtering out
background noise. Similarly, it might control an array of antennas to permit
improved communication with nearby H21s that, as a result of a better
signal-to-noise ratio, use less power. Multiple antennas mounted on the roof
of a building, as well as incoming terrestrial lines, connect through E21s to
high-bandwidth, local-area N21 networks.
Through the N21 network, an E21 can connect unobtrusively to H21s in the hands
or pockets of people in an intelligent space. It can display information on an
H21 display in a person's hand or on a nearby wall-mounted display; it may even
suggest that the person step a few feet down the hall.
E21s provide more computational power than H21s, because they are limited
neither by size and weight constraint imposed by portability nor by batteries
as a source of power. Initial prototypes are based on commodity workstations.
Later prototypes will use Raw computational fabrics
to provide better cost-performance.
This computational power provides an E21 with more powerful perceptual
resources than an H21. For example, an E21 may steer an array of microphones
to pick up a speaker's voice, use video imaging of lip movements to improve its
speech recognition, or take into account an image-based understanding of a
person's gaze, expression, and intent. Its computational power enables
effective integration of multimodal perceptual resources.
E21s can be viewed as massive computational sheets, embedded in the walls of
buildings, that provide large amounts of aggregate computational power
throughout the environment. Nearby H21s can use this power to offload work.
An E21 can use its computational power to track people in an intelligent
space, to recognize faces, and to provide the most convenient service it can to
each person in the space.
E21 software is robust. It recovers from failures by trying tasks again,
trying them with a different selection of resources, or trying different plans.
E21s provide software repositories, from which H21s can download code to
reconfigure themselves for particular tasks.
E21s provide nearby H21s with additional high-bandwidth connectivity to
satellite and terrestrial networks.
The Intelligent Room is a highly interactive environment that uses embedded computation to observe and participate in normal, everyday events. Microphone and camera arrays enable it to listen to people and observe what they do. People can speak with, gesture to, and interact with it in other complex ways. Smart sketching and design tools enable people to express their ideas and the room to record them. Other tools capture and record the essence of meetings in support of human collaboration. A robust agent-based software infrastructure supports the operation of these tools. (Trevor Darrell, Vision Interface Group; Howard Shrobe, Project AIRE; Randall Davis, Design Rationale Group)
Cricket provides information about location, orientation, and geographic spaces. It works indoors, where line-of-sight to GPS satellites is unavailable and electronic equipment may interfere with traditional magnetic compasses. Cricket beacons, mounted on walls or ceilings, transmit ultrasound and RF signals; compact listeners, attached to mobile or static devices, use the difference in signal arrival times to determine where they are. Cricket allows users and applications to discover their locations without tracking them; its operation and administration are completely decentralized. (Hari Balakrishnan, Networks and Mobile Systems; Ken Steele, Oxygen core team; Erik Demaine, Theory of Computation; Seth Teller, Computer Graphics Group)