A Vision of a New Human/Computer Interface
by Tom Riley
[Editors note: This paper is part of a larger series of papers. Read this article for an overview and list of papers.]
The Challenge
From the earliest days of computing, contests have been proposed and taken up wholeheartedly. Deep Blue won the difficult chess master contest but Turing’s test timed out in the year 2000 without a clear winner. Such contests provide a focus for research and innovation while promoting open communication among the competing groups. The beginning of a new millennia deserves a new contest: The Great Gold Rush of 2002.
The goal of the contest is to develop a new environment for the interface between humans and computers. The winner must greatly increase people’s productivity and, above all, provide an experience that the user loves. This outcome is quite possible, and Intelligent Agents (IA) undoubtedly will be prime players.
To date, all human/computer interfaces have been designed primarily to meet the limited capabilities of machines. Not only were the machine limits of critical importance, but the programmers who developed the operating systems had little understanding of the needs, limitations, and capabilities of their users. There simply was not enough information available to produce a lasting design.
Present-day machines have a gross excess of hardware capability, but software development has lagged far behind. Production of new software is then the missing key to greater human productivity. And through greater productivity is the path to higher standards of living and the availability of great resources to solve our problems.
In the last two decades, extraordinary additions to our understanding of the human brain have produced an exciting research climate. This information can now be applied to the design of a new human/computer interface. For the first time the actual abilities and limitations of the human brain can be taken into account and provide a balance to our detailed knowledge of the workings of the computer.
Devising a top-down design is an important activity for the early stages of the design process that is much heralded in design books and courses. This effort balances the technical people’s natural tendency for bottom-up detailed design that has lead to such great strides in computer hardware. We love to build new building blocks and let the bit picture take care of itself but eventually these two design efforts must meet in the middle. Top-down designs for the human/computer interface have not yet been particularly well explored but the time is ripe.
Continued growth of the world economy requires the continuous growth of human productivity. Wealth based on real productivity gains is sustainable; wealth based on flash, promises, and hype is cyclical. It is always a lot more fun if you have placed a small bet on the outcome of any contest. You could say that the economic prosperity of the entire world is at stake in this one.
The User
What, then, can we say about the users of computers?
First of all, they are human beings. We can safely write off the chimpanzee, dolphin, and space alien computer markets. They simply do not buy enough computers to matter.
We have learned a lot about human beings in the last fifty years. An enormous amount of accessible material is flooding the literature. Much of this is based upon solid academic studies and is often presented in a popular form that is accessible to most programmers.
Starting right now, we can design a system that people love, like they love their cell phones and the Web. I am not talking about a computer interface they tolerate, or love to hate, or merely put up with. I am talking about a billion people loving the thing. This would be a quantum leap.
Modern hardware can now match human input/output preferences quite well. Language is simply the primary method of communication for human beings and is much loved as is demonstrated by the enormous popularity of the Cell Phone. The most likely input for the new interface then will be the human voice. Most people could use a voice computer input device that resembles a cell phone (called a Handy1).
Human beings have vision as their primary input sense, so large, flat, color monitor screens (supported with quality sound) can be the primary computer output device. Again building on their demonstrated love of TV and movies. The Handy will also have a small screen, stylus input, and headset sound for private or remote interaction.
02'ers in the Great Gold Rush
We can think of this challenge as a gold rush. Anyone can enter at any time by coding their own virtual 49'er (or 02'er). The first person or team to find the mother lode gets one billion-zillion dollars. The second place finisher gets %25 of that (just to keep the Feds away). Everybody else gets chicken feed.
The human psych will be the ultimate judge of the winner, and the decision of the judge is final. Harking back to ancient times, the judging will be held in the marketplace.
IA Interface
My early pick for the contest winner is a team of personified intelligent agents (Virtual Humans (VH)) that do your bidding at home, in the office, and on the Web. They will appear on the monitor as computer-generated characters of some sort. They will act as subordinate members of your personal support team. They will buy stuff for you; they will take dictation; they will do accounts. They will even direct your life as through you were the protagonist in some great drama. That is, they will do all these things and more if you tell them to.
What people love most is interaction with other people2. The best indicator for success in life is skill at interacting with other people3. A computer interface that is much like their interactions with real people, then, is our most likely access to the love and productivity we seek. The key here is the capability of the machine to interact with the user to such an extent that the user is satisfied. Still, that interaction must be quite limited, because on one wants subordinates who outshine them or who are a poor substitute for interactions with real people.
Avatars
One of the most perplexing computer contests of the last century was the effort to write software that demonstrated plain common. We failed this one miserably. It is therefore, clear that our intelligent agents must not be required so show simple common sense. We simply do not know how to program it.
One way around the common sense problem comes from the Hindu culture. Their literature contains wonderful stories of gods who appeared on Earth as beings that exemplify only one human emotion. The presentations were the avatars: if an avatar’s state of being was lust, then it was lust incarnate.
Our agents do not have to exhibit much human common sense. They must only exhibit one human-like state of being and then only in a way limited to a few specific situations. In our example, they are subordinate team members and each has the skills of a subordinate team member in only two areas. One area is as a player in your favorite game, and the other is as a life helper. The two skill areas are independent.
The design will be successful only if the interaction with the Intelligent Agent is satisfying to the user. No one is unhappy because Bugs Bunny is not a real bunny, but people would be very upset if Bugs Bunny did not act consistently as Bugs. Similarly, once people accept personified agents for what they are, their level of satisfaction with them will depend on how well they perform their duties and interact consistently. Their incompleteness as a human will be what is expected of them.
There is a critical threshold below which people will hate the personified agent idea, and above which they will love it. This threshold is a quality of the human mind that detects cheats and phonies. This level must be over-vaulted by the productivity of the IA’s. If the IA occurs to the user as a pretend person, or is unresponsive to the user’s real needs (the Microsoft paper clip man), then the user will be angry. This is a real love/hate toggle.
It is clear from our experience with computer games, that a personified agent’s lips will have to move with its speech4, and it must display at least modest hand movements and body language.
A Dynamic Example
The appearance of the team and their virtual environment will be a very personal choice. Some people will want their IA’s to be fully personified, perhaps even appearing as popular characters like Dr. Spock. Others might prefer animals, choosing dinosaurs, for example. Some people will want an active, dynamic setting for a team; others will want something more serene.
My own personal team would be the crew of a sailing ship, racing through now stormy, now calmer seas, always seeking a true course to the safe harbor that lies ahead. My computer would boot to a scene of wind in the rigging and I would be the captain. My crew would stand ready to do my bidding. They would be capable and highly trained both in handling the ship (my computer) and in commerce with the world (over the Internet).
You might prefer an active team that plays basketball or soccer or climbs mountains. Whatever it does, your team certainly will be capable of winning competitions against other users who use the same concept. This approach will be particularly attractive to young people who have grown up with video games.
A Serene Example
The virtual team agents will live in a space that is alive in a human sense. But perhaps you would prefer a virtual space that manifests itself in the architectural5,6,7. Your virtual team might be a garden that changes with the seasons or a house that people have inhabited and loved for generations.
Your computer might open on a garden in spring. A carved wooden bench sits beneath a lush arbor. Behind it is a high, thick stone wall pierced by a heavy oak gate. There are nooks in the wall, plants trained to it, and a small window-like opening a few steps down. A variety of plants of your choosing fill the areas between the paths. The sky above the wall changes with the time of day and with the seasons.
The bench is your favorite place to sit, to meditate, or to talk to a friend. The carving on the back of the bench can morph into a face when you address it directly. When you confer with a distant person, a representation of them appears on the bench, that is build from a photo your computer’s memory and that displays modest hand and lip movements.
The gate is graced with a carving (perhaps a praying mantis). It controls access to your garden (computer) and destroys pests (viruses). If addressed, the carving morphs into a more complete form and turns to speak to you.
Some of the garden’s plants are IA’s as well. The rosemary is for remembrance8 and manages all your computer’s memory resources. The rue is medicinal and aids you in managing your health.
You must take several steps down a path to see through the opening, as the wall is thick5. The opening is small; a nook that pierces the wall. Through it you see the virtual world and by entering you can explore the Web.
The Players
Your agents are all intelligent in that they learn your preferences from working with you9 although each can serve only limited functions. Even so, they will perform more advanced functions than those supported by current operating systems.
One IA will be your security agent. This agent may appear as a person of military bearing (or a mantis carved in a gate). It protects the integrity of your system and is forever vigilant.
Another certainly will be your communications agent. Handling e-mail, answering the phone, and maintaining your address book will be its basic functions. Beyond that, it will be your human networking agent. Tell it your problem, and it will suggest the names of all the people who have helped you with similar problems in the past. It knows who to call.
The most advanced IA would be the producer. It would manage your calendar and your address book, but it will go much farther than that: it will be in the business of producing the best possible life for you. The only reason you don’t have a rich, dramatic life like those in the movies is that you don’t have someone out there making it happen for you. You don’t have a producer. Soon you can have just such a person, or, at least, a product/agent.
In a sense the producer will act as your cruse director or social secretary. It will keep you informed of all the latest happenings in the activities you love. It will be able to work with other people’s producers to manage family activities and gatherings of friends. To see what will happen when two of these IA’s are given permission to manage a love affair, we will just have to wait.
What other IA’s you choose will, again be a truly personal statement.
Pitfalls
One of the great strengths of working out a top-down design is that you can uncover pitfalls in time to address them. One of the greatest of these for personified IA’s is Eliza10.
Eliza was Fortran program originally written in the 1960's that mimicked the conversation of a Freudian psychiatrist. Basically Eliza invited you to talk about personal problems and then mindlessly turned what you said back into more questions. It also steered the conversation toward family relationships and the user’s dreams.
It worked all too well. Many of the college students who tested it got into very extremely personal conversations that should have been held only with a medical professional. In face, Eliza couldn’t provide any real assistance at all. The initial tests were suspended.
The lesson from Eliza is that personified IA’s must not serve as surrogate humans. They must, instead, direct the user to human beings. They must occur to the user as something quite different from real humans, as Bugs is different from a real bunny. The user must find them satisfying for what they are and accept what they are not.
Conclusion
The time is right for a total rethinking of the relationship between people and computers. Issuing a new challenge will provide a focus for people’s actions and insure strong, open communication. This is an enterprise best done in the spirit of a great public competition. 
Notes
1. Michael Dertouzos, The Unfinished Revolution, Human-Centered Computers and What They Can Do for Us (HarperCollins, 2001) [Return to text]
2. Michael Argyle, The Psychology of Happiness (Routledge, London, 1987) [Return to text]
3. Daniel Goleman, Emotional Intelligence, Why it can matter more than IQ (Bantam Books, 1994) [Return to text]
4. K. C. Scott, K. S. Kagels, S. H. Watson, H. Rom, J. R. Wright, M. Lee, K. J. hussey, "Synthesis of Speaker Facial Movement to Match Selected Speech Sequences", http://www-dial.jpl.nasa.gov/~john/papers/ASVpub/ASVPerth_Paper.pdf [Return to text]
5. Christopher Alexander, The Timeless Way of Building (Oxford University Press, 1979) [Return to text]
6. Christopher Alexander, Sara Ishikawa, Murry Siverstein, A Pattern Language, Towns, Buildings, Construction (Oxford University Press, 1977) [Return to text]
7. Richard P. Gabriel, Patterns of Software, Tales from the Software Community, (Oxford University Press, 1996) [Return to text]
8. William Shakespeare, Hamlet, Act IV, Scene v http://www.bartleby.com/70/4245.html [Return to text]
9. Joseph P. Bigus, Jennifer Bigus, Construction Intelligent Agents with Java, A Programmer’s Guide to Smarter Applications (John Wiley, 1998) [Return to text]
10. Robert C. Goerlich, ELIZA as a JAVA Applet, February 3, 1997 http://philly.cyberloft.com/bogoerlic/eliza.htm [Return to text]
[an error occurred while processing this directive]
The Interface Mafia - www.interfacemafia.org
News - About Us - Articles - Reviews - Submit Software - Links - The Mafiosi
this page last updated on
site design by tobyrush.com - web space by westhost.com
follow this link for copyright information