The Distro Interview: Microsoft Principal Researcher Bill Buxton
Bill Buxton has spent most of his career getting between humans and computers. While his initial focus was on music and digital instruments, that eventually led to an interest in human-computer interaction, and pioneering work with multitouch systems and other user interfaces. He worked with the famed hotbed of innovation Xerox PARC in the late 1980s and early 90s, and was later Chief Scientist for software firm Alias Wavefront before claiming the same title at SGI Inc. when that company acquired the former in 1995. After a time running his own Toronto-based design and consulting firm, he moved on to Microsoft Research in 2005, where he continues to serve as the organization's Principal Researcher.
We recently had a chance to pick his brain and get his thoughts on a range of issues, including state of design at Microsoft, the future of natural user interfaces, and whether we're really entering a "post-PC" era.
Your title at Microsoft is Principal Researcher. Can you explain a bit what that entails? What is your day-to-day like?
Perhaps the best way to describe work-a-day life is "varied." Within MSR, we have a fair bit of flexibility in terms of choosing how and where we spend our time. We all publish work in peer-reviewed scientific literature. I also spend a chunk of time interacting with different groups, often in other MSR labs -- especially MSRC in Cambridge UK, and MSRA in Beijing as well as with the product groups on tech envisioning and problem solving. Finally, I spend a fair bit of time on outward facing activities - speaking at conferences, visiting companies of interest, and especially engaging with students at various universities. My rule for success in such ventures is that I learn as much from the students and companies, as they do for me. I love this part of my job. (Actually, I love almost all of it!)
What's your assessment of the state of design at Microsoft these days?
There have been pockets of design excellence within the company for years. The hardware group that designs our mouse and keyboard products is just one. However, since Microsoft has traditionally been perceived as a software company, their designs have been frequently overlooked when people think about design at Microsoft
But from the long-standing pockets is emerging a collective transition whereby design is joining technological excellence as a core value in the development of products and services. The emergence of the Metro design is a reflection of this; likewise the incorporation of technologies such as voice, touch and gesture, and the pursuit of ever more "natural" ways of interacting with systems. Kinect is a great example of this. It is really interesting how many different parts of the company were touched in its development -- a great reflection of how design and innovation are ever-more permeating the culture of the company. The value and impact of this is reflected in the rapidity with which Kinect, and the speech and gestural modalities that it supports, have migrated to applications beyond games.
As always with design, this is a work in progress, since design is a moving target. Do I wish that we were further along, and doing even more? Of course. Anything else would lead to complacency -- and, it is my job to be impatient. By the same token, I am extremely excited by the momentum gained over the past five years, and the potential that I see in the next five.
What do you think of the notion of the "post-PC" era?
I have mixed feelings about the term and what is meant by it. As someone who worked with Mark Weiser on developing the notion of ubiquitous computing at Xerox PARC, I am prone to think of a future where, if you are aware of the computer, that is an indication of a failure of design. So, in that sense, I dream about the "post-computer era" -- not because there are no more computers, but because they are so seamlessly and transparently integrated into the tools that we use in our lives that we lose consciousness of them as distinct entities. This is largely already the case in our automobiles, about 1/3 of the cost of which is in the embedded information systems (with all of the hot topics such as "sensor networks," "ambient intelligence," "distributed, parallel, real-time computing" etc. included), and where few of us know or care what processor is being used, how fast it is, what networking protocols are, or what the operating system is.
On the other hand, I often interpret talk about the "post-PC era" as implying a world where the desktop PC, with its keyboard and mouse, has disappeared. That, I think, is a misguided notion. In my analysis, what is happening with the emergence of new and varied form factors and classes of device is an augmentation of the overall eco-system, not one class of device rendering another obsolete. Of course, there will be some of that. But in general, that is not how things work. Cinema did not replace live theatre, nor television cinema. Yes, each new technology may change the overall distribution of the market of "drama watchers," but in that case, for example, the overall market grew as well. Following my claim that everything is best for something and worst for something else, the desktop PC will continue to exist and develop for the many things for which it is well suited, and other devices will take over from it for the things for which it is less well suited. That is all good, and what we have predicted since the early days when we first articulated the concept of ubiquitous computing.
What occupies my mind has less to do with the survival of any one of the ever increasing classes of device. Rather, it is the question of how to make all of these devices work seamlessly and transparently together.
You've said that you expect Surface-like devices to eventually be in people's homes. Is that something you still see happening soon?
If you contrast the size, cost and performance of the original Surface device with Surface 2.0, it is clear that the trend points in that direction, and supports my contention. Large format displays are getting ever more affordable and higher performance. Tools for interacting with them, be they touch, stylus, voice and gesture (as with Kinect for Xbox 360), are offering ever more appropriate ways to go about doing so. And, Moore's law, advances in networking technology, coupled with the cloud suggest that the means are being put into place whereby new form factors will not only be affordable for the home (and office, and shop, and library, ...) but also "play nice" with both users and the other technologies in the ecosystem. Making sure that happens is a matter of research and design -- something that we are seriously engaged in, which echoes back to your first question.
Are there areas that you think could benefit from natural user interfaces that haven't yet?
I would say that we have just scratched the surface in this regard. We live in the physical world, and for a long time there was no digital world. Today we have some connections between the two worlds, but when we can truly blend them together, we get something completely new, something we are only now beginning to understand. This is why this is the most exciting time in my career since the first time I used a computer 41 years ago. Compared to what we have done in the past, what we can do today is fantastic. Compared to where we have the potential to be in 10-20 years, we still have a lot of work to do. We still work with computers. But reflecting what I said above, that is just a stepping stone to getting to the point where we are unaware that we are dealing with computers. As the saying goes, people don't want a hammer or nail, nor even a hole in the wall. They want their picture hanging on the wall at the spot where they want it. That is the high order task. Every time you encounter an issue dealing with some intermediate step or tool in doing some higher order activity, that may well be an opportunity for a more natural, or appropriate means of accomplishing it.
In the future, neither the physical world nor the digital world will be sufficient by itself. The ability to translate your real-world experience metaphorically into the things that you want to do in the virtual world is key.
Are there any works of science fiction that have inspired your work? Or anything that you feel has been particularly prescient?
I confess that I have read only one science fiction book in my life -- Stranger in a Strange Land. And, given that he is such a fan, I am really relieved that Rick Rashid, who founded Microsoft Research, still hired me knowing that I (still) have never seen a full episode of Star Trek, or any of the movies. I tend to read history. Sure I enjoyed films like Tron, The Matrix, Terminator II, etc., and not just because I used to work in the 3D software industry making tools for visual effects and animation. It is just that I have a really big library, and so many books that I am still dying to read, that science fiction has just not made the cut. And actually, just so that it doesn't feel singled out, for the most part, I have stopped reading most fiction of any type over the past 15 years. Anyhow, my life is better than science fiction!
You've built up an impressive collection of gadgets over the years. Can you talk about some of your favorites? What drives you to collect them?
Yes, I have been collecting interactive gadgets, and therefore much of the history of my craft, over the past 35 years or so. I never intended it to be a collection, per se. I just kept things that captured my interest -- sometimes because they were so cool, and sometimes because they were so bad. What they all have in common is that they provide really valuable lessons about design. You can read about something, and understand the concept. But if you have a chance to actually experience it because you have the gadget, then your way of knowing is at a far deeper, visceral level.
The devices in my collection are not just about remembering, respecting and learning from the past. Pretty much every one of them teaches some lesson that is really relevant about the future. Take, for example, two watches that I have, both from Casio: the AT-550 and the DB-1000 TeleMemo. Each had a capacitive touch screen covering the entire crystal, and rather than using the touch screen to touch virtual buttons, one entered data into the watch by writing characters with your finger on the touch-sensitive crystal. Yes! They had built-in character recognition. The AT-550 was a calculator watch on which you printed numbers and arithmetic operators, and the DB-1000 did that, and had an address book as well, so it could recognize both numeric and arithmetic characters. That is, in this age of texting, Twitter and touch screens, these watches teach us an eyes-free, heads-up method to enter text into our mobile devices. It is a way that lets me look at the address or phone number on a piece of paper while I copy it into my phone; or a way that I can tweet about a lecture while keeping my eyes on the speaker and the slides. Of course, this is only true if designers knew about them and appreciated their lessons.
Now here is the kicker: these watches were available commercially in 1984 -- the year that the first Macintosh came out -- and they sold for under $100. Now think about it: this was over 19 Moore's Laws ago, a factor of over 262,144. What these devices teach me is humility. Rather than puffing out our collective chests as to how well we are doing, they really beg the question, "What have you, as an industry, been doing for the past 20 years? You can, and you should, do better!" My collection keeps me honest.
What do you see being the biggest trends in technology over the next three to five years?
I see a shift to a place where we won't be dazzled just because a product is well designed and works well. Our collective customers should be able to take that for granted, and it is our job to make it so. But that is not enough. The problems of design and complexity do not go away, even if we all surpass that bar. Rather, they just move to a different place: the complexity that is emerging in terms of how all of these (individually) easy to use devices work together. We need a comprehensive ecosystem that combines elements of each to produce an integrated set of experiences for people, so they don't have to manage each of the underlying separate devices.
The challenge and the target trend is in the direction where every device and service you acquire not only delivers great value and experience on its own, but that value and quality of experience is enhanced by every other product and service that you already have, and each of them enhance that which you just acquired. And, in the process, by adding more of the right technology, we end up with an overall reduction in complexity for our users.
Let me conclude with one existence proof: my car and my phone. Each works just fine on its own. But when I jump in the car with my phone, seamlessly, they couple and Windows Phone 7 no longer presents a Metro design (if it did, and I used it while driving, I should lose my driver's license). If I get a text, it is read to me. If I want to rely, I just dictate it. If the phone rings, the stereo is turned down and becomes a speaker phone. If I park before the call is over, I just turn the car off, pick up the phone, and the conversation makes a seamless transition to the hand-set. I want all of my devices to have this same capability to seamlessly and transparently aggregate and disaggregate -- and I believe that it will happen. Helping make it happen, in an appropriate manner is, in fact, my job. And I love it.
This interview first appeared in Distro issue 37. Cover illustration by Sean McCabe.
