I like to think of Interaction Design in its purest form as being about shaping the perception of an environment of any kind. Yes, today the discipline is so closely tied to visual displays and software that it almost seems to revolve around that medium alone, but that’s only because as of now, that’s pretty much the only part of our environment over which we have complete control.

The one field that has come closest to overcoming this limitation is the video game industry whose 3D games are the most vivid and complete alternate realities technology has been able to achieve. Game designers have control over more aspects of an environment, albeit a virtual one, than anyone else.

Lately I’ve been thinking a lot about this idea that interfaces can be more closely integrated with the environment in which they operate. I’d like to share some of what I’ve learned from the universe of video games and how it might be applicable to other kinds of designed experiences.

In Designing for the Digital Age, Kim Goodwin criticizes the term “Experience design” as being too presumptuous because we don’t really have the power to determine exactly what kind of experience each person with their own beliefs and perceptions has. Even when we work across an entire event from start (e.g. booking a flight) to finish (arriving at the door), there are still countless factors outside our control that can significantly impact how a person will experience it.

Video game designers on the other hand can orchestrate a precise scenario since almost every detail in their virtual world is for them to determine. They can arrange exactly what kind of person sits next to you on a flight no matter who you are or how many times you take that flight.

That isn’t to say that videogames don't have their limitations. Of course, it isn’t completely true that game designers can determine who sits next to you. They can only determine who your avatar sits next to. The most significant weakness of videogames is the inability to truly inhabit a designed environment or narrative. As much control as we may have over a virtual world, as long as we are confined to experiencing it through television screens and speakers, it won’t be anywhere near comparable to our real world.

Fortunately, there’s a growing effort to address this lack of immersion.

A key area of the problem lies in how we’re presented and interact with complex information diegetically, that is, interfaces that actually exist within the game world itself.

Before continuing, it helps to be familiar with some basic concepts and terminology around diegesis in computer graphics, the different spaces of representation between the actual player and their avatar. The diagram above illustrates the four main types of information representation in games.

Non-diegetic representations remain the most common type of interface in games. In first person shooters, arguably the most immersive type of game since we usually see the scenery through our avatar’s view, the head-up display has remained an expected element since Wolfenstein 3D first created the genre.

Photo by Delcio G.P.Filho.

The big apple.

Many say it’s the greatest city in the world. Whether or not you agree, there’s no denying it’s an incredibly dense place with an overwhelming amount of people and things to do. Not only are there over 40 million tourists annually, jostling to see the sights and get a taste of the cultural capital but there are also over 8 million people living here − struggling to manage the tasks of daily living amongst all the tourists. That’s a lot of people with very different goals. How do they all figure it out?

(For those of you not in New York, you might want to consider pressing play for some mood music.)

The usability of cities

I’ve been on the road for the past few weeks and am struck by how some cities are easier to use than others. Since I’m in the business of interfaces I’ve been thinking about it in those terms. Just like software, smaller cities with few features are generally (but not always) fairly easy to use. Once you have a large, complex city with many features − like NYC − it gets much more challenging to maintain that ease of use.

New York City is an incredibly powerful interface with multiple entry points and endless features. One might say it has feature bloat. It overloads the senses and it’s not always easy to navigate and understand, yet people learn to use it effectively and often grow to love it.

In that way it’s like Adobe Photoshop - optimized for expert users, perfect for their needs once they have taken the time to learn how it works, but very intimidating to novice users. Over 40 million of tourists enter the city each year and have to navigate the New York City ‘interface.’ How do they figure it out?

Amazement operated on many levels during the Opening Ceremonies of the Beijing Olympics. During each performance, my mind struggled to process what I was seeing. What is this? How in the world did they pull this off? Where does an idea like this even come from?

TV: These small boxes will now take the form of a keyboard, and the keyboard will sprout a peach blossom.
Doug: ... Huh.
TV: Now the small boxes, which have made precise, machine-like movements for the last ten minutes, will reveal that humans have been operating them the whole time.
Doug: ... Wait, what? ... How ...
TV: Now a globe will rise, and dozens of people will fly around it in precise circles.
Doug's brain: [explodes]

In a Wahington Post editorial, Roger K. Lewis recently wrote that NBC didn't once mention the architects of the venue, Beijing National Stadium. Hmm. That's funny. I didn't mention them during the telecast either, but that's because my brain had been reduced to a pre-verbal state.

The BBC miniseries based on Steward Brand's How Buildings Learn became available on the Internet a few days ago. It's chock-full of provocative stuff, and lays out compelling arguments about how structures succeed or fail in satisfying the needs and goals of people. (Let's hear it for design on TV! First Mad Men, now HBL. It's a televisual golden age!)

The Ferry Building

As I watched the opening episode, I thought of the quintessential local example of a learning building: The Ferry Building in downtown San Francisco. Built in 1898, it served as a ferry terminal for points around the Bay; as San Francisco changed and bridges eased the traffic burden, it gradually fell into disrepair. In 2004, it re-opened as gourmet food court, serving the prosperous downtown lunch crowd. San Francisco changed, and the Ferry Building "learned" to address a new set of needs. Beautiful.

Is architecture really a good analog for IxD?

Aside from all of the fascinating examples of the ways in which our built environment responds (or doesn't respond) to change, what the miniseries reveals to me more than anything is the limitation of using architecture and construction as models for software design and development. Architecture serves as a helpful stand-in when you're talking about the macro stuff — the planning process, the rough apportionment of the screen "real estate," and discussions around extensibility or repurposing — e.g., Is this thing the first piece of the big structure, or is it the temporary thing that we live in while the big structure is built?

But when you're talking about the way people experience things in a digital environment, architecture is a limited analog. Software is made up of subtle, nuanced interactions and ever-evolving technical capabilities. Interacting with software is conversation between two active participants; it's fast-paced and packed with immediate possibilities. For example, changing context in software seems more akin to a change in facial expression than, say, a movement to a different room. (It should, anyway). The ever-evolving technical capabilities have created a world in which we're all often experiencing some particular digital interaction for the first time; in fact, if someone wrote a book about how software is experienced, it could be called something like, "How Software Teaches Us How to Use It."

Solutions begat problems, problems beget solutions

Of course, there's another side of Brand's perspective that's relevant to our work: Most design projects (at Cooper, anyway) begin with what is presented as a straightforward task: Design a solution for the problem the clients have identified. Architects probably experience a transformation similar to ours, because the real problem is often quite different than what the client has articulated. Brand's perspective is interesting to consider here, because our solution often simply modifies (or modulates) the problem -- makes it smaller, hopefully — but still: Will our solution be able to handle the need to evolve to further reduce the problem? Of course, if anything could learn and teach at the same time, it's software. But software that can learn ... Hmm. Something sounds fishy about that. Remember that part in Terminator where they're talking about how the computers took over?

[Start with Episode 1 of How Buildings Learn at Google Video, and thanks to smashingtelly for the tip]

Community design centers are non-profit organizations that provide high quality design to underfunded and underserved areas of a community. They're usually established as extensions of colleges and universities, and they're intended to positively impact the surrounding community though design — usually through the physical build.

Back when I was pursuing my degree at the University of Cincinnati’s college of Design, Art, Architecture and Planning, I worked for one, with the intention of helping to revitalize one of the more depressed parts of Cincinnati. The focus was the design of a farmers market, an initiative that included contributions from Architecture, Planning, Industrial Design, and my own discipline of study, Graphic Design. The end result of our work is a vibrant, exciting environment, and this experience got me thinking about ways in which my current discipline could take part.