Where do you start when you're approaching a complex software design problem? If you work on a large development team, you know that software engineers and UX designers will often approach the same design problem from radically different perspectives. The term "software design" itself can mean very different things to software architects, system programmers, and user experience designers. Software engineers typically focus on the architectural patterns and programmatic algorithms required to get the system working, while UX designers often start from the goals and needs of the users.
In the spring of 2009, I participated in a research study that looked at the ways in which professional software designers approach complex design problems. The research study, sponsored by the National Science Foundation, was led by researchers from the Department of Infomatics at the University of California, Irvine. The researchers traveled to multiple software companies, trying to better understand how professional software designers collaborate on complex problems. At each company, they asked to observe two software designers in a design session. At my company, AmberPoint, where I worked at the time as an interaction designer, I was paired with my colleague Ania Dilmaghani, the programming lead of the UI development team. In a conference room with a whiteboard, the researchers set up a video camera, and handed us a design prompt describing the requirements for a traffic control simulation system for undergraduate civil engineering students. We were allotted two hours to design both the user interaction and the code structure for the system.
Jim Dibble and Ania Dilmaghani at the whiteboard in their research design session
As I'm sure you're thinking right now, two hours is not enough time to fully grasp the intricacies of traffic control, let alone to design the mission-critical software architecture and the user interaction model of a traffic control simulation system. As designers who approach problems from the user's perspective, Ania and I spent most of our allotted time considering the user's experience. We explored the real-world problems involved in traffic control and the tools that users would need to construct maps, set traffic light timings, seed traffic, and evaluate the results of a simulation run. We touched briefly on the way in which the simulation engine might work, but definitely did not provide sufficient detail to enable programmers to get started on the innards of the system.
The final whiteboard diagrams after the two-hour design session
Our team was selected for study
Several months after our design session, the researchers wrote to tell us they were planning a conference on studying professional software design at UC Irvine. They chose our design session as one of three to be analyzed and interpreted by invited participants. The organizers put the call out to researchers from a wide variety of disciplines, including software engineering, design studies, human-computer interaction, cognitive science, and psychology. Participants were given access to both the videos and transcripts, and were encouraged to analyze the design from any of a number of perspectives, including design strategies, design notation, interpersonal communication, decision-making strategies, comparisons to established software design methodologies, and design outcomes. Because the organizers felt that the conference participants would appreciate the ability to discuss their observations and theories with design practitioners, they invited us to attend.
In February 2010, we traveled to UC Irvine to attend the conference. Walking into a lecture hall with more than 50 researchers who had viewed our videos repeatedly, we had a small sense of what Brad Pitt and Angelina Jolie might feel like when walking into a restaurant in LA. Heads turned, and people pointed and whispered.
Researchers from multiple disciplines came from around the world, from as far away as Australia, Brazil, Japan, and Europe. Over a period of three days, we heard over 30 papers analyzing our design strategies, our communication patterns, and our final design solutions. And at the end of each day, we heard closing thoughts from luminaries in the fields of computer science, design studies, and psychology, such as Frederick Brooks (professor at UNC Chapel Hill and author of "The Mythical Man-Month"), Nigel Cross (design studies professor at The Open University, editor of Design Studies, and author of "Designerly Ways of Knowing"), and Michael Jackson (computer science professor at The Open University and author of "Software Requirements and Specifications").
The panel of luminaries speaks at the close of the conference: (L to R) Michael Jackson (The Open University), Mary Shaw (Carnegie-Mellon University), Clayton Lewis (University of Colorado, Boulder), Nigel Cross (The Open University), Barbara Tversky (Stanford and Columbia), and Frederick Brooks (University of North Carolina, Chapel Hill)
The three design teams had taken very different tactics in approaching the design problem. Because we came from a user interface background, our design strategies focused on exploring the problem from the user's perspective: how does traffic signal timing work, and what tasks do civil engineering students need to perform to gain a better understanding of traffic signal timing? The other teams, composed of software architects, worked from the inside out: how does traffic signal timing work, and what architectural patterns can be used to implement the traffic simulation system? While one of those teams touched briefly on the user interface, the other team completed their architectural design in an hour, waving their hands to indicate that all that remained was a basic user interface design.
How our design fared
So, how did the researchers assess our designs? Well, their evaluations depended on their research perspective. Several computer science researchers evaluated the design session based on the technical skills exhibited: the ability to accurately frame the technical problem based on the explicitly stated requirements and the ability to choose an appropriate architectural pattern for the implementation. Design studies researchers tended to evaluate the design on the "soft skills" associated with design: interpersonal communication, ability to articulate the reasoning behind design choices, and the ability to examine and evaluate requirements based on the end-users' needs. Other researchers, rather than being prescriptive, examined the processes and strategies used by the design teams: how did the designers structure their design sessions, how did they explore the breadth and depth of the problem, and how does this compare to the way that the academy teaches software design students to approach problems?
Not many designers have the opportunity to hear a team of cross-disciplinary researchers examine their work. We heard much in those three days, reminding us of the importance of explicitly planning a design session, of framing the design problem, and of conversation as part of the design process. Throughout the workshop, we also passed on our observations to the researchers, letting them know that real-world design problems seldom begin from a complete problem statement or requirements document, and that architects and developers can easily jump to the system's architectural structure while overlooking the user's goals, needs, and outcomes.
Mary Shaw and Frederick Brooks discuss the conference proceedings during a break
After the conference, Ania and I discussed how differently the design session might have turned out if either one of us had been paired with a software architect. We definitely would have spent more time addressing the system's architecture, but we were unsure whether we would we have been able to frame the problem around meeting the user's needs. In engineering-led organizations, designers are seldom included in early-stage design sessions, which can lead to products that don't meet the target users' needs. Collaborating with engineers and architects in early-stage design can be incredibly difficult because we often don't share the same criteria for evaluating design decisions. While many engineers are starting to express requirements in more user-centered ways, for example through the use of agile user stories, they often have difficulty moving these beyond low-level narrative that describes the purpose of technical features.
Several publications have come out of the work of the conference, including a special issue of Design Studies (Volume 31, Issue 6, November 2010), a special issue of IEEE Software (Volume 29, Issue 1, January/February 2012), and a forthcoming book on studying professional software design. Because of our participation in the conference, the researchers encouraged us to write an article based on our experiences in software design, for the special issue of IEEE Software. Our article, Strategies for Early-Stage Collaborative Design, (Ania Dilmaghani and Jim Dibble, "Strategies for Early-Stage Collaborative Design," IEEE Software, vol. 29, no. 1, pp. 39-45, Jan./Feb. 2012, doi:10.1109/MS.2011.124), offers a set of ten ground rules for conducting collaborative design sessions. In our article, we encourage engineers and architects to develop the "soft skills" of design, and to evaluate design solutions against user needs and scenarios. These ground rules, based on our experience and our learnings from the conference, will be helpful to anyone running a collaborative design session, whether UX designers or software architects.
The experience has us thinking about early collaboration outside the research laboratory. UX designers and software architects each bring unique understandings to problem-solving. Depending on how an organization structures design work, these differing perspectives can be either complementary or conflictual. We're familiar with how it works in our business, and curious about how it works in yours. How does early-stage design work in your organization? Do your interaction designers and architects work collaboratively from the outset to understand the problem? How do you establish a feedback loop to bring understandings from each design approach back to the overall solution?