Be the Villain

Inclusive Design is the practice of making products and services accessible to, and usable by as many people as reasonably possible without the need for specialized accommodations. The practice was popularized by author and User Experience Design Director Kat Holmes. If getting you to discover her work is the only thing this article succeeds in doing then I’ll consider it a success.

As a framework for creating resilient solutions to problems, Inclusive Design is incredible. However, the aimless idealistic aspirations many of its newer practitioners default to can oftentimes run into trouble. Without outlining concrete, actionable outcomes that are then vetted by the people you intend to serve, there is the potential to do more harm than good.

When designing, you take a user flow and make sure it can’t be broken. Ensuring that if something is removed, it can be restored. Or that something editable can also be updated at a later date—you know, that kind of thing. What we want to do is avoid surprises. Much like a water slide with a section of pipe missing, a broken flow forcibly ejects a user, to great surprise and frustration. Interactions within a user flow also have to be small enough to be self-contained, so as to avoid creating a none pizza with left beef scenario.

Lately, I’ve been thinking about how to expand on this practice. Watertight user flows make for a great immediate experience, but it’s all too easy to miss the forest for the trees when you’re a product designer focused on cranking out features.

What I’m concerned about is while to trying to envision how a user flow could be broken, you also think about how it could be subverted. In addition to preventing the removal of a section of water slide, you also keep someone from mugging the user when they shoot out the end.

If you pay attention, you’ll start to notice this subversion with increasing frequency:

• Domestic abusers using internet-controlled devices to spy on and control their partner.
• Zealots tanking a business’ rating on Google because its owners spoke out against unchecked gun violence.
• Forcing people to choose between TV or stalking because the messaging center portion of a cable provider’s entertainment package lacks muting or blocking features.
• White supremacists tricking celebrities into endorsing anti-Semitic conspiracy theories.
• Facebook repeatedly allowing housing, credit, and employment advertisers to discriminate against users by their race, ability, and religion.
• White supremacists also using a video game chat service as a recruiting tool.
• The unchecked harassment of minors on Instagram.
• Swatting.

If I were to guess why we haven’t heard more about this problem, I’d say that optimistically, people have settled out of court. Pessimistically, it’s most likely because we ignore, dismiss, downplay, and suppress those who try to bring it to our attention.

Subverted design isn’t the practice of employing Dark Patterns to achieve your business goals. If you are not familiar with the term, Dark Patterns are the use of cheap user interface tricks and psychological manipulation to get users to act against their own best interests. User Experience consultant Chris Nodder wrote Evil By Design, a fantastic book that unpacks how to detect and think about them, if you’re interested in this kind of thing

Subverted design also isn’t beholden design, or simple lack of attention. This phenomenon isn’t even necessarily premeditated. I think it arises from naïve (or willfully ignorant) design decisions being executed at a historically unprecedented pace and scale. These decisions are then preyed on by the shrewd and opportunistic, used to control and inflict harm on the undeserving. Have system, will game.

This is worth discussing. As the field of design continues to industrialize empathy, it also continues to ignore the very established practice of threat modeling. Most times, framing user experience in terms of how to best funnel people into a service comes with an implicit agreement that the larger system that necessitates the service is worth supporting.

To achieve success in the eyes of their superiors, designers may turn to emotional empathy exercises. By projecting themselves into the perceived surface-level experiences of others, they play-act at understanding how to nudge their targeted demographics into a conversion funnel. This roleplaying exercise has the effect of scoping concerns to the immediate, while simultaneously reinforcing the idea of engagement at all cost within the identified demographic.

The thing is, pure engagement leaves the door wide open for bad actors. Even within the scope of a limited population, the assumption that everyone entering into the funnel is acting with good intentions is a poor one. Security researchers, network administrators, and other professionals who practice threat modeling understand that the opposite is true. By preventing everyone save for well-intentioned users from operating a system within the parameters you set for them, you intentionally limit the scope of abuse that can be enacted.

Don’t get me wrong: being able to escort as many users as you can to the happy path is a foundational skill. But we should also be having uncomfortable conversations about why something unthinkable may in fact not be.

They’re not going to be fun conversations. It’s not going to be easy convincing others that these aren’t paranoid delusions best tucked out of sight in the darkest, dustiest corner of the backlog. Realistically, talking about it may even harm your career.

But consider the alternative. The controlled environment of the hypothetical allows us to explore these issues without propagating harm. Better to be viewed as the office’s resident villain than to have to live with something like this:

If the past few years have taught us anything, it’s that the choices we make—or avoid making—have consequences. Design has been doing a lot of growing up as of late, including waking up to the idea that technology isn’t neutral.

You’re going to have to start thinking the way a monster does—if you can imagine it, chances are someone else can as well. To get into this kind of mindset, inverting the Inclusive Design Principles is a good place to start:

• Providing a comparable experience becomes forcing a single path.
• Considering situation becomes ignoring circumstance.
• Being consistent becomes acting capriciously.
• Giving control becomes removing autonomy.
• Offering choice becomes limiting options.
• Prioritizing content becomes obfuscating purpose.
• Adding value becomes filling with gibberish.

Combined, these inverted principles start to paint a picture of something we’re all familiar with: a half-baked, unscrupulous service that will jump at the chance to take advantage of you. This environment is also a perfect breeding ground for spawning bad actors.

These kinds of services limit you in the ways you can interact with them. They kick you out or lock you in if you don’t meet their unnamed criteria. They force you to parse layout, prices, and policies that change without notification or justification. Their controls operate in ways that are unexpected and may shift throughout the experience. Their terms are dictated to you, gaslighting you to extract profit. Heaps of jargon and flashy, unnecessary features are showered on you to distract from larger structural and conceptual flaws.

So, how else can we go about preventing subverted design? Marli Mesibov, Content Strategist and Managing Editor of UX Booth, wrote a brilliant article about how to use Dark Patterns for good—perhaps the most important takeaway being admitting you have a problem in the first place.

Another exercise is asking the question, “What is the evil version of this feature?” Ask it during the ideation phase. Ask it as part of acceptance criteria. Heck, ask it over lunch. I honestly don’t care when, so long as the question is actually raised.

In keeping with the spirit of this article, we can also expand on this line of thinking. Author, scientist, feminist, and pacifist Ursula Franklin urges us to ask, “Whose benefits? Whose risks?” instead of “What benefits? What risks?” in her talk, When the Seven Deadly Sins Became the Seven Cardinal Virtues. Inspired by the talk, Ethan Marcotte discusses how this relates to the web platform in his powerful post, Seven into seven.

Few things in this world are intrinsically altruistic or good—it’s just the nature of the beast. However, that doesn’t mean we have to stand idly by when harm is created. If we can add terms like “anti-pattern” to our professional vocabulary, we can certainly also incorporate phrases like “abuser flow.”

Design finally got a seat at the table. We should use this newfound privilege wisely. Listen to women. Listen to minorities, listen to immigrants, the unhoused, the less economically advantaged, and the less technologically-literate. Listen to the underrepresented and the underprivileged.

Subverted design is a huge problem, likely one that will never completely go away. However, the more of us who put the hard work into being the villain, the more we can lessen the scope of its impact.