Designing with Data: Improving the User Experience with A/B Testing

Designing with Data: Improving the User Experience with A/B Testing
By Rochelle King, Elizabeth F Churchill, Caitlin Tan

List Price: $39.99
Price: $22.96 Details

Availability: Usually ships in 24 hours
Ships from and sold by

53 new or used available from $15.01

Average customer review:
(4 customer reviews)

Product Description

On the surface, design practices and data science may not seem like obvious partners. But these disciplines actually work toward the same goal, helping designers and product managers understand users so they can craft elegant digital experiences. While data can enhance design, design can bring deeper meaning to data.

This practical guide shows you how to conduct data-driven A/B testing for making design decisions on everything from small tweaks to large-scale UX concepts. Complete with real-world examples, this book shows you how to make data-driven design part of your product design workflow.

  • Understand the relationship between data, business, and design
  • Get a firm grounding in data, data types, and components of A/B testing
  • Use an experimentation framework to define opportunities, formulate hypotheses, and test different options
  • Create hypotheses that connect to key metrics and business goals
  • Design proposed solutions for hypotheses that are most promising
  • Interpret the results of an A/B test and determine your next move

Product Details

  • Amazon Sales Rank: #88998 in Books
  • Brand: O Reilly Media
  • Published on: 2017-04-20
  • Original language: English
  • Number of items: 1
  • Dimensions: 9.00" h x .70" w x 5.90" l, .0 pounds
  • Binding: Paperback
  • 370 pages


  • O Reilly Media

Editorial Reviews

About the Author

Rochelle King is Global VP of Design and User Experience at Spotify where she is responsible for the teams that oversee user research and craft the product experience at Spotify. Prior to Spotify, Rochelle was VP of User Experience and Product Services at Netflix. where she managed the Design, Enhanced Content, Content Marketing and Localization teams at Netflix. Collectively, these groups were responsible for the UI, layout, meta-data (editorial and visual assets) and presentation of the Netflix service internationally across all platforms. Rochelle has over 14 years of experience working on consumer facing products. @rochelleking

Dr Elizabeth Churchill is a Director of User Experience at Google. Her work focuses on the connected ecosystems of the Social Web and Internet of Things.

For 2 decades, Elizabeth has been a research leader at well-known corporate R&D organizations including Fuji Xerox’s research lab in Silicon Valley (FXPAL), the Palo Alto Research Center (PARC), eBay Research Labs in San Jose, and Yahoo! in Santa Clara, California.

Elizabeth is an ACM Distinguished Scientist and Speaker. She is also a member of the ACM's SIGCHI Academy, in recognition of her contributions to the field of Human Computer Interaction. She served in the ACM SIGCHI Executive Committee for 8 years, 6 years of those as Executive Vice President, and is the current Secretary/Treasurer of the ACM.

Elizabeth has contributed groundbreaking research in a number of areas, publishing over 100 peer reviewed articles, co-editing 5 books in HCI related fields, contributing as a regular columnist for the Association of Computing Machinery's (ACM) interactions magazine since 2008, and publishing an academic textbook, Foundations for Designing User Centered Systems. Her upcoming book, Designing with Data, will be published in 2016. She has also launched successful products, and has more than 50 patents granted or pending.

Customer Reviews

Most helpful customer reviews

1 of 1 people found the following review helpful.
5The old is new again.
By Jerry Saperstein
I cut my teeth on mail-order marketing, what they now call direct-response. Might even be called something else now.

One of the cardinal rules of mail-order marketing was – and remains – test, test, test.

You tested everything. The headline, the copy, the color of the paper, its weight, everything until you had tested enough determine the most efficient marketing package.

This book’s primary authors are eminently qualified and highly experienced. Even better, they are graceful writers.

The authors define “A/B testing [as] a methodology to compare two or more versions of an experience to see which one performs the best relative to some objective measure”. In other words, you test to find out what works best.

The book is intended to acquaint designers and product managers in launching digital products using data to guide the product’s refinement. In other words, the book how to show designers and product managers how to use the wealth of data available to better market their product.

Over the course of the first six chapters, they do precisely that. This stuff is really good. The authors, one with Spotify in her background, the other with Netflix, truly understand the concept, mechanics and worth of testing.

The last two chapters smelled too much like political correctness for my taste and, in my opinion could have been left out without harming the value of the book.

If you are not thoroughly experienced with the concept of A/B testing in marketing vehicles, you will benefit from this book.


0 of 0 people found the following review helpful.
4Improve the user experience and the product and save money, touches on Six Sigma principals, but has it's own spin on the detail
By Courtland J. Carpenter
The book could be more adaptable to various industry, but the work they do discuss reminds me of the test design and data used from a Six Sigma course I took from Purdue a few years back. You can design your process by creating process or experiments that show how you can improve the overall production. One of my professors described his best finder of flaws with walking the plant floor to discover where the process fell down and then when some of those flaws are fixed you stop getting outliers data and help to streamline the process.

While this is not exactly what the book is about, it's not totally Six Sigma, it does touch on the same things just from a more data centered way. What is does do is give you ideas how to design those experiments with the kind of data you use and then how to read the results. Testing itself is very data centered. I work as an embedded software tester, and the design starts with customer requirements, that turn into system requirements which include both hardware and software, then become software requirements for the testing I do from which I create software test verification requirements. Data becomes closer and closer the more you get removed from the customer requirements as a central part. It often becomes difficult going that path to get a good design.

Here's an example from my experience. A new type of emission control system was being put on all the trucks my former company produced. They had a new multiplexed line of Line Haul or what you may call Semi trucks coming out. They had a couple years to design the new emissions control system, but do to the pipeline of not going from the data needed to control the request system, the system requirements detailed a set of two state diagrams that described the function that were horrible. One had 13 state and over 30 transitions the other about 11 states and over 25 transitions, it was implemented correctly in software, but the operation was klugy at best. Engine variants made it worse and the vehicle launch had to be put back because of the failures. The test group and system group working from the data central side of things designed two state diagrams one with 4 states and 5 transitions, the other with 3 states and 4 transitions, and it did the same thing, but worked nearly flawless. It took six weeks working 7 days a week to re-implement the software, and the loss of revenue, and poor launch probably cost millions. Which shows one of the reasons to consider the methods this book supports. Not perfect, but a good start, recommended.

0 of 0 people found the following review helpful.
5Solid addition to the O'Reilly lineup, excellent book on data-driven design
By Larson
I've always been pleased with O'Reilly books, and this one is no exception.

A/B testing is kind of the gold standard when it comes to data-driven marketing, and this book does a pretty solid job of explaining it both from a conceptual level and a practical level.

Obviously, your testing will depend heavily on what you're hoping to measure, and this book has plenty of thorough examples of A/B implementations.

From a technical level, this is written for someone who is at least passing familiar with UI design and UX principles, but doesn't assume the reader is an expert. Like many O'Reilly books, the author does a good job of covering things in sufficient detail so that novices won't feel lost, but assumes that the reader is an intelligent person who will understand without belaboring things too much.

See all 4 customer reviews...

Connect with defaultLogic
What We've Done
Led Digital Marketing Efforts of Top 500 e-Retailers.
Worked with Top Brands at Leading Agencies.
Successfully Managed Over $50 million in Digital Ad Spend.
Developed Strategies and Processes that Enabled Brands to Grow During an Economic Downturn.
Taught Advanced Internet Marketing Strategies at the graduate level.

Manage research, learning and skills at defaultLogic. Create an account using LinkedIn or facebook to manage and organize your IT knowledge. defaultLogic works like a shopping cart for information -- helping you to save, discuss and share.

  Contact Us