Ad Experience

ux research
Goal

To understand what a good online shopping ad experience is, and how it affects users’ attitudes and behaviors.


Team
Jeanny Xu
Maggie Chen
Prasad Gaikwad
Nathan Khuu


My Roles & Methods Used
︎ Secondary Research
︎ Card Sort / Reaction Cards Owner
︎ Diary Studies
︎ Interviews




Our team worked with an industry client to conduct foundational/exploratory research on ad experiences. Our client was interested in understanding what makes up good online experiences at a high level, and we narrowed down the scope to online shopping ads. We employed qualitative UX research methods including diary studies, card sort (reaction cards), and interviews to understand the components that define good ad experiences and how they affect users’ attitudes and behaviors.





CONTEXT

Scope & Research Goal


We started off with the high-level questions asked by our client: What makes up a "good ad" experience in web, mobile, app environments? What do these ads lead users to do?

We conducted initial research on online ad experiences, and found industry research reports like Coalition for Better Ads and Google’s Ad Experience Report. They outlined forms and contents of an ad that would annoy or evoke negative emotions, and gave advice to avoid making these ads.

After discussion with our client, our team narrowed down the scope to users’ attitudes and actions. We chose shopping ads, because the a shopping ad’s experience is easier to define and measure, and they are more representative of the ads that an average user would encounter online.


What components (ad content, ad creative, ad contexts, etc.) define a good ad experience?
How do good ad experiences affect users’ attitudes and behaviors?




With a focus on users with the following profile:

  • Adult (over 18), comfortable with technology
  • Frequent online shopper - at least 3 times/week (any form of online shopping including browsing, purchasing, etc.




METHODS

Screening & Selection


We recognized that “good” is a generic term that can take on different meanings to different people, so we selected research approaches that’ll provide us with rich qualitative data, allow us to ask open-ended questions, and give participants opportunities to define what’s a “good” ad experiences to them: diary study, interviews, reaction cards.

Participants were recruited from our social circles, and those selected to participate passed a screening survey to ensure that they match our targeted user profile.



Diary Study


This method provided us rich qualitative data collected from participants in situ. It helped us understand users’ thoughts and behaviors when they encounter ads while browsing, and served as a starting point to get a grasp of how people view and interact with shopping ads.


Length of study: 5 days

Number of participants: 18 (3 dropped, 15 in effect)

Compensation: Choice of beverage (up to $5)

Procedure: Participants upload screenshots of ads that they consider a “good” ad experience, and answer related questions via Google Form on a daily basis.

Outcome: ~70 entries in total, 3 themes seem to emerge after synthesis:

  • Ad Creative: Visual appeal
    “The bright purple color definitely made the ad stand out on the website that I was on, and the simple graphics gave off a bold and playful vibe.”
  • Ad Content: Straightforward-ness
    “The caption was very catchy [...] and the demonstration was also short and sweet.”
  • Ad Context: Relevancy
    “I already have the matching palette with this and now I want the new release so I can have the collection. Honestly not even like super great ad in and of itself but it’s so ducking targeted.”



Interview


We then followed up with some of the participants from the diary studies who have provided interesting insights, and conducted a more in-depth interview to collect richer, deeper insights about their experiences with ads.

Length of study: 30 min - 1 hr each

Number of participants: 6

Procedure: Participants were asked to elaborate on their diary study entries, and discuss more broadly their experiences with online shopping ads.

Outcome: Insights around people’s preferences for visuals, company values, and creepy/relevant ads:

  • People have different preferences for ad creatives due to varying aesthetic preferences
  • People care about company values and ethical practices
  • Creepiness and relevancy come hand-in-hand, accompanying concerns about privacy



Card Sort (Reaction Cards)


Immediately following the interview, we asked participants to arrange adjectives (adopted from Microsoft Desirability Toolkit and our diary study responses) to define aspects of a good ad experience they value and prioritize.

Length of study: 5 - 10 min each

Number of participants: 6

Procedure: A list of 26 descriptive words were prepared in alphabetical order (including words that are positive, ambiguous, and negative) on Trello in an “Unsorted” pile. Participants were instructed to:

  1. Consider their current experiences with online shopping ads, and sort the words in “Unsorted” into “Yes” and “No” piles according to whether they can be used to describe their experiences.
  2. Rank top 3 words in the “Yes” pile
  3. Explain each word choice and why they were chosen
  4. Repeat Step 1-3 while considering their ideal experience with online shopping ads

Outcome:

Top adjectives for current ad experiences: Familiar, Busy, Relevant, Simple
Top adjectives for ideal ad experiences: Ethical, Trustworthy, Eye-catching




Current ad experience

Ideal ad experience





INSIGHTS

Relevant vs. creepy ads are distinguished by being shown exact tracking of past browsing history vs. new relevant information.


Personalized new information is good:
  • Likely to act on relevant ads, e.g. clicking or taking mental notes

Exact tracking of viewing history is creepy
  • Participants notice when their online viewing histories has been tracked
  • Dislike for “creepy” ads
  • Might evoke stronger reactions like avoid shopping with the company/site in the future, and using incognito for searches



Company values and ethical practices matter!


Company practices and values affect user behaviors.
Qualities like honesty and ethical values Sales events and cheap prices are effective.



People have varying preferences for visual presentations/ad creatives.


Some prefer ads that are concise and to-the-point
  • Not too busy or stressful to look at
  • E.g. flashing ads, quickly switching between multiple slides or images

Some prefer ones that are fun and enjoyable
  • Visually appealing
  • Colorful and eye-catching


RECOMMENDATIONS

Bring awareness to new items relevant to past preferences


Don’t immediately advertise the exact same product/service that were previously searched.



Make it concise and to-the-point


Highlight objective points (e.g. price, use cases, quick demos) and show honest representations of the product/service.



Explore the idea of targeting based on ad creative


No clear consensus on what a good ad looks like - why not present them based on their aesthetic preferences?





REFLECTION
︎

We were given the freedom to plan and execute a cohesive research project from beginning to end, and it was also amazing to have the opportunity to work with a very established industry client for this project. I was exposed to realistic conversations an industry practitioner would have with their clients/partners, despite this being a school project.

︎

It was an extremely challenging prompt to start with. Ad experience is a very broadly defined term, and the experience viewing an ad varies largely depending on the viewer, the platform they were on, their subjective preferences, and many other factors that are hard to predict. A “good” ad experience is even harder to define, since “good” carries different meanings under different contexts.

Our team struggled a lot at the beginning with the scope of our study. We want to meet our client’s goal of gaining foundational understanding of the subject, but we still want to present valuable, actionable insights. After extensive research on the problem space, with constant communication with our client, and a lot of hard work, I’m very proud of what our team were able to accomplish, and the entire learning experience was extremely rewarding.

Mark



designed & illustrated by maggie