Using Social Media For User Testing

I am Monika, one half UX researcher, quarter marketeer and quarter designer at CanvasFlip. We are a small team but each one is superfluous with ideas all the time — I haven’t figured it out yet, is it due to the brilliant in-house coffee or their passion for the product? 😉 I hope it’s the latter! Every time my team dreams of a new idea or researches for a new feature to enhance the product, I like to test it out before I go to the product team with the idea. In the beginning of my job, I was hesitant in reaching out to almost the same set of users almost twice a month. It’s annoying. Isn’t it? Having said that, I did not give up on testing before proposing, but I found a better way to do it! 😎

For most digital products, social media is a dominant form of interaction with our users. Same is the case with my company. Leading platforms such as Facebook and Twitter count hundreds of millions of users each month. Why not harness this channel to test your concepts and hypothesis? In this article I will talk about how I used social media to collect data for user research.

How to do user testing on social media?

1. Define relevant hashtags and user accounts for your test

Hashtags are definitely more than just drawing attention — it’s about ease in finding, following, and contributing to a conversation/cause. Therefore, in order to get your post found and contributed to, you need to set up the search query (hashtag), or queries, that are going to be used.

Identify at least 10 hashtags that are closely related to the feature or product you are testing. For example, when I tested for one of the ideas (UX Inspire), few hashtags I used were — #UXflows, #usersessions #uservideos #UXanalytics etc..

You can also tag pages on Twitter that have followers aligned to your persona.

2. Shortlist the social media channels that work best for the company

When it comes to choosing which social media platforms for user research, select those that offer the best potential for reaching your ideal audience and broadcast the tone you’ve decided is best suited for your company.

Every social media works differently for every company. Some might be great for getting leads for the business, while others can be great for user engagement. Closed channels on Slack, Facebook groups, hangout groups etc are also great places to share your prototype for feedback.

A Social media metrics might be really helpful in shortlisting the channels. This chart is just a framework, you can modify it as per parameters you want to consider —

For most of the features in testing, I use few Slack groups, Facebook groups, LinkedIn closed groups and sometimes Twitter too.

3. Share the magic URL

(What is a magic URL?)

You can share the prototype with reviewers(/users) on social media by providing them a social URL (also termed as magic URL) of the prototype. It is self generated in the CanvasFlip tool. Grab the link and paste it on the social media channels you shortlisted. You might want to add a line to your post/tweet telling your reviewers what they have to do.

The ground rule of social media still stays intact — make the post as interesting as possible, so that you get maximum reviewers for your prototype.

Anyone who reviews the prototype from the magic URL, has his session recorded in CanvasFlip. Other UX insights such as heat maps, conversion funnels, time taken for a task etc are also recorded.

4. Analyse the UX data

Analysis is the process of breaking down information into smaller pieces, and examining what that information means. Without analysis, all of the information gathered in research might be measured, but it has no meaning. Analysis of information allows us to make connections. For example, you might research how many people successfully complete a booking process, and measure the number of people who you could complete a task in estimated time. Analysis is then used to provide context and answer essential questions.

By sharing the magic URL, here are few of the insights you get automatically generated — User video of each reviewer, time spent on each design screen, average time taken for a UX flow, number of interactions required for a task flow, engagement on the design from the heat map, drop-off numbers, error rate etc.

A Case study of “UX Inspire”

(Still in BETA)

Inspire by CanvasFlip was built as a collection of inspirational UX flows across all categories. It has prototypes of all famous apps and also the ones that inspire most designers.

Quite recently, someone came up with a brilliant idea (We named it UX Inspire) — Since we are into user testing of prototypes, why not let visitors on Inspire watch their own sessions after they try the prototype! Of course the raw idea has undergone multiple modifications after rounds of user test, and what will be live is hopefully magical for users :) I executed this entire user test on social media and slack groups.

Here’s the best part about testing on social media —

1. We got a large sample sized data, after which you can always funnel down the relevant ones. Believe me, I got 122 (and still counting) user videos on the prototype I shared!
2. Not just user videos and heat maps, we also got many comments on the prototype pointing to elements and flows.
3. We tested with paper prototypes. And this turned out to be really helpful for us. Because many users wrote back to us with suggestions and additions. That’s the beauty of testing with paper prototypes! People know that there is scope for additions and modifications. Also, a sense that they are an important part of the making process.

I’m extremely happy with the feedback and insights I got from the user testing on social media. I’m sure it will benefit you as well. :)

Would you please try out the UX Inspire prototype as well? Thanks. 🙏🏻

When to use this technique (and when not to)?

Having done this for quite some projects now, I have an estimate of when social media works best for user research, and when it kind off backfires…The technique of doing user research with social media is particularly well suited to certain project situations:

• when there is a wide focus on a particular service, such as a discovery or requirements-gathering phase;
• when there is no budget for recruitment or a venue, or no time to wait for the recruitment to be organized;
• when you need to do some design research to understand the context of use;
• when you have run out of ideas of what to do next.

Also, it is not so well suited to other common project situations:

• when you need to test a prototype that is not publicly available;
• when there is a highly focused brief (or worse solution) that is not up for debate;
• when you are fixing something clearly broken, such as a siloed information architecture;
• when you are optimising something that works OK for users, such as a shopping basket and checkout.

Conclusion

Social media provides a rich source of data for user researchers. It allows researchers to tap into the recent experience of people without the formality of interviewing or user testing. And while it is not without its disadvantages, it is illuminating, and you can get started for free. It helped UX Inspire by CanvasFlip realise the importance of displaying cumulative data as well. So, why not add user research with social media to your toolbox and see what you find? I’m sure it would be quite interesting :)