Hala, 🤩

Okay, if you did not read “Design Introduction — We Are Designers By Nature! ” Read it and back again.

As a reminder, you are reading this article as a part of the 4I’s series:

1. Whatever The Approach, It’s a 4I’s!
2. The 4I’s – Introduction to Inspiration.
3. The 4I’s – Introduction to Ideation.
4. The 4I’s – Introduction to Implementation.
5. The 4I’s – Introduction to Iteration.

It’s all about validating our prototypes. during the iteration phase, we iterate a loop which starts with user-testing and ends with updates. Till we figure out and fix all the problems and usability issues. it’s important for you to understand that Changes is not about the art direction. but, it’s more about the experience – Ease of use, Learnability, Pleasantness, Accessibility, – and in some cases, we should consider “Safety, Security” as well. through the article, we will discuss the methods, principles, and we will spot the light on the best practices as well.

Your experience will not deliver the right solution, and if it did, that means you did not measure/test the experience accurate or right way. Many of new technologies and unusual ideas might be less usable from UX Principles point of view, however, in many cases, it could be much better when we test it in action.

Iteration Methods

Mainly, there are two different types of iterations. The first one relays on the feedback come from real users through the usability testing sessions. the second one could be measured and decided through analytics for the remote user behavior by numbers. next, we will discuss how the usability testing, and other helpful tools we could use to analyze and measure the remote user behavior. and at some point, we will clarify important points about the A/B Testing.

Usability Testing

When you put your product in real users hands and you start observing them completing a specific task to discover the problems and experience confusion, That’s what we call usability testing. through usability testing, we measure lots of different aspects of design & behavior, such as the awareness of context, navigational level, colors and visual, placement and accessibility, time and many more depends on the product itself.

• There are two main types of usability testings, the first is Explorative, that happens when we have a brand new product. the second one is Comparative and it happens when we have a new version of an old product, and we wanna measure which one is better (A/B Testing). here we will talk mainly about the explorative usability testing, then we can cover The Comparative one.

Now, we will dive in depth of these points:

• Planning a Usability Testing.
• Measuring The Usability Testing.
• Notes on Usability Testing.

Planning a Usability Test

Simply we need to define the scoop, equipment, team members, participants, and the matrix. let’s break it down:

• Scoop: Why we are running the usability test, be more specific and define which part we will cover such as navigation, specific feature or even task.
• Equipment: Such as recording camera, notes, etc.. includes the location and the physical setup which must be available during the usability testing time.
• Team members: and their different roles, who will moderate and run the test, who will handle recording and note-taking etc.
• Subjective Metrics: Such as the questions you will ask before, after, within completing a specific task.
• Quantitative Metrics: The Quantitative data you will use to measure your test such as successful completion rates, error rates, etc.

Measuring The Usability Testing

In a large-scale product, I don’t believe that Error rates, completion rates will be enough to measure the usability of your experience. you need to dig in depth of your user mind, background, and cognitive load in each second. Think about the usability test as a formative and a summative, simply a usability evaluation from a user in person point of view considering all the possible personal basis.

There are several metrics that you may want to collect during the course of testing.

• Successful Task Completion: The scenario is successfully completed when the participant indicates they have found the answer or completed the task goal.
• Critical Errors: Essentially the participant will not be able to finish the task. A participant may or may not be aware that the task goal is incorrect or incomplete.
• Non-Critical Errors: Non-critical errors are errors that are recovered by the participant and do not result in the participant’s ability to successfully complete the task.
• Error-Free Rate:  Error-free rate is the percentage of test participants who complete the task without any errors (critical or non-critical errors).
• Time On Task:  The amount of time it takes the participant to complete the task.
• Subjective Measures:  These evaluations are self-reported participant ratings for satisfaction, ease of use, ease of finding information, etc where participants rate the measure on a 5 to 7-point Likert scale.
• Likes, Dislikes, and Recommendations:  Participants provide what they liked most about the site, what they liked least about the site and recommendations for improving the site.

These Links will help,

• Formative and Summative Usability Testing / UX.StackExchange
• Usability Evaluation Methods – usabilitybok.org
• Planning a Usability Test – usability.gov
• Usability Metrics – nngroup.com
• 3 Metrics to Measure and Quantify Usability
• 10 ESSENTIAL USABILITY METRICS
• 8 Ways To Measure Satisfaction (and Improve UX)
• Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated?
• Google for more!

Notes on Usability Testing

• #1 Good design qualifications and principles changes based on the nature of the project. in case it could be (efficiency, effectivity, learnability), in another it could be more about safety and security, there are many different cases and qualifications and you should care about specific qualifications and principles from the very beginning, to save your time and keeps yourself focused on a certain area.
• #2 It is not a test for your skills as a designer. it’s a test for your opinions which might be right or wrong. no shame.
• #3 Based on the project nature and age, care about the learnability and recalling.
• #4 Satisfaction, will never stand alone. roughly, you can’t decide a change based on a single measurement role especially this one.
• #5 Durability. Many of UX and UI had a short time, edge technologies should be tested very well before implementation.
• #6 Most of the time, users don’t have any idea about what they really need. you should figure it out!
• #7 Pleasantness, yup UX is not enough. Build an enjoyable UI.

Now I recommend if you read these links as well,

• Learnability: 5 Principles of Interaction Design To Supercharge Your UI (3 of 5)
• MIT – Reading 2: Learnability, Part 1
• Make it Easy on the User: Designing for Discoverability within Mobile Apps
• MIT – On Learnability, Complexity, and Stability
• 5 important things to think of when conducting usability testing of voice interaction using voice-controlled assistants.
• 4 forgotten principles of usability testing
• Tests Go First. Usability Testing in Design.
• Creating usability test tasks that really motivate users
• Booking.com guerilla usability testing
• Guerrilla Usability Testing the New Experiences Feature on Airbnb’s iOS App
• Never stop reading, Google for more!

Best Of User Tracking Technologies

For a reason or another, in many cases, you may don’t have the option of meeting user in person. that might leads you to rely on remote usability testing methods only. of course, it’s not the best scenario, but you should be able to deal with such one.
in best cases, you will be able to validate the user testing results by remote tracking tools results as well to clarify your inputs which will lead for a specific decision, output.

Record almost everything

If you have the ability to record and track almost everything happening during the test, do it. Using technologies such as eye-tracking will help you understand the scan behavior of the user, which clarify and validate your visual hierarchy. The Tracking the mouse movements in many cases will help you figure the effort and time user needs to complete a specific task. or even the micro-interactions.

Heatmaps

It not only helps a lot when you need to measure the real scrolling and scanning behavioral, but it makes you able to test the emphasization for the different Interface components and what users are looking for and interested in.

Analytics?

For example google and similar tools, it helps you define where users usually stuck (the exit point). and how long time he takes to decide a specific decision. Now it’s time to cover a very important situation where we have an old and new version of the same product and how to measure it.

A/B Testing

When you have two different versions of a design (old, new) and you want to decide which version is better for your users, that’s what we call A/B Testing.

In web analytics, A/B testing (bucket tests or split-run testing) is a randomized experiment with two variants, A and B. It includes an application of statistical hypothesis testing or “two-sample hypothesis testing” as used in the field of statistics. A/B testing is a way to compare two versions of a single variable, typically by testing a subject’s response to variant A against variant B, and determining which of the two variants is more effective. Continue reading through Wikipedia

• Google tested 41 different shades of blue.
• Amazon initially decided to launch their first personalized product recommendations based on an A/B test showing a huge revenue increase by adding that feature. (See the second paragraph in the introduction.)
• LinkedIn tested whether to use the top slot on a user’s stream for top news articles or an encouragement to add more contacts. (See the first paragraph in “A/B testing with view based JSON” section.)
• Amazon determined that every 100ms increase in page load time decreased sales by 1%. (In “Secondary metrics” section on the last page)
• Google ’s latency results showed a similar impact for a 100ms delay.
• Kayak tested whether notifying users that their payment was encrypted would make users more or less likely to complete the payment.
• Khan Academy tests changes like letting students know how many other students are working on the exercise with them, or making it easier for students to fast-forward past skills they already have. (See the question “What is the most interesting A/B test you’ve seen so far?”)

There are few things you should know before you go, let’s discuss it on the next few lines.

Customer Funnel

The idea is that users usually lands on the Home page, then they explorer website deeper – example – the user flow might look like: homepage > category page > product page and so on – Visually example is here :

The funnel in the real world is not that easy, there are many things happens before the user will move from a location to another, sometimes before even scroll the page itself!

Click Through Rate (CTR)

This is one of metric which can be used to major an A/B Testing, simple it’s if we have for example 10 pages view, and 2 clicks on the call to action button so the CTR will be (10/2).

Click Through Probability (CTP)

It’s very similar for CTR in its concept, but it major actually the ratio between unique visits and clicks, so for example if you have two users one of them did not click through a CTA button, and another one clicked 4 times, the CTP will be 1/2 = 2.5 which means Unique visitors who click/ unique visitors to the page. The number of clicks here won’t matter, what matters is numbers of users who click.

It would avoid us lots of technical issues like, what if the users click many times but the button is not working, what if the page load time is not so fast so he visited it many times, and many other cases.

Binomial Distribution:

It is used to model the probability of obtaining one of two outcomes, a certain number of times (k), out of a fixed number of trials (N) of a discrete random event. A binomial distribution has only two outcomes: the expected outcome is called a success and any other outcome is a failure.

The binomial distribution formula is:

b(x; n, P) = nCx * Px * (1 – P)n – x

Where:

b = binomial probability

x = total number of “successes” (pass or fail, heads or tails etc.)

P = probability of a success on an individual trial

n = number of trials

If you can google it, just keep its title & concept in your mind. For more information, check this link.

Without going in details, we can use lots of tools to visualize binomial distribution, like tableau and here is a visual example

Also, you should take a look at the Confidence interval, this link actually contains fun samples.

Null hypothesis

(in a statistical test) the hypothesis that there is no significant difference between specified populations, any observed difference being due to sampling or experimental error.

For the Math side, I would like you if check this Article, it will let you know how it will be implemented in A/B Testing with examples.

This article also is really interesting, How to Write a Solid A/B Test Hypothesis – Optimizely blog contains lots of good ideas. There are many interesting programs and courses for A/B Testing, but I really enjoyed this one which comes for free from Udacity.

What’s Next?

4I’s Draft 0.1 – Wrap Up

The 4I’s is not a new concept or idea, it is just a way to clarify design in general simple way. It’s just a draft version of what we are preparing right now to be published officially on The 4I’s labs by Moha.Studio.

Many of the content of the article is not something new, yeah it is right, we are not inventing the wheel! if you have any improvements or fixes feel free to share it with me through my different social channels. and sure, Feel free to visit The 4I’s labs now!

Salamat 🌹