Prototyping Smart Mirror — UX Case Study
Conceptualizing, designing & evaluating an interactive IOT device
Introduction
In our ‘Prototyping Interactive Systems’ class, we were asked to design an interactive physical prototype around the Internet of Things, which can be either tangible user interface or ambient information display. We had 2 weeks to conceptualize, build & evaluate the prototype.
NOTE: Though research is an quintessential phase in every design process, the focus here was more on learning + using a variety of the sketching and prototyping techniques from ideation to the implementation of a system/experience.
Problem Statement
How can we make an ordinary object into a personalized interactive device which displays electronic information in a real-world environment in a subtle, non-distracting and fun way, all at one place?
Why Smart Mirror?
While brushing teeth, admiring our beautiful visage in the morning, or whether we are getting ready for work and even for a night out with friends, a mirror is an important object which we use everyday. With the IOT devices being more and more ubiquitous, the modern smart home needs a hub — so why not make a mirror into a personal assistant which can show more than just your reflection.
Exploring Ideas & Sketching
We started by brainstorming ideas and concepts with low-fidelity sketches of smart mirror concentrated on several aspects: Customized music or news articles after facial and emotional recognition; the means of interaction with the mirror, for example, physically tapping the mirror. The first sketch shows how users would interact with the mirror, context of usage, and several key features such as facial recognition and a map showing routes.
For the second and third sketches, we brainstormed ideas as a group to specify on what kind of information should be displayed on the mirror. Our ideas included customized messages such as notifications that were generated after facial recognition; news and music feed based on emotion recognition; sleep analysis of the night before based on the connected devices such as waistband or a health app.
Version 1.0
We built the first version of prototype after the brainstorming sessions. We decided that the users would generally interact with the mirror by voice and not touchscreen as they would not likes fingerprint marks on their mirror. The user journey starts with a facial recognition by the camera on the upper side of the mirror. Then a customized sentence such as “Good morning, John” appears on the screen. Then a system message such as “What can I help you today?” appears to guide the users to interact with the mirror. Lastly, the users can perform actions such as opening an app, calling someone and receiving and sending a voice message.
1. Facial Recognition:
2. Voice interaction:
3. Making phone calls
4. Receiving and sending voice message
Key findings from initial critiques and walkthroughs
After initial critics and walkthroughs, we made several changes to the interfaces and decided to use gestures instead of voice as major interaction. Some people may be reluctant to use voice because they do not want to disturb other people nearby or they find it more comfortable to directly manipulate their devices. We also considered accessibility issue and concluded that having multiple options to interact with the device would help the product reach a broader audience, who may have weak sight or difficulty of speaking.
The changes are summarized as follows:
- Decide on gestures for users interaction such as scrolling the pages, opening the menu, and launching an app. The main gestures are displayed in the image below:
- Change real time weather forecast and add “to do list”, making them visible on each page.
- Adding menu icon in the main page and change the icon of sleep analysis as it was not clear what it means.
- Minor changes on the layouts to make it look more clean, compact and aesthetically pleasing.
Significant Changes From Earlier Rounds
Since we didn’t have time to iterate on the physical device, most of our significant changes came from the digital prototype that we designed using Sketch and Flinto. Here is a shot of the before/after of our prototype:
The new version of the interfaces are shown below:
Implementation
We found some YouTube videos of people who had already created similar products and went through similar steps. Basically, we needed a monitor and a two-way glass mirror to create our device. In the video we found, the person used wood to cover the monitor, but we opted for a less expensive alternative and used foam instead. Here is a photo of what we used:
Once the mirror and monitor were crafted into a single device, we needed a digital prototype to put on the screen. We made our prototype using Sketch for UI design and Flinto for motion design. Once the prototype was ready, we plugged our computer into the monitor and had a mirror through which you could see a screen as well.
Final Prototype
Evaluation & Testing
After analyzing basic use cases of the Smart Mirror, it came to our attention that further analysis and a test with real users is crucial for improving the overall usability. After thorough preparation of test scripts, we gathered a group of few participants, who used a current version of the product. We invited these users for the testing of our product. During the private sessions, we recorded their interaction with the product while they were performing typical tasks. In addition to evaluation with participants, we also conducted heuristic evaluation inside the group. We used Jakob Nielsen’s “10 Usability Heuristics for User Interface Design” for this evaluation.
Key Takeaways
- First of all, we learned that our participants preferred voice commands over gesture in controlling the mirror. But the participants also reported that it was fun to use gesture and they would feel comfortable with using the gesture once they learn it. Therefore, now we can be more confident with our choice of keeping both gesture and voice as means of interaction.
- From the testing, we learned that our product needs a tutorial that explains each gesture or voice keyword and meaning of icons as a part of on-boarding process. As most of use were Mac users, we did not take into account the possibility that the users would not know what each icon meant.
- Furthermore, we also discovered that some participants felt there were too much information in the main screen. This feedback was also valuable as we did not realize the layout could be overwhelming for some users.
Lessons learned & Conclusion
Throughout our design process, we could discover several problems we had not realized in the earlier stages. For example, we originally considered touch-based interface, but we realized the users would not like the mirror to be covered with fingerprints. We also dropped the idea of emotion recognition system because it seemed most users would be always feeling tired in the morning, which leads to same cheer-up messages and music from the system that can be annoying after some time. We learned that taking a second look at the ideas is important, as we did not see any problems with those ideas when we first thought of them. We also benefited greatly from discussing ideas with each other, as it ensured not only more ideas flowed out but also a person who could point out problems that were usually not seen by other people.
In future iterations, we would like to implement additional features we could not include due to time constraints, such as writing — or drawing — a memo on screen, selection of news headlines according to personal preferences, weather and time information in different time zones, detecting facial expressions, makeup tutorial, health statistics (connected with wearable device), and AI technology applied to the mirror for more personalization.
Thanks to the amazing team: Kenzo Nawa, In Kwon Choi & Sisi Jin
