An all-in-one mobile APP to help customers discover discounts and loyalty programs,
immerse in AR experiences, and order with ease
TYPE
Hackathon by Square
UI/UX Design
AR Design
TEAM
Kaya Yilin Niu, UI / UX Designer
Lingshuang Kong, Design Technologist
DATE
April - Jun 2023 (2 months)
ABOUT
In the summer of 2023, Lingshuang Kong and I participated in an exciting hackathon organized by Square. Square asked entrants to build an app or add new features to an existing app by using their APIs and other emerging technologies. We submitted ShopX as our solution for this challenge.
◉ PROJECT CONTEXT
Coupons, discounts, and loyalty points play an important role.
Online shopping platforms like Amazon, Doordash, and eBay have grown significantly, showing the rising influence of virtual shopping.
Recent data shows 40% of consumers are motivated to shop online by coupons and discounts, while 25% are driven by loyalty points.
◉ CONTEXT
However...
Customers often miss out on offers unless they visit stores in person.
◉ NARROW DOWN THE DIRECTION
How might we create a product to help consumers discover discounts, loyalty programs, and order smoothly?
◉ TECHNOLOGY EXPLORATION
AR + Geo-fencing:
Improve the whole experience
Research shows that using AR technology for product viewing significantly impacts customer behavior. AR can enhance purchase inclination. Its interactive nature provides comprehensive information, boosting customer confidence in the shopping experience.
By Geo-fencing technology, the system can send location-based notifications to the customers so they will never miss valuable deals.
◉ SOLUTION
An all-in-one, user-friendly app to help consumers discover discounts and loyalty programs, immerse in the AR experience, and order with ease.
◉ WORK PROCESS
How do we build it in a short time?
01
Ideation
• Do research on Square APIs and other technologies
• Generate ideas from our design direction
• Prioritize ideas based on feasibility and user feedback
• Define MVP version product
02
Design
Kaya Niu:
Design the UI and UX, utilize the iPhone's 3D scanning feature to generate 3D models, and create Instagram AR filters using Spark AR
03
Develop
Lingshuang Kong:
Built the backend and database, integrating Square API (Merchants, Locations, Catalog, Customers, Loyalty, Orders, Payment, and OAuth), Google Maps SDK, and Google ARCore SDK & Develop Android UI
04
Testing
Conduct usability testing with consumers and merchants to ensure ShopX meets their needs; The valuable feedback we received guided iterative improvements to both design and functionality
◉ SOLUTION
Onboarding.
◎ FEATURE 1
Discover Discounts & Loyalty Rewards with Ease.
Discover discounts and loyalty programs on the homepage. Explore merchants' product or service details and seamlessly add items to your virtual shopping cart.
Shopping made simple!
◎ FEATURE 2
Navigate with
Map & Notifications.
Easily find nearby merchants and exclusive offers with our user-friendly map. Customize your search by distance, offer type, and AR availability.
Enable notifications for deals within a one-mile radius, ensuring you never miss out on discounts and loyalty programs.
◎ FEATURE 3
Immersive in AR Shopping.
Explore ShopX's unique AR feature! Visualize products realistically, make informed decisions, and seamlessly add items to your cart directly from the AR page. Effortless and immersive shopping!
◎ FEATURE 4
Enroll in Loyalty Programs.
Easily join loyalty programs and redeem rewards to maximize savings on your orders using accumulated loyalty points. Enjoy the benefits!
◎ FEATURE 5
Spread Shopping Fun
with Social AR.
Use delightful AR filters to share on Instagram or Facebook, recommending your favorite merchants and items.
Spread the word about remarkable offers and turn sharing into an enjoyable and influential experience with ShopX!
◉ CONECEPT
Enhancing user engagement:
Integrating text with advanced hand/face recognition
After a lot of brainstorming and user feedback sessions, I finally developed two concepts to make the text more interesting and engaging on Instagram, which addresses the challenge of low user engagement by innovatively combining textual elements with state-of-the-art face and hand recognition technologies. This integration aims to elevate user participation to new heights, injecting a sense of excitement and interactivity into the user experience.
◎ CONCEPT ONE OVERVIEW
Eating Words
A text effect that helps users express their accomplishments and wishes
◎ CONCEPT TWO OVERVIEW
Floating
Letters
A text effect that helps users share their experience with others
The summer was all about learning how one of the most design-mature organizations in the world solves problems at scale. I spent my free time networking with designers, researchers, engineers, and managers from various teams to learn more about their teams and favorite parts of their jobs. This internship taught me a lot about embracing ambiguity, asking the right questions, how to ship products at a scale, and finding impact. I’ll forever be thankful to Meta, my managers, and the close friends I’ve made for changing my life. What I worked on is private, but feel free to reach out to me and we can talk further about my role at Instagram Design Team.
My first day at Meta
Office view
Me and my manager
Color-changing Instagram logo in the office
Instagram Headquarter
Team dinner with talented designers
An AR feature that helps users find suitable clothes and improves their efficiency of online shopping for Lululemon
Join our teamTYPE
Group Project
TEAM
Zhiyue Wang
Yu Cheng
DATE
Oct - Dec 2021
(10 Weeks)
TOOLS
Figma, Miro, Unity
ABOUT
This project takes Lululemon clothing shopping platform as an example to study how to improve the efficiency of online clothing shopping for users. We integrated AR technology into the clothing purchase process to solve the pain points of online clothing shopping and improve user experience.
MY ROLE
Conducted interview and observation in user research. Participated in the distribution and editing of need finding questionnaire. Designed the paper prototypes, Hi- Fi prototypes and AR prototypes. Conducted a usability testing focused on viewing the details of the product feature.
Create
If you don't know if the clothes online are right for you, how about creating "Another Self" and having him/her try it on?
Place
If you want to make the effect more realistic, how about putting "Another Self" in your real room?
Change
If you want to change the color or size, you can simply selecting the choices.
View
If you want to see the details of the product, try to approach the avatar.
BACKGROUND
Users encounter a variety of difficult problems when shopping online for clothing during the epidemic.
With the increase of rent, labor cost and the prevalence of COVID, more and more brands are encouraging users to shop online. However, when shopping online, users often encounter a variety of difficult problems. For example, the size does not fit, the material and color is unimaginable, the logistics and transportation are slow, it is not clear how to match the clothing, etc.
Past
Now
Initial Design Question
During the COVID, how can we design an online platform to improve the clothing shopping experience for people?
Interview & Observation
What are people's attitudes to clothing
shopping online?
Semi-structured Interviews
Survey
Observation
Activity examined:
1. We examined users’ online shopping experience and their attitudes towards online and offline shopping.
2. We examined the process of returning and changing products online.
3. We examined the process of how users selecting and purchasing online.
Methods:
1. Needfinding interview
• Interview tasks:
Understanding what features matter when users shopping online.
Finding out what they like and dissatisfied with the existing online shopping experience.
• Interviewee: Interviewed 6 online shopping users aged 20-30, including income and no income user groups.
2.Observe the online shopping process through users’ shared screens.
• Gave the interviewee a link to a shopping website, including Gap, Lululemon, Taobao, and a shopping website that the interviewee chose voluntarily.
• Asked them to buy a skirt or top.
• Observed their behaviors and attitude during the process.
"Clothes bought online are often returned because the size is not the right and the quality is not as expected."
“The virtual experience should be more realistic, otherwise it will backfire.”
"The pictures should be more detailed and true, otherwise you don’t even know that the clothes are transparent."
"I often use Photoshop to put myself and clothes together to see if the clothes suit me"
needfinding
Identify user needs.
- User needs a way to know if the size of the clothes is suitable for him/her.
- User needs a way to match clothes and get feedback quickly and clearly.
- User needs a way to know the size of each piece of clothing, the height and measurements of the model, and the size of the clothes on the model.
- User needs a way to let his/her friends help him/her select the clothes again after selecting online.
- User needs a way to experience a more realistic try-on virtual experience.
- User needs a way to see the details of every product clearly, like material and color.
- User needs a way to get clothing recommendations based on browsing history, and they want to see them in obvious places.
- User needs a way to see more reviews and photos from buyers to judge if the product is worth buying.
- User needs a way to try on clothing before checking.
- User needs a way to increase efficiency of fitting.
- User needs a way to get discount informations easily.
- User needs a way to check recommendation items from influencers easily.
- User needs a way to try more different clothing in a very short time.
- User needs a way to see more photos and description about the clothes.
- User needs a way to return goods simply and efficiently.
- User needs a way to invite friends when they are trying on the clothes virtually in real-time.
- User needs a way to store his/her body data and don’t need to spend too much time on selecting size.
- User needs a way to have a clear, smooth, and design-conscious online clothing shopping experience.
Affinity Diagram
Analyze our datausing an affinity diagram.
Affinity Diagram
By dividing our thoughts and interview records into categories for comparison and observation, we found that size, time, and design categories account for a large proportion. People are more concerned about the size of the products in the existing online clothing shopping platform, hope to save time, and look forward to improving the online clothing shopping experience through the new design of the platform.
INSIGHTS
What insights have we gained?
Efficiency
Users seek efficiency in online shopping, they want to find the style of clothes they like quickly, then get the information and reviews about the clothes quickly, and finally buy the exact size of clothes that fits them.
Experience
Users hope their shopping experience is enjoyable and fun. For example, some users want to shop with friends, and some users hope online shopping could be as rich and interesting as shopping in stores.
Refined Design Question (POV)
How might we improve the efficiency and experience of shopping for Lululemon users?
With the brand lululemon as our starting point, we want to investigate the problems of online clothing shopping more deeply. We hope to explore how to improve the efficiency and experience of online shopping for our users and give them a sense of pleasure and accomplishment in the process.
User Profile
Who is our target user?
Based on the data collected in all user needs, we created four types of behavior variables and mapped 12 surveyed persons to behavior variables. Subsequently, we found two clusters within the scope of behavior variables as the basis for constructing user profiles.
User profile 01
User profile 02
Join our team
Brainstorming
Main ideas
of improving the efficiency and experience.
To improve the efficiency and experience of online shopping for lululemon users, we conducted brainstorming and listed possible solutions to this problem. Finally, after discussion and voting, the following solutions were selected:
1. Use AR try-on models to display product information
2. Recommend products suitable for users’ body size through intelligent algorithms
3. Allow users to see the details and features in an more interactive way
Join our teamWe created three paperprotypes and they ask users to build a virtual model that fit themselves, place an AR model in their room and see the details of the product they choose in an interactive way.
Solution #1 (Rational choice)
Recommend products suitable for the user's body size and style to users through intelligent algorithms
One of the biggest problems users encounter when shopping at lululemon is that they don't know how to choose a size. This solution helps users enter their body size information simply and efficiently, and synthesize a virtual 3D model based on this data. If users know their specific body data, they can choose to enter the numbers directly. If they do not know, they can adjust the body shape by adjusting the slider.
Solution #2 (Most likely to work)
Use AR try-on models to display product information
This design showcases true-to-scale Lululemon products through AR fittings. The user could place virtual models in their room through an iPhone camera to simulate the product's try-on. The model could show the size, color, collocation, product details, etc., to customers more intuitively to improve shopping efficiency.
Solution #3 (Most likely to delight)
Restore offline shopping experience through AR, allowing users to see the details and features of the product they choose in a more interactive way
Users no longer need to spend time making appointments with friends offline, and they can choose clothes with friends in the virtual fitting room. When users approach the model, they can see the details clearly. After clicking on the message button, users can also view the latest user's messages on clothing-related details.
USABILITY TESTING
What are the problems with the 1st version?
Usability testing goals:
1. Select the core functions that can improve shopping efficiency as the design basis for the next iteration.
2. Use Usability testing to find out the user experience and accessibility problems in the prototype.
3. Through user testing, further dig into the user's attitude and demands for AR virtual fitting.
Core question:
1. How can we help users build virtual profiles so that they can better choose the right size and view the look?
2. How might AR models fitting display improve the efficiency of chosing size and color when shopping online?
3. Whether users can see the details and the comments of the product efficiently?
Overall takeaway:
1. Virtual fitting feature has improved the efficiency of Lululemon shopping
This includes customizing the user's body model, showing products one-to-one and comparing products through AR, and viewing details that help users improve the efficiency of fitting.
2. Size recommendations and virtual profile generation are a relatively new experience for users, and the interaction process should be as simple and routine as possible
Users want to view virtual models with simple operations rather than complex processes.
3. User feedback helped us understand the lack of some features
This includes rotating and moving delicate models, adding shopping cart and wishlist with one click, returning to the previous menu on some pages.
TOP 3 Changes
What improvements will we make?
1. Simplify the functionality of the interactive fitting and keep only the core features.
2. Modify the colors and fonts using lululemon style, so that the User interface is coherent and consistent with Lululemon's original shopping app.
3. Improve user interaction with AR models through a gamification way. Including imitating the way of customizing avatar in the game, rotating, zooming in, checking the details in the game, etc.
ar design process
How can I test it in a more realistic way?
I created different clothing materials for the virtual model, imported the .fbx model into the Unity Scene, and then built it in IOS system for testing.
hi- fidelity
Final design.
01
Customize your body model
After inputting the figure data, users can get a virtual 3D model that matches their figure. If they don't know the specific data, they can adjust the slider until it suits the body.
02
Try on clothes in your room by AR
Users could place virtual models in AR room through an iPhone camera to simulate the product's try-on easily. They could also choose different sizes and colors to view the real effect in the AR space.
03
Match clothes
After selecting the color and size of the clothing, users can see similar product recommendations and matching recommendations in the drop-down interface. Users only need to click to view the try-on.
04
View clothing details and features
When the user walks into this virtual model in a real scene, the user can see the material details of the clothing clearly. If they click the buttons on the screen, they can also see the characteristics of different parts of the product. Users can decide whether to buy or not from a more realistic perspective.
◉ reflection
What have I learned on this journey?
01
Learning Journey
in E-commerce and Retail Realms
During the hackathon, we delved into the e-commerce and retail realms, embracing the challenge of understanding Square's API and merchants' specific needs. This invaluable and interesting experience, despite our initial unfamiliarity, expanded our knowledge and equipped us with a deeper understanding of this domain.
02
Holistic Integration
for Impactful User Experience
We seamlessly integrated Square API with tools like Google Maps, Google ARCore, Google Pay, and Meta Spark AR. This comprehensive approach caters to every aspect of the user's shopping journey, empowering us to navigate the intricate interplay of various technologies for a robust solution.
◉ IF I HAVE MORE TIME
Next Steps.
◎ Personalized Recommendations Using AI Technology
Harnessing AI technology, we aim to offer personalized store and deal recommendations tailored to each user's unique interests, enhancing merchant exposure and boosting order success rates on ShopX, leveraging the immense potential of AI technology.
◎ Support Service Appointment
We are actively working to enable online booking of merchant services soon, aiming to improve the user experience and benefit service-oriented merchants.
◎ Support More Loyalty Configuration Ways
Currently, our platform supports loyalty programs based on points for merchants, with plans to add product category information soon, empowering merchants to enhance offerings and optimize customer experiences.
◎ Expand AR Scenarios with Square
We believe AR activations foster emotional connections crucial for loyalty, with projections suggesting 75% global AR user frequency by 2025. We'll explore additional AR scenarios, like integrating AR message boards for users to seamlessly leave personalized messages in their surroundings.
You may also like
prototype
You may also like
Why working with us?
100% remote
Lorem ipsum dolor sit amet consectet adipiscing elit et ac adipiscing.
Unlimited PTO
Lorem ipsum dolor sit amet consectet adipiscing elit et ac adipiscing.
Flexible hours
Lorem ipsum dolor sit amet consectet adipiscing elit et ac adipiscing.
Medical insurance
Lorem ipsum dolor sit amet consectet adipiscing elit et ac adipiscing.
Career growth
Lorem ipsum dolor sit amet consectet adipiscing elit et ac adipiscing.
Great culture
Lorem ipsum dolor sit amet consectet adipiscing elit et ac adipiscing.
What our clients say
Lorem ipsum dolor sit amet consectetur adipiscing elit et ac adipiscing quis enim mi turpis etiam.
“The best CRM platform, ever”
Lorem ipsum dolor sit amet consectetur adipiscing elit massa enim neque tincidunt aliquam vel urna orci ante. felis vel elementum cum elit arcu hendrerit id ultrices nulla molestie posuere non arcu accumsan.
John Carter
CEO & Co-Founder
“Techplus X is the next-gen CRM””
Lorem ipsum dolor sit amet consectetur adipiscing elit massa enim neque tincidunt aliquam vel urna orci ante. felis vel elementum cum elit arcu hendrerit id ultrices nulla molestie posuere non arcu accumsan.
Matt Cannon
Lead of Design
“The best CRM platform, period”
Lorem ipsum dolor sit amet consectetur adipiscing elit massa enim neque tincidunt aliquam vel urna orci ante. felis vel elementum cum elit arcu hendrerit id ultrices nulla molestie posuere non arcu accumsan.
Sophie Moore
CEO & Co-Founder
You may also like
Open Positions
Lorem ipsum dolor sit amet consectetur adipiscing elit et ac adipiscing quis enim mi turpis etiam.
Can't find the perfect position?
Contact us anyway. We may have an available opportunity for you in the future.
Contact