HoloJamo: Augmented Reality Browser & Navigation

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form

Overview

iOS, B2C, Augmented Reality

Company
ContextGrid
Client
Bloomingdales
Date & Project Length
2015-2018 - 3 Years
Role
Cofounder & UX Designer
Tools
Sketch, Invision, Surveys,
Framework
Lean UX

Maps + Pokemon Go + Web Browsing = HoloJamo.

Imagine a world where digital information is more accessible then the current web. A world where useful digital information lives right on top park monuments, store products–anywhere you can think of, indoor and outdoor–and that information is universal accessible on any phone. This is the problem that we sought out to solve, and in response, we created HaloJamo, a first of it's kind visual browser for viewing location specific information in the form of augmented reality experiences.

This browser is built on top of WorldRegistry, our CMS for placing holograms in stores and places. 

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form

Discovery

Iteration 1: Using Lean UX To build our MVP

Assumptions Survey

The survey was taken by our Stakeholders and included client needs, how they can be solved, who's the user, most important features, etc.

Affinity Mapping

I mapped out the survey data (using Mural.ly) into a tertiary pattern of business assumptions.

I then took that data remapped it into our business assumptions.


Hypothesis to Requirements

With Our assumptions defined, I created our Hypothesis statements about WorldBrowser.



Getting "Out The House"

Our Assumptions were then tested at the Guggenheim. We picked the Guggenheim because the Guggenhiem app has a similar feature, the "what's around me," feature which uses beacons to give location based information about paintings.

After talking to several app users at the Guggenheim who use the "what's around me" feature, and the help desk which give out iPods that have the app, we proved 2 of our assumptions and disproved 1.

Sketching & LoFi Wires

AR Location States & Icons

These are the representations of the User, and AR Locations on the 2D Map. From left to right you can see Empty, On, & Off states which are represented with different colored icons.

AR Location Cards

These cards allow a User to enter into AR Mode if they click on a AR Location that is active (meaning they are close enough to interact with the Location IRL).

AR Mode: Content Cards

AR Content Cards are the key component for taking digital actions in the real world. Theres 2 types Content Cards, Single Action and Multiple action cards. Multiple Action cards open up to a separate page with the list of available Action List links. The Links are then handed off to an app or browser.

Iteration 2: Updating the Experience

For our latest update, we used feedback from Bloomingdales customers and created a new persona: Cassie, AR Enthusiast.

Knowing Cassie's needs, we created a story map of what we needed to build to give her the best experience.

Story Mapping

Working with the Bloomingdales team, our developers, and our biz dev stakeholders, I lead a workshop using narrative story mapping to create our design requirements. The work was done remotely, so we used Murally to facilitate the workshop.

The story went as a so:

1. Cassies at the mall because she needs sportswear. She sees the Bloomingdales she's near has AR Experiences.

2. Cassie taps the Bloomingdales for more information.

3. Cassie Sees that theres holograms all over Bloomingdales, and all different, some of which have actions (URLS).

4. Cassie goes into AR Mode for Bloomingdales.

5. Cassie sees a UFO hologram and is amazed!

6. Cassie sees she can get directions inside Bloomingdales, and gets directed to the Woman's Sportswear.

7. Cassie is directed with AR arrows to the spot. When she's there she sees a dancing alien hologram.

8. Cassie takes a video of the alien and shares it to IG, Twitter, etc.

9. Cassie taps the alien, and gets a 50% off coupon. She buys her sportswear and tells all her friends about the experience.

With our story and requirements identified, we mapped out what sprint each requirement belonged to, and then placed them into Trello.

Note: The following files are all for Iteration 2 of WorldBrowser called Holojamo

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form

Ideation

Flowcharts

click to see full screen :)
click to see full screen :)

Map Mode: See Whats around you

Users can see what Places around them have AR experiences. If they're in range, the AR Button and place pin become active, signifying they can go into AR Mode. If they aren't in range of an AR experience, they can still tap the AR button to see the closest place, or enter AR mode anyway.

When they tap the AR button, the system opens the Place Card modal for the closes Place to them. This card lets our user enter AR Mode.

Left: AR Out of Range
Right: AR In Range

Place Cards

The user can see how many holograms are at place, the address, and some context about that place, whether they are in range or not.

Left: Out of Range
Right: In Range

Hologram Lists and Associated Actions

Users can see all the holograms available and filter by hologram type. They can see a larger thumbnail of the hologram anchor (anchor: the point or object where the hologram is attached to) and access any attached URLs.

Left: Anchor/Hologram List
Right: Anchor Details

Viewing Holograms and Indoor Navigation

Once they're in AR Mode, the user can see and interact with the holograms provided by at the Place. They can also take a picture or video, sharable to their social networks. The user can also get directions to different points in the store using AR.

Left: AR Mode
Right: AR Mode Directions, Step 1
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form

Prototyping

We then created a simple prototype using sketch cloud for testing the flows (note: this prototype's has the updated visual design from the Palo Alto tests).

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form

Testing

Testing Prototype in Palo Alto

Prior to our testing, the UI was very simple.

Image result for student focus group

We did a study with 6 high school student in Palo Alto and found that although they liked the concept of the app, it didn't speak to them visually. There biggest concern was the visual design. They called the colors, Typeface, and patterns "industrial" and too business-focused. They also hated the name WorldBrowser. The dry design also impacted how they perceived it's usability. The general consensus was that it looked hard to use

Introducing: HoloJamo

Punny Typeface, Background Blur, and Gen-Z Yellow

We wanted typeface thats versatile, has readable weights, is bold, beautiful, and fun. So we went with the google font Poppins as the primary font and Open Sans as the Secondary. We then researched the most liked color by Gen-Z highschoolers and found its this vibrant yellow color. Then to add a coolness factor, we added background blur at 5px to all cards and overlays.

Making Holograms Easier to Find

Although the UI and flows made sense to the kids, there was some technical pain points in how we delivering the hologram content. One of the major challenges user testing showed was that it's very hard for users to find our computer vision anchors. This is because they are markerless. An anchor can be any object or point in a store, so there is no signifier like a QR/barcode code. In order for the hologram to be triggered, the user has to point their phone at it using their camera. In short: the user has to find a hologram experience, rather be provided one.

While this works well for some use cases,  users expect the experience to be present without any work. To improve their experience, we added 2 new channels:

  1. GPS for outdoor holograms.
  1. IndoorAtlas, a tool for indoor holograms and navigation.

The UI Challenge

The GPS, IndoorAtlas, Computer Vision-based Hologram experiences all function the same at the user level: they allow a user to view a hologram, and they allow the user to view any attached data to a hologram (a URL to a webpage, video, image, etc). But, due to technological constraints, we need to show them very differently.

My challenge was to make a simple, unified experience, providing a birds-eye view of all the holograms at a place, and what data they carried (if any).

The Solve: Inside, Outside, & Hidden Holograms

The user doesn't need to know which technology is behind their experience. They just need to know where holograms are. I started by thinking of the architecture based on where they will interact with the Holograms. I came up with 3 types: indoor, outdoor, and hidden holograms.

If a users looking for inside holograms are shown on the provided floor plan. Outside holograms are shown on google maps. Hidden Holograms can be placed and trigged anywhere, so they are represented with a thumbnail.

GPS Outdoor Hologram With Action

The outdoor and indoor holograms maps show an active hologram range of 100 meters to improve whats called collusion meshing (basically it's how 3d objects look in the physical environment), Anything out of range is shown with a grey icon. This takes the convention of in/out of range pins from the main map to make an understandable experience for our user.

A user can see if a hologram has any data by tapping on an icon, as shown to the left. If there is no action available, the popover just shows the hologram name. I made this tap 1 & 2 interaction the convention for viewing holograms with data across channels. To give prominence to holograms with URLS, I added a shadow to those icons/thumbnails.

For addtional context, we show the total # of Holograms in the modal title, and holograms per type in the tabs. I used a half-screen modal to make a  better experience  in AR mode. If the view was full screen in AR mode, the user would have what's called broken presence. When they reenter, all holograms would have to reload.The half modal reduces the negative impact of the technical contstraints and create a more cohesive experience.

Left: 2D Mode – Hidden Hologram, With Action
Right: AR Mode – Indoor Hologram,  No Action
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form

Final Product

Results

Unfortunately, the pilot fell through. But, the company has received interest from major entertainment and sports corporation such as the Oakland Raiders for the indoor navigation capabilities.

Most recently, HoloJamo was the US 2019 AWE (Augmented Reality's Biggest Conference) app for indoor navigation.

The product was then rebranded as HoloGrid and is available on the AppStore.

View on Appstore

HoloGrid on Appstore Link

Demo