Jumbli

I've been wanting to investigate webVR for some time and recently the Amazon Sumerian AR/VR challenge gave me the perfect excuse to jump in and see what I could do. Happily, I came away being the winner of the Best Brand Engagement and Retail category.

When looking for uses for VR in business, and in particular marketing, it's easy to fall into the trap of just using it as a novelty or gimmick. I wanted to look at ways it could really benefit a business and serve a legitimate purpose for the customer.

I previously created a website for the www.HorseArc.co.uk and I knew they faced particular challenges that VR could help with, so I decided (with their permission) to use them as a case study.

I've only tested the project on Oculus Rift, Oculus Go and desktop browsers so far.

To try in VR you will need:
Firefox 61 or newer
Chrome 69 or newer (see notes for enabling webVR)

1) Ensure audio output is set to your VR device
2) Launch Horse Arc VR
3) Click any popup displayed to allow the microphone
4) Click the VR icon in the bottom right corner
5) Put on your headset and click the play button in the centre of the video screen.

To try in your desktop browser

The same version of the code runs in: Google Chrome, Firefox or Microsoft Edge.

Launch Horse Arc VR

I've always felt that webVR would be an important technology in the future. After working with the tech I'm pleased to say that we are already much further along than I thought.

Entering the Sumerian challenge meant that I didn't need to evaluate various frameworks. With a few clicks I was signed up and sitting in the Sumerian editor.

With no previous webVR experience I expected a tough road ahead. Coming from developing in Unreal Engine I quickly felt familiar with the editor, moving objects around, importing models and textures and defining materials. There's even Blueprint-style behaviour models that you can use to implement logic by adding and connecting nodes. All concepts I was comfortable with.

As always, when you have fixed objectives in mind, you hit restrictions where you need to delve a little deeper. In this case you venture into JavaScript and APIs. Having previous experience of JavaScript made this less daunting too.

A cool thing with Sumerian is that it's already set up to access the AWS JavaScript SDK which means I can easily make use of Amazon resources in the cloud such as database and email functionality. This adds a great deal of power and using serverless computing means I have a robust, scalable solution without the overhead of running and maintaining a server.

Keeping the customer's attention is an important part of making a sale. Getting them to sit through a full sales video can be challenging, especially if your product is complex or has a wide range of unique selling points to get across.

The approach I took was to stream a narrated video to guide the user through the product features, while using interaction to keep them engaged.

Adding interactions

I task the user with building the product using a simple drag and drop mechanism. Each component is described, and it's benefits conveyed, as the user moves it into place. Keeping the user engaged should increase the chances of them continuing through the experience while also focusing their attention to better absorb the information.

Synchronisation

Moving from a video to a video-driven experience brings benefits, such as having a menu to allow returning users to jump straight to the desired part of the video / experience. It also requires more thought and consideration.

As the video timeline progresses, objects in the virtual world need to be displayed, hidden or re-positioned to keep in sync with the video.

Sometimes the video needs to pause if a user interaction has not been completed and then resume when the action is performed. It's a simple idea, but you need to be able to confidently detect when actions are complete.

Loading media on demand

Keeping the initial download size to a minimum should be a consideration of webVR projects. Using streaming video (in my case streamed directly from Amazon S3) to convey a lot of the details helps with this by only loading the media after the user is inside the experience.

It's possible to embed HTML 3d components into webVR and present HTML, including video, but this component isn't suitable for VR due to the high performance cost. However, video streaming to a texture is much more efficient and practical.

As VR becomes more mainstream the novelty value will fade. It's important to recognise how the technology can be used to solve problems or greatly improve the customer experience rather than just porting existing marketing material.

For the Horse Arc, a barrier to sales is that people need to assess whether the product is suitable for the size and number of their horses. Quite often they will travel to see the product at another customer site before making a purchase. This is not ideal for the business or customer.

Scale

Representing accurate scale is where VR really shines. The Horse Arc model built for this case study is exact scale, down to the thickness of the wooden cladding. Being able to walk around and through the product in first-person conveys so much more than any video or blueprint ever could.

The importance of height

Your height and the floor level need to be accurately reflected to properly gauge scale. Sometimes in VR (particularly in mobile VR) the floor level may not be correct. To work around this I allowed the user to change their height using a simple slider.

Simulating product use

The arc needs to comfortably shelter multiple horses. I allow the user to drag and drop two horses through the arc and adjust their scale so that they more accurately match the user's own horses.

In general terms, if you allow a user to see a product at true scale, interact with it and provide other props to help prove its suitability, they should be much more confident in making a purchase. I can see this becoming the norm in the future - from buying a house, to something as trivial as a mug.

In any marketing material it's important to have a "call to action" where you encourage the customer to take the next step towards making a purchase or completing a certain action. We could simply display a phone number or website address, but this adds a disconnect in the process and risks loosing customers. We could capture the user's email address using a virtual keyboard, but these often feel a little clunky and are slow to use.

The solution I'm testing in this case study is to allow the user to record a voice message, giving them a chance to ask any questions and leave their contact details. This approach suits the Horse Arc. Being a high-value product, people usually feel more comfortable making the purchase after speaking to someone, so a voice message is a good way to initiate the dialogue.

To record the message, you click and hold on the microphone icon. You can play back your message, rerecord it if required and send it when you're ready.

The recording is transmitted to an Amazon Lambda function call and then simply forwarded as an email attachment via Amazon SES - arriving in seconds.

Teleporting is a common solution for moving around in VR. Simply look at the floor where you want to move to and click the mouse. Gamers often prefer smooth / sliding movement - but they are not the target market here and that approach can cause nausea. I think non-gamers will also find teleporting easier than using buttons or thumbsticks for navigation.

When teleporting, there is a tendency to move too close to the object you want to investigate. To reduce this problem I prevent the user from teleporting right up to the edge of the Horse Arc. I also added a large ring around the teleport point to encourage people to position themselves a little further away from objects.

Another benefit of teleporting is that you can auto correct the user's position as they move around. If the user teleports anywhere on the platform in front of the video screen, they are automatically centered on the platform.

While VR is the best way to view the Horse Arc, I wanted to maximise the reach of the experience and make it available in the desktop browser. This greatly expands the audience to include anyone owning up-to-date versions of Firefox, Chrome or Microsoft Edge.

Viewing and navigation

I use an orbit camera in the browser, but bring the orbit radius right down so it simulates your head turning. The user clicks and moves the mouse to look around, the same as you would when viewing a 360 degree photo. The clipping on the camera was lowered to the minimum, allowing you to examine objects up close.

Teleporting is also used in the non-VR mode bringing consistency for people that use both versions.

Height
The desktop browser experience still allows you to control your height. While not as effective as the VR mode this still allows you to more accurately appreciate the scale of the scene.

Interactions

Interactions have been implemented in a consistent way in VR mode and desktop browser mode. Everything is based on clicks or clicking and dragging. Where items need to move and reorient themselves in 3d space, this is done automatically as they are dragged between two points.

In any marketing activity you want feedback on the effectiveness of the campaign. VR has the potential to deliver deep insights into the user experience.

In this test case, I simply count the number and type of interactions and periodically update the stats on Amazon DynamoDB via a Lambda function call. This allows me to analyse anonymous data and see how far users progressed through the experience, how long they spent in it and what they interacted with.

This could be taken much further, such as analysing what users spend their time looking at.

To maximise reach I was keen for the project to work on mobile devices. I've tested it on the Oculus Go and the experience seems acceptable.

I kept the scene simple and vertex count low, removed lighting and dynamic shadows and kept the number of meshes and materials to a minimum.

Savings are also made by only showing objects that the user currently needs to interact with and by spreading assets around the user so they don't all need to be rendered at the same time.

I would advise you to start as simply as you can and then gradually add complexity while performance remains satisfactory.

I need to add code to detect when the video is buffering and show a "buffering" graphic. Currently the video turns white when it is waiting for more data.

I'll be extending the experience to show the height of the horses (in hands) and to allow optional extras to be added to the arc.

I'd like to add some dummy shadows, to ground the objects more effectively.

I'll also be doing more testing and general bug fixing.

WebVR is already at a stage where you can deliver real business benefits.

If you control browser versions / settings and hardware (such as in the workplace) you could roll out solutions right now.

Experiences intended for consumers at home would benefit from supporting mobile phones and desktop browsers to increase their reach, while preferably supporting VR headsets to build your skills and capitalise on the benefits they bring.

Browser support is improving and the webXR API is being progressed. I'm looking forward to when this ships as standard in all major browsers.