Jumbli

Developing the UI in my previous game was a hassle. It was something I had to endure and reluctantly keep improving and it sucked time away from the core development. Who cares about menus anyway! Yes, I was that naive. Fortunately I came to my senses and I now enjoy developing the UI.

Having recently scrapped a recent attempt, I'd like to share the thoughts behind my latest solution. As I develop, I tend to adopt certain principles. They help guide my decisions and designs. Here are ones I feel strongly about that relate to immersive user interfaces:

The UI should exist as interactive objects in the virtual world

No opening menu screensNo pause button to get back to the menuIt should be operated by touch, with either hand

Make it intuitive and fun

It should be obvious how to interact with the UIInteraction should always give audio and haptic feedback

Avoid language where possible

Have as little text as possible - preferably noneIf it needs lots of explaining it's not intuitive enough

I decided to use holographic-style displays for a couple of reasons.
The user will intuitively reach out and touch them, which means they don’t need to figure out which buttons to pressYou expect your hand to pass through it, so doing that doesn’t break immersion

Having a holographic display necessitated a projector of some kind. One of my design principles is that if it looks like you can move it, you should be able to. So I made it possible to knock the projector with your controller and temporarily disable the display. It may sound like a trivial detail, but I'm always looking for ways to make the virtual world act in expected ways because that helps increase immersion.

Initially I developed the UI as flat textures on a holographic display. The interaction worked well but it didn’t feel interesting or add to the experience. So, I decided to model the icons in 3d. Now I could move my hand inside the model to select it and that felt better. This allowed me to add other subtle features that make the UI feel more responsive and real.

At a distance the holographs appear solid but as you make contact they fade to a wireframe to make it feel right that your hand is passing through.

By initially appearing solid, it gives the impression you can knock the icons around. Wherever possible I want to make the user's assumptions correct, so when you first make contact there is a physics reaction that quickly diminishes as the icon fades.

Just for fun I made the icons take on the colour of the controller that hit them and fade out from the impact point.

Hopefully the UI now feels interesting to interact with and adds to the overall experience of the game.

Where possible I'm trying to avoid language so I can appeal to all countries without worrying about translations. I've tried to make the icons fairly meaningful and resorted to animated effects for particularly complex subjects. After feedback from the community I'm going to add tool-tips to aid with understanding, but hopefully people who do not use English as their first language will gain some understanding from the icons alone.

Users may want to navigate menus quickly, so I made the transitions quite quick. While you can select an icon by touching it for a couple of seconds, it's also possible to use the trigger or grip button to progress instantly. This feature is not explained, but easily discovered. My thinking is that users who want to progress quickly will say, "I wish I could bypass the progress bar by clicking the trigger. Oh, I can! Cool".

In my current project it will be possible to access various holographic icons at any time by just approaching and touching them. I use the progress-bar approach to stop a user accidentally making a selection as they move around. I'll be adding an additional test that ensures they are looking towards the icon while selecting it.

Connect with me for more news and insights