Buy Now!

Behind the scene – Binaural Audio

Sound design may be one of the most overlooked aspects of making a video game. The tragedy of it is that, when it is well thought-out and greatly executed, it usually goes unnoticed as it perfectly blends in with the overall experience. One of the key intents of sound design is generally to feel natural and to remain consistent with the other components of the game. That being said, there are many interpretations of what “feeling natural” means, many ways to achieve it, multiple priorities to set and different technologies to choose from.

In this story, we walk you through what objectives we pursued for Space Junkies with its sound design with the help of our Lead Sound Designer, Charlie Atanasyan.

Information is Power

At its core, Space Junkies is a First Person Shooter (FPS), in which teams of players duke it out with all sorts of weapons in Deadly Orbital Arenas. That simple statement has a great number of implications in terms of sound design, starting with the FPS aspect.

In FPS games, audio plays a key role in delivering crucial, life-or-death information to the player during the heat of combat. Whether it’s shots being fired, jet packs flying nearby or a helmet cracking under enemy attacks, a lot of essential feedback is given to the player through audio. This is especially true for everything happening outside of his / her field of view.

In the case of Space Junkies, the primary objective of our sound design was to provide the player with relevant information about what is happening to them and around them, without being overwhelming. Early in the project, our sound designer set up a list of do’s and don’ts to work on that design. One of them was for instance to avoid or limit any idle, static or extra-diegetic[1] sound. The emphasis was put instead on spatialized sources.

One of the big challenges in the design was to limit, as much as possible, the quantity of information given to the player at the same time. The more information you receive, the less accurate the feeling of spatialized audio.

Another key aspect of Space Junkies is that matches take place in Orbital Arenas. In these spherical battlefields, enemies are coming at you from virtually every direction – being shot at from above or hearing an attacker cruising below you are common occurrences. To deliver reliable information, the positioning of the various sound effects must be on point on both a horizontal and vertical perspective. 3D accuracy for sound effects is essential if they are to be of any use to the player.

Lastly we have immersion, audio obviously plays a key role in feeling immersed in a specific environment. In our case, we use many sound effects and tricks to convey a believable depiction of space while supporting the game’s style and general vibe.

Getting There

Early in the project, our sound team spent a few months researching the different alternatives in terms of sound technology to use for Space Junkies. Possible solutions included 5.1 and Binaural audio. Traditional stereo was ruled out early, as it did not seem strong enough for accurate 3D positioning (especially regarding vertical positioning).

Here’s an independent comparison of Binaural audio and Stereo sound:

5.1 audio, on the other hand, is able to provide great positioning and deliver very accurate information to the listener. Its implementation in certain movies such as the recent Dunkirk for instance is often nothing short of breathtaking. The major downside of 5.1 for us is that it requires well-calibrated multi-channel speaker system. Although many gamers today, use such gear, the vast majority of VR players generally use a pair of headphones for the improved immersiveness. 5.1 also does not support head-tracking which was required for Space Junkies. As a result using 5.1 audio was not as interesting to us in order to provide that fully immersive experience.

We decided to focus on Binaural rendering, which provided the most immersive and accurate sense of spatialized audio. This technology uses HRTF[2] to simulate natural sounds emanating from any point in a 3D space, including behind, above or below the listener.

Binaural audio was famously used in the Virtual Barber Shop 3D sound demo[3]:

This recording technique was created in the 70s, composed of a dummy head equipped with a pair calibrated microphones inserted into anatomically correct ears. In recent years a significant breakthrough was achieved in this field with the development of real-time encoders that simulate binaural rendering. We are now using binaural simulation capable of calculating dynamically, the position of audio sources anywhere in a three-dimensional space.

Another deciding advantage for us was that binaural audio works and provides great results for the listener with a simple pair of headphones. Its main drawback is that Binaural audio can’t be encoded to, or from, 5.1 audio; the decision to use it must be made very early. Fortunately, Space Junkies uses Brigitte, a custom engine we built from the ground up for VR gaming. This allowed us to tailor an audio system around the specific elements of the game at an early stage, instead of adapting what already existed. Binaural rendering is even used for our in-game voice chat system, which is spatialized to help the player quickly locate his teammate in the environment.

All in all, Space Junkies uses a combination of 4 audio technologies:

  1. The game runs two real-time binaural encoders simultaneously:
    • One used to position and spatialize most of the sound effects. It also handles a room-modeling engine with early and late sound reflections[4].
    • One dedicated to voice chat and foley sounds.
  2. Native binaural recording with “Robert”, the studio’s custom dummy head was used to record all sounds with a fixed position such as character voices and breathing sound systems.
  3. Many audio sources were “binauralized” using a specific technique of re-recording that has been very useful for most of the head-relative sounds such as SFX, UI and HUD sounds.
  4. Ambisonic audio format has also been used, mostly for head-relative[5] effects that needed to be dynamically spatialized inside the character helmet.

Supporting an Identity

Space Junkies goes for a retro sci-fi vibe, inspired by the 80’s when space was a strongly fantasized object that was present everywhere in pop culture. Our sound design aims at fueling this specific identity and style which can be heard clearly in the following samples:

It can be heard as well in the synthesizer-loaded title track that greets the player following neon-bright Ubisoft and Space Junkies logos: Space Junkies: Game Theme Song

Some of the sound effects we recorded are truly inherent parts of the game’s identity / personality. We thought it’d be interesting to give you more insight on how we designed and recorded some of them.

Hearing a character breathe in his / her spacesuit is a prevalent trope in movies which helps emphasizing the loneliness of space or tension in a scene. This feeling of immersion in space was something we wanted to bring to the game. Furthermore, breathing sounds add a lot of character to, well, our characters, with each one having his / her own specific ways of breathing (or clicking, in the case of Shellshock). So, not only does it help for immersion, but it also gives valuable feedback to the player on his health status: a character will exhale loudly and run out of breath if he / she takes a lot of damage, circling back to the part about adding tension to the scene.

We wanted to make these sound really natural and immersive, as if they were coming from the player’s own mouth. After rounds of experimentation on how to record and process these sounds, our solution was to put BOB’s ears in a fish bowl and record our breathing sounds within it. Every vocal sound of the game came from this single fish bowl:

SJ Human Breathe Sound Clip

All characters who wear helmets hear cracks when they receive damage; all that don’t, utter a loud grunt. Obviously both are supposed to inform the player that things are going poorly and that they must react. The intensity of the cracks actually increase with the impact force.

To create these bright creaking sounds, we recorded the destruction of car windshield, which we then binauralized to render a 3D effect.

SJ Helmet Creeks Sound Clip

For Shellshock’s eerie squeal, we used the sound of fireworks being lit and blowing up underwater as a baseline.

SJ Alien Source Fireworks Sound Clip

In Space Junkies, players get to fully embody their character; their head and arms movements are tracked and reproduced in-game to allow for this greater sense of control. To expand on this perception, we designed sound effects for collisions and frictions with the environment. These are important to give players feedback on where they are and what they in contact with.

In order to give players the sensation of sounds transmitted via vibration through their body, we used a contact microphone that captures vibrations through material.

What the Future Holds

Binaural audio shows many benefits in terms of gameplay and immersion, and has the potential of becoming a standard in the game industry. It is also very accessible and doesn’t require any expensive sound system, a simple pair of headphones is enough to experience a wide and spherical sound.

There are still areas where real-time binaural audio simulation can be improved:

  1. It’s still not used as a standard for consoles and PC games when headphones are used where it could very well be. The challenge is that binaural audio must be part of the audio design from the start.
  2. The CPU load of these real-time algorithms could be better optimized allowing for more dynamic reverberation.
  3. The creation of custom, person-specific, HRTF equations that adjust to the shape and features of the player’s ears and head.
0
0
Category : News

Leave a Reply

Your email address will not be published. Required fields are marked *