FOLD is an experimental VR exploration game where the environment is created by projections of three-dimensional fractals. Other fields of experimentation are the non-linear and fragmented narrative with audio-only feedback, unique movement mechanics, and navigation by spatialised audio cues.
The project started in November 2017 as a part of Game Design and Production studies at the Aalto University, Helsinki. The team is:
Valtteri Bade – art, design, programming
Matias Harju – sound, music, design
Helena Sorva – story, design, programming
Ville Talonpoika – design, programming
From the sound design perspective there are several areas of interest and challenges. One is the use of audio as the key component in the narration. As the game world is abstract and not from this world, the human voices and carefully created sonic elements might help the player to feel empathy and interconnect with the story.
The game environment is actually “virtual reality” inside the virtual reality: all the forms and shapes are just 2D projections of mathematical functions in front of the player inside the 3D game world. This creates a fascinating starting point for sound design; how does it sound inside the fractals, and is the player inside them in the first place or just being “hallucinating”? Do the fractals have a sound? Can the player interact with the surroundings and will that produce sound? For the demo we decided that the different environments in the game should emit different kinds of music, but a practical problem aroused: where should the audio sources be located and how the musical elements should be divided between them?
A technical challenge is to use binaural spatialising to its full potential in order to enable the player to navigate in the abstract environment by using only her/his ears. Currently I’m using Wwise as the audio middleware. For the first demos I used Google Resonance spatialiser, but now I’m testing Auro-Headphone by Auro-3D thanks to a free academic license from Aalto University. So far I haven’t managed to get realistic-sounding results from either of the plugins, but it might be a question of finding the right settings. As soon as DearVR releases their spatialiser plugin for Wwise I would very much like to check that out as I have been happy with their approach to this kind of general binaural modelling.
Finally, still from the sound perspective, an interesting topic is to study procedural audio and music to enhance the narrative and overall experience. In the current version of the FOLD some interactive sound elements are already implemented, such as atmospheric synthesisers changing their base pitch according to the orientation of the player. However, further experiments will be conducted during the spring 2018. One technical approach that has already been tested would be creating own plugins for Wwise using Pure Data and compiling them e.g. with heavy by Enzien Audio.