Recently I've been interested in generating "3D" acoustic soundscapes; I looked into tools that allow creation and rendering of such immersive audio scenes.
First and foremost I looked into what tools exist in this space. Here is an incomplete list:
- 3dti AudioTookit "a standard C++ library for audio spatialisation and simulation using headphones". This one stands out because it is accompanied by a very comprehensive publication [1] that not only describes the technical background of 3D audio spatialization but also lists a number of other tools that do similar things. 3dti works by simulating sound and considering head-related transfer function (HRTF), a function that that describes how ears reveive sounds from a point in space. Some of the tools and libraries mentioned below come from the publication about the 3dti AudioToolkit.
- The Anaglyph Binaural Audio Engine "is a VST audio plugin for binaural spatialization".
- CSound is a sound and music computing system that comes with facilities for spatialization using HRTFs and realtime rendering.
- earplug is a realtime binaural filter that allows spatialiation of sound in realtime.
- Facebook 360 Spatial Workstation "is a software suite for designing spatial audio for 360 video and cinematic VR" that Facebook is ending support for on 2022-05-16 "due to the abundance of quality tools for creating 3D immersive, spatial audio".
- hrir interpolates head related impulse reponses at different distances to locate sources at different distances from the listener.
- Icram Spat "is a software suite for spatialization of sound signals in real-time intended for musical creation, postproduction, and live performances".
- IEM Plug-in Suite "is a free and Open-Source audio plugin suite including Ambisonic plug-ins", created with the JUICE framework.
- OpenAL Soft "is a LGPL-licensed, cross-platform, software implementation of the OpenAL 3D audio API". Development is very active and the OpenAL API it implements is used in numerous games.
- pyrommacoustics "is a software package aimed at the rapid development and testing of audio array processing algorithms". It consists of a Python API, a fast C++ implementation for efficient generation of room impulse responses and propagation simulation as well as reference implementations for popular algorithms (STFT, beamforming, direction finding, adaptive filtering, source separation, denoising).
- Resonance Audio, open source SDKs, tools, and source code for "high fidelity spatial audio at scale" (the "at scale" appendage is likely a remnant from the Google where the project started). There is a SDK for the Unity game engine, a SDK for the web among other things but the core is the Resonance Audio C++ library. It is all about creating experiences with VR googles where realistic 3D sound can enhance the level of immersion.
- slab3d is a real-time virtual environment rendering system originally developed in the Spatial Auditory Displays Lab at NASA Ames Research Center. It seem fairly outdated (last release 2018-08-23 as of this writing).
- SoundScape Renderer is a cross-platform software framework for real-time spatial audio rendering. Development seems moderately active; they have extensive documentation and rely on an XML-based Audio Scene Description Format [2].
- Spatial_Audio_Framework is "an open-source and cross-platform framework for developing spatial audio related algorithms and software in C/C++". The framework is under active development and supports HRIR processing, room simulation as well as augmented-reality devices. The SPARTA collection of VST audio plug-ins for spatial audio production is based on the framework.
- Spatial Sound in Unity and open source plugins for Unity 3D (Windows and Android) with spatial audio DSP processing implementations.
- Steam Audio SDK is a 3D sound API (through HRTF-based binaural rendering) for video games. There is a Unity, Unreal, FMOD Studio plugin and a C API.
- VRWorks Audio is a GPU accelerated audio plugin fo rthe Unreal Engine that uses GPU accelerated path-tracing to render 3D soundscapes.
It is clear that quite a lot of work has gone into the research and development of spatial audio. WIth the growing popularity of virtual and augmented reality, spatial audio is more important than ever. Spatial audio greatly amplifies the degree of immersiveness. But academic interest exists long before AR and VR came along due to the importance of binaurality in speech intelligibility.
Cuevas-Rodríguez, María, et al. "3D Tune-In Toolkit: An open-source library for real-time binaural spatialisation." PloS one 14.3 (2019): e0211899. ↩︎
Geier, Matthias, et al. "ASDF: Ein XML format zur beschreibung von virtuellen 3D-audioszenen." 34. Jahrestagung der Deutschen Gesellschaft für Akustik (2008): 10-13. ↩︎