DSpAST: Disentangled Spatial Audio Spectrogram Transformer
Checkpoints of DSpAST: Disentangled Representations for Spatial Audio Reasoning with Large Language Models.
On our system, the performances obtained with our provided checkpoints are:
| Binaural Encoder | mAP (↑) | ER20° (↓) | MAE (↓) | DER (↓) | | :---: | :---: | :---: | :---: | :---: | | SpatialAST | 49.90 | 24.43 | 17.87 | 32.50 | | DSpAST (stage 1) | 53.05 | 98.56 | 95.57 | 97.58 | | DSpAST (stage 2) | 52.64 | 20.31 | 14.44 | 28.35 | | DSpAST (stage 3) | 54.53 | 20.28 | 14.44 | 28.03 |
Similar performance improvements can also be observed when using DSpAST as a binaural encoder for spatial audio reasoning with LLMs. Please have a look at our paper for further information.
If you use the checkpoints for your work, we kindly ask you to cite the following papers:
and the original BAT paper, which is the foundation of this work: