Independent benchmark & evaluation portal
RIR-Mega: reproduce, evaluate, challenge, extend
A community entry point for independently benchmarking the RIR-Mega room-impulse-response (RIR) ecosystem. Everything here is reproducible, and every number traces to a committed evaluation artifact or to the datasets' own published metadata.
What RIR-Mega is
RIR-Mega is an open corpus of room impulse responses with acoustic metadata (reverberation time, clarity, direct-to-reverberant ratio, and more), plus baselines and tooling for estimating these quantities. It is designed for research on reverberation, dereverberation and reverberation-robust speech.
Scientific questions it addresses
- Can acoustic descriptors be estimated reliably from RIRs?
- How do simulated RIRs compare to measured ones?
- How does reverberation degrade downstream ASR?
Known limitations
- The corpus is largely synthetic; synthetic-to-real gaps exist.
- Some published metadata fields are not populated for every sample (see the Explorer for exactly which fields carry data).
- Licensed CC-BY-NC-4.0 (non-commercial).
How independent researchers can evaluate it
Track A — Acoustic validity
Recompute RT60 (T20/T30), EDT, DRR, C50, C80 and D50 directly from the RIR waveforms with a documented Schroeder-backward-integration estimator, and measure agreement with the dataset's published metadata (bias, MAE, RMSE, bootstrap 95% CI).
Track B — Synthetic-to-real generalization
A reproducible protocol (not a data redistribution) to compare RIR-Mega against independently measured corpora such as ACE and REVERB under their own licenses. Enforces matched sample rate, channel convention and held-out separation, and reports domain shift and failure cases.
Track C — Downstream audio ML (ASR)
Wraps the published whisper-rirmega-bench pipeline: WER/CER on paired clean↔reverberant speech. Documents model versions, splits and compute. A lightweight dereverberation (WPE) baseline is optional.
Resources (all verified 2026-10-19)
Datasets
- mandipgoswami/rirmega — 1,000 rows, revision main, CC-BY-NC-4.0
- mandipgoswami/rirmega-eval — 53,500 rows (ASR pairing manifest)
Code & papers
- mandip42/rirmega — main package
- whisper-rirmega-bench — ASR robustness pipeline
- arXiv:2510.18917 — main paper
- arXiv:2601.19949 — RIR-Mega-Speech paper
- Zenodo DOI 10.5281/zenodo.17387402
Integrity note. This is an independent evaluation portal. It presents published, independently reproduced and newly contributed results as clearly separate categories, and never implies endorsement of a result simply because it passes automated validation.