Skip to content

Research participation

Reproduce, report, contribute

External researchers can reproduce results, report discrepancies, contribute evaluations, or propose collaborations — without contacting anyone for setup. Local runs are fully anonymous; public submissions need only enough to attribute the work, with explicit consent.

Tell us you're using RIR-Mega

Using the dataset, the Speech corpus, or the benchmark? Let us record it. It's a short, voluntary, consent-based form — share only what you choose. Each entry is a public, dated record of independent adoption (and a great way to get credited).

Share how you use it →

Report a discrepancy or ask a question

Open a GitHub Issue with your environment, the exact command, and what you observed vs. expected.

Open an issue → · Start a discussion →

Propose a collaboration

See the Collaborate page for open questions and the most valuable independent evaluations, then reach out via GitHub Discussions or LinkedIn.

Submit a result (pull-request workflow)

  1. Run a track with --out my_result.json, or author a JSON by hand.
  2. Shape it to the result schema (version 1.0.0). Validate locally: rirbench validate my_result.json.
  3. Add it under submissions/<team>-<date>.json and open a PR.
  4. GitHub Actions runs schema validation automatically. A green check means schema-valid, not endorsed.
  5. A maintainer reviews and assigns a verification tier (below).

Minimal valid submission

{
  "schema_version": "1.0.0",
  "submission_id": "lab-example-2026-10-19",
  "date": "2026-10-19",
  "track": "A",
  "dataset": "mandipgoswami/rirmega",
  "dataset_version": "main",
  "benchmark_version": "1.0.0",
  "split": "test",
  "method": "my-rt60-estimator",
  "sample_size": 400,
  "metrics": [
    { "name": "MAE_RT60_T20_s", "value": 0.11,
      "unit": "s", "ci95_low": 0.09, "ci95_high": 0.13,
      "ci_method": "bootstrap-1000-seed0" }
  ],
  "reproduction_command": "rirbench track-a --dir ./wavs --meta ./metadata_v2.parquet",
  "contributor": { "name": "A. Researcher", "affiliation": "Some University",
                   "consent_to_publish": true }
}

Everything in contributor is optional. For a public listing,consent_to_publish must be true. Local, anonymous runs never need this block at all.

Verification tiers

Self-reported

self-reported

Submitted by a contributor and schema-valid. Not independently checked. Passing automated validation does not imply scientific endorsement.

Artifact-checked

artifact-checked

A maintainer confirmed the submitted code/artifacts run and produce the reported output format on a small sample.

Independently reproduced

independently-reproduced

A maintainer or third party reproduced the reported metrics within the stated uncertainty from the submitted instructions.

A tier is assigned by human review, never by passing CI. Automated validation checks only that a submission is well-formed — it is not a scientific endorsement.