Synchronized Audio, Video and Biosignal Data Capture

Direct answer

Having an MP4, a WAV, and a CSV from the “same session” is not enough. Those files are synchronized only if they can be mapped onto one timeline with known offset and drift. MI-RA Lab’s public description states that cameras, microphones, and other modalities are spatially arranged and synchronized using hardware-based techniques. This page explains what that claim must mean in practice. It does not publish a measured microsecond or millisecond accuracy figure, because none is given on the website.

Who, what, where, why, how

• Who. Human-centric AI, HCI, speech, and physiology researchers using MI-RA Lab at IIT Mandi.
• What. Time-aligned multi-view video, spatial audio, optional OptiTrack motion, and biosignals.
• Where. Anechoic capture chamber and multi-sensor rooms; annotation and workstations for later QA.
• Why. Cross-modal learning needs events, not independent clips.
• How. Shared clocks or hardware triggers, calibration, session metadata—not folder names alone.

Temporal synchronization, timestamps, and trigger signals

Each device has a clock. Video cameras tick in frames (for example 24, 30, 60, or higher). Audio is sampled in kHz. EMG or EEG may run at hundreds to thousands of Hertz. If each device stamps only its local clock, the streams walk apart (clock drift) and may start at different wall-clock times (offset).

Common research practice—used across labs, and consistent with “hardware-based” synchronization—includes: a genlock or word-clock style reference; a shared SMPTE or PTP/NTP time where applicable; an electrical trigger that marks t = 0 on all recorders; or a clap / LED flash used only as a coarse check, not as the sole method for physiology.

Clock drift, sampling rates, sensor latency, and frame alignment

Drift is the slow change of one oscillator versus another. Latency is the fixed delay of a sensor or USB stack. Frame alignment is the mapping of video frame n to audio sample m and biosignal sample k. High-speed cameras and biosignals make sloppy alignment obvious: a 40 ms lip-sync error is visible; a 40 ms EMG-to-gesture error can invert a scientific conclusion.

Exact rates for a booking are protocol-specific. Do not treat the table as MI-RA’s guaranteed configuration.

Calibration, metadata, session IDs, and quality control

• Calibration. Camera extrinsics and mocap-to-camera transforms belong with the session, not in a forgotten spreadsheet.
• Metadata. Session ID, operator, participant code, language, task, sensor serials, trigger method, and hash of raw files.
• Quality control. The Annotation Room and workstations support review and labeling after capture.

Common timeline (conceptual)

The diagram is a teaching schematic. It is not a wiring diagram of the live rack.

Limitations

Do not cite this page as a datasheet. Ask the lab for the sync topology of your booking. Related reading: volumetric capture facility, Indian multimodal datasets, facility types compared.