The benchmark cohort is the 50 AI-labelled videos. Switch between the validated optimized pipeline and the initial prototype. Compare encoder compute with the measured local pipeline. The same 50 clips are capped at 30 seconds. These timings exclude network downloads and initial model/text-bank loading. The extrapolation assumes the same clip-length distribution.
Read the benchmark conditions before comparing models
Optimized PEAV: batch sizes and loader-worker counts are selected from measured trials. The full pipeline includes decoding from compressed files, selected-frame resizing, padded batching, transfer, encoding, label scoring and buffered output writes. CPU loading overlaps GPU work. Every reported configuration passes embedding-agreement checks against the original single-clip vectors. All trials and validation · Optimized CSV. The following paragraphs describe the initial prototype for comparison.
CLAP uses batch size 8, eight audio preprocessing workers, float32, two warmup passes and three measured repeats. PEAV uses one clip at a time, bfloat16 and sequential preprocessing, with a warmup for each modality. PEAV pipeline times sum the timed decode, preprocessing/transfer and encoder stages; final label scoring is excluded. CLAP pipeline repeats also include AudioSet scoring. The pipeline implementations are not equally optimized; this is a comparison of the measured configurations, not a definitive architecture speed ranking.
The fixed-16 PEAV models distribute 16 frames over the full clip. The current decoder still decodes and resizes all native frames before selecting those 16. Native-frame variants retain every decoded frame within 30 seconds. Selective decoding, batching and CPU/GPU overlap could substantially improve the prototype pipeline. Warm storage and concurrent work on other GPUs may affect wall-clock timings.
Audio + video is a separate joint encoder measurement. It is not the sum of two independently measured runs. Text embeddings are precomputed, and output serialization is excluded. No paid inference service or GPU runs inside this Space; all results were computed beforehand.
Detailed measurements: CLAP configurations and repeated timings · PEAV configurations and timing distributions. Hardware: NVIDIA GH200 120GB, with 102.0 GB visible to PyTorch for each run.