Open Speech Analytic Technologies Evaluations

The NIST Open Speech Analytic Technologies (OpenSAT) Evaluation Series has been designed to bring together researchers developing different types of technologies to address speech analytic challenges present in some of the most difficult acoustic conditions with the end goal of improving state-of-the-art through objective, large-scale common evaluations. OpenSAT had its first pilot in 2017 with three tasks: key word search (KWS), speech activitity detection (SAD), and automatic speech recognition (ASR) for various data domains and followed up with a first formal evaluation in 2019. In 2020, OpenSAT expanded to new tasks with the addition of speaker diarization in collaboration with the Linguistic Data Consortium (LDC) and ASR targeting low resource languages in collaboration with IARPA. It is envisioned that OpenSAT will be the NIST portal for speech technology evaluations. Click on the links below to find out more about recent evaluations:

Active Evaluation(s)

IARPA OpenASR 2020 Challenge - ASR of low resourced languages

Past Evaluation(s)

Fearless Steps Challenge: Phase III - SAD, Speaker ID, Diarization, ASR, and Conversational Analysis on NASA Apollo mission audio data LDC DIHARD III - Speaker diarization in collaboration NIST OpenSAT 2020 Evaluation - KWS, SAD, ASR for public safety communications