Hy-SimulMT

Hy-SimulMT is a simultaneous translation model from Tencent Hunyuan. It supports bidirectional speech-to-speech translation between Chinese and English, receiving audio as a continuous stream and producing translated text and speech while the speaker is still talking. Real-time voice cloning keeps the speaker's vocal characteristics in the output.

Tencent reports a StreamLAAL of about 3 seconds on the RealSI long-form audio benchmark, the lowest among the systems it compared. Its GEMBA score exceeded those of the compared open-source models and four of the five commercial APIs. On short-form benchmarks (FLEURS, CoVoST2, ACL6060, RealSI-Sentence) it scored 84.7 on RealSI-Sentence, the highest in the table, and trailed other systems on some of the other benchmarks. Tencent also compared it with professional interpreters on 18 recordings. These are vendor-reported results. A YouTube summary mentions a 30B MoE with roughly 3B active parameters and quantized versions; these details are not confirmed in the provided README excerpt.

The GitHub repository provides the SimulInfer inference service and SimulTrain training tools. Speech output and voice cloning require a separately configured TTS service. End users can try the model in the Hy Translate App (Android and iOS). The repository license is listed as NOASSERTION by GitHub, so the exact terms should be checked. A YouTube summary says Apache 2.0. The model covers only the Chinese-English pair.