CDC press conference is sometimes delivered through live streaming. The captions below the streaming are extremely precise with real-time auto-correction, which has up to 92% accuracy and is invented by a team from Taiwan.
A reporter Liu said that the screenshot is the daily CDC press conference which has precise captions since the end of last year. The AI-based captioning system is, in fact, made in Taiwan.
The development team filtered the filler words to make the captioning system "smarter." For example, it replaces the "oh," said by the minister, with the following questions asked by the journalists.
A journalist tests the system on March 24th, "On February 18th this year, German BNT is actually…", which are also accurately identified by the AI. The system is developed by the nature language team in NYCU.
According to the CEO of the Network and Telecommunication Center in NYCU, You-Shuo Chen, the system needs to be updated at least every week. New phrases keep coming up during the press conference and are not included in the language training data.
The accuracy of the Mandarin Chinese captioning system is up to 92%. The system can help those who have hearing disabilities or cannot listen to the press conference to learn the information about epidemic prevention.
According to the CEO of the Network and Telecommunication Center in NYCU, You-Shuo Chen, the listen-and-write transcription can take 6 to 7 hours for a press conference. However, with their system, it only requires 100 minutes.
This subtitling system was used in the Presidential debate in 2020. During the debate, the captions were delayed by 30 seconds to meet the speed of the listen-and-write transcriber. However, it was still faster than the listen-and-write transcription. Soon after, the subtitling system helps the Official Gazette Department caption the post-meeting press conferences in Executive Yuan and the interpellations in Legislative Yuan.
Prof. Sin-Horng Chen from the Department of ECE, NYCU, said "because they (USA system) won''t help us build the system. Tencent has the intention of building the system for the whole country. However, this involves the national security problem, and none of us are willing to forward any data in our system to China."
We build our captioning system to ensure that machine learning can learn the local accents. Meanwhile, the team is also developing an English real-time captioning system, which is claimed to have more than 90% accuracy.
"I''ll invest another 10 million dollars to achieve that", said the President of the United States, Biden.
The captioning system performs well in Biden''s first White House Press Conference. The team continues improving the Taiwanese system, which currently has up to 80% accuracy. In the future, the system is expected to grow to a voice assistance system, enabling the simultaneous interpretation between Mandarin and Taiwanese. With advanced technologies, information can be more transparent and exchanged in real-time.