VLC to add support for offline AI subtitles and translations

[harlan4096]

VideoLAN's open-source video player has reached 6 billion downloads. The developers celebrated the milestone by announcing a new feature at CES 2025, AI generated subtitles.

Subtitles are useful when you want to watch a movie or a TV show that is in a foreign language, or if you have troubles hearing. But subtitles may not always be available for the video that you wanted to watch, and sometimes they can be hardcoded or use weird fonts or colors.

Additionally, a good chunk of Internet users may not know how to find, download, and integrate subtitles for local media.

Live Captions

Windows 10 and Windows 11 have a feature called live captions, you can access it with the following hotkey: Windows + Ctrl + L. It is a handy feature that you can use for any media, including streaming videos on YouTube, Netflix, and other video streaming sites. Admittedly, it sometimes struggles to provide accurate subtitles, but it is better than nothing and improving. Android, iOS, macOS also have similar live captioning features.

VLC automatic subtitle generation and translations

Maybe VLC's offline subtitle generator can be a better solution for this. But it is not the first to introduce such a feature. There are other tools that use Open AI's Whisper to generate subtitles, like Vibe or Subtitler. But these apps are not user-friendly per se. What VLC seems to have built could be an easier solution.

VLC to get automatic AI subtitling and real-time translations

VLC's GitLab has some merge requests related to implementing a speech-to-text technology in the video player. It will use Whisper, but it's not relying on OpenAI's Whisper model. Instead, VLC will be using whisper.cpp, a C-based fork of Open AI's Whisper automatic speech recognition (ASR) technology.

VideoLAN's AI subtitling tool will work offline without connecting to online services, i.e. without any requirements for an account or service.

That also means it will be privacy-friendly. Besides, it also eliminates the need to manually search for subtitles, download them, and enable them for the video.

Continue Reading...