Language is the foundation of information communication, and barrier-free communication has always been a dream of mankind. However, due to differences in dialects, languages, speaking styles, etc., there are great communication barriers between people. At present, the development of artificial intelligence is breaking down the barriers of language communication. Many multilingual speech recognition products have appeared on the market. In addition to Chinese Mandarin and English, these products also support multiple languages and dialects, breaking the barriers of national boundaries and regions for communication between people. Multilingual speech recognition is full of challenges when trying to achieve high-accuracy recognition because of its understanding of multiple languages.

CHALLENGE

Lack of Prior Knowledge of Each Language System

There are more than 6,000 different languages in the world, and dialects are countless. The pronunciation systems and pronunciation skills of many languages vary greatly from one language to another. These all require specialized linguistic analysts to study, as prior knowledge for constructing different language systems is necessary. However, due to the large number of languages and dialects, there are too few professionals familiar with these pronunciations and annotations. Because of this, the idea of separate modeling of different language features has still not been brought to fruition. A further complication is that many languages have a very limited number of speakers, much less specialized linguistic analysts. A third of the world’s 6,000 languages are spoken by fewer than 1,000 people each.

Difficulty in Collecting Many Languages

Among the many languages, some languages are spoken by only a few people, which makes language acquisition difficult. At present, industrial-level speech recognition systems are trained with tens of thousands of hours of data, and the general speech recognition accuracy rate is about 98%. However, the migration from popular languages to small languages also requires a certain amount of data of minor languages. Therefore, the collection of minor languages data is the key to improving the accuracy of speech recognition of minor languages.

SOLUTION

Intersection of Speech Recognition and Linguistics

Speech recognition involves the study of speech and language. Many colleges and universities have established phonetics study, but there is a lack of linguistic study. While linguistics is a branch of literature, multilingual speech recognition is inseparable from the construction of language systems and the analysis of language prior knowledge. Therefore, combining phonetics study with language study is the key to solving the lack of prior knowledge of language systems.

Multilingual Data Collection

Data collection is very difficult due to the small number of people who speak the minor languages. It is a great challenge to collect multilingual voices covering a wide range of areas, covering many speakers in a large volume. For AI algorithm researchers collect data, it will take up a lot of time and energy, and the cost will be high. Therefore, a professional data company team is needed to help us break down the barriers of multilingual speech recognition - the lack of multilingual data. Magic Data is a leading global AI data solutions provider, with over 400 speech datasets in more than 60 languages and dialects, including: English, Mandarin Chinese, Tagalog, Japanese, Thai, Spanish, Arabic, Urdu and other languages. These datasets cover various scenarios such as in-vehicle speech data, conversational speech, and recording studio data. Some examples are as follows:

MDT-ASR-E076 Filipino/Tagalog Conversational Speech Corpus

MDT-ASR-F021 Bahasa Indonesia Conversational Speech Corpus

MDT-ASR-F027 Brazilian Portuguese Conversational Speech Corpus

CONCLUDING REMARKS

In addition to providing multilingual voice datasets, Magic Data also provides customized AI data collection and annotation services. For more information, visit www.magicdatatech.com or contact business@magicdatatech.com.