TEAM LANGTECH STORY
African languages have been neglected for far too long, I am Swi Innocent Che closely assisted by Mafor Anna Ngwenyi and together we are Team LangTech.
PROBLEMS0
As an African I like to protect my culture and preserve our cultural expression, Statistics from UNESCO tell us that Africa has more than 1000 languages and at least 200 of these languages have less than 500 speakers. Now for a language to be alive, we need at least 100000 speakers. Ethnologue on the other hand, presents Cameroon as having over 283 languages and 274 of these languages are living, 17 are dying and 9 are already extinct.
The electronic use of African languages is not encouraging, as you find very few languages actually implemented in devices like computers or smart phones.
International experts find it difficult to collaborate with local experts because they do not understand the languages the local population speaks.
Very little Documentation and structuration of many of these languages causing the digitalization extremely challenging
Most African state policies do not encourage the national integration of local languages making youths lose interest in learning at such a gradual decline in the language and in the culture.
METHODOLOGY
Step 1
Data collection and preparation
First data, that is parallel sentences was collected locally, inputted into an excel sheet then profiled beforehand with the help SAS visual analytics tool to obtain some critical information like the uniqueness percentage between each data column, percentage of null values, pattern count, data length, minimum length, maximum length, and others.
Building the Model
From the SAS Boot monitor, we gained access to the Jupyter notebook, it permitted us to write code to train translation model for the different language pairs. The principal Natural language processing library used was Keras. But we can update when need is to SAS deep learning model.
Application
We have built a simple web application using Python flask framework and for the purpose of the demo, the models are hosted with a flask API. As a future plan, we intend to use host our models using SAS API to better manage the numerous translation models for each unique language pair.
Again we have created a diagram using SAS Decision Manager modeling our intension on how to manage a multi-head translation system. Behind each block will be code to govern its functionality.
Demo
As a demonstration of the system, we are going to test a few sentences and observe the results.
- I am a gentleman
- We are going to school
- What is the name of your mother
Notice that for each of the sentences we get a corresponding translation and observing the rest API, we notice the translation time and some other information produced during the time of inferencing the translation models.
