Building technology tools for every language in the world with Felix Laumann
Picking the brain of Felix Laumann on his experience building accurate natural language technology for every language in the world.
In today’s post, we have Felix Laumann. He is the CEO of the language technology company NeuralSpace which offers a no-code natural language processing platform for more than 80 languages. The post will cover Felix’s first start in natural language processing (NLP), being early to the world’s most popular and useful neural network architecture Transformers, and the joy of building applications for every language in the world.
Let’s Dive into it.
Hal:
Tell me more about yourself.
Felix:
I am originally from Germany, where I met my co-founder Ayushman who came to Germany to study for his master’s. We studied and worked together during our time in Germany. I found the possibility of working together on language-related technology interesting. Ayushman speaks several languages including English, German, and the Odia language which is the language spoken in the east of India, the region he was born and raised in.
He suggested that we start the company together and use our knowledge to build the company in the language technology space. We started building a chatbot in English. Soon after we tried applying the same techniques to Hindi, and found that the technology did not transfer well to Hindi. We had much less Hindi training data (5% of what we had in English) and the model architecture needed additional improvements in order to make the natural language processing model work well with limited amounts of data.
Hal:
When exactly did you build it?
Felix:
It was sometime in 2016-2017.
Hal:
Can you tell me a bit more about the specifics of building a chatbot back then?
Felix:
We focused heavily on building natural language understanding models for user intent recognition and slot labeling. The goal was to build models that understand the user intentions very well first, before designing the flow of the dialog.
At that time we did nothing about the business, and it was more about fun exploration around building natural language understanding for chatbots.
Back then, there was very limited data available to train such models. We had to create some of the data ourselves. For Hindi, my co-founder collected and annotated data himself to train the models. Despite our best efforts in creating the data, our LSTM (best performing NLP model at that time) intent classification model failed to get anywhere close to a reasonable accuracy that could make to commercial product.
Hal:
What happened when state-of-the-art Transformers came out in 2017?
Felix:
It was a complete paradigm shift for natural language processing technology. Transformers were pre-trained on a huge corpus of text data and then fine-tuned on a particular task. The entire community was speaking about them.
And then the open-source Transformers library from HuggingFace 🤗 (yes that’s the name of the real company) came out. I followed them right from the beginning when they had only 10 stars on GitHub (now they have 67k+ stars). We leveraged them right away for building our intent recognition and slot labeling technology for chatbots.
We did more than just download their source code and use the pre-trained models from HuggingFace 🤗 . We pre-trained several Transformer models ourselves that combined data from multiple languages together, and further customized the Transformer architecture to fit our needs. As we were doing it, we started to build the engineering platform to handle the entire life cycle of natural language processing models.
Hal:
Did the described platform handle data processing, model training, evaluation, and deployment altogether?
Felix:
Exactly! It handled all that you mentioned, and deployment was the core part of the platform. Deployment of a large model is not easy. It is possible to deploy these large models for a little cost, but the user would need to wait for a relatively long time to get a response (high latency response). This can easily destroy the interest of the user. On the other hand, you can spend a lot of money and have your large model output response very quickly (low latency response) but it will not be very economical.
Hal:
How did you then make the deployed model have low latency with low cost?
Felix:
We spend a lot of time researching ways to reduce the size of these models. We distilled the original large model into the smaller-sized model without any loss in accuracy. We also quantized the weights and developed some in-house engineering methods to further reduce the latency.
Hal:
Very interesting! Sounds like you have had a lot of experience training and deploying natural language understanding models since 2016.
How did you end up working on your current company NeuralSpace which offers no-code tools for low-resource languages?
Felix:
We were always thinking about building a product that focuses on low-resource languages because there are so many speakers of those languages.
While we were building our NLP platform, we accumulated a lot of knowledge on how to build very accurate language technology models for applications with a small amount of data. At that time there were few models supporting low-resource languages, and unlike us they were not accurate enough to be commercially viable.
Hal:
What did you discover while building and selling such software? Was it more of a technological or a scientific challenge?
Felix:
Technology was a bigger challenge. It is not hard to build a natural language processing model that understands 60% of users’ intents in the chatbot. But the experience of using it would be very frustrating. When using such technology, the chatbot’s response would be wrong half of the time. You need to build the models with more than 90% accuracy to make them commercially useful for chatbots, even for very low-resource languages such as Telugu or Tamil.
This is the value that NeuralSpace can bring. Multiple highly accurate language technologies (entity recognition, machine translation, intent recognition, language understanding, and more) for more than 80 languages all in a no-code environment. Even the most hard-core data scientists are telling us that they do their work 10x faster using our platform.
Hal: This is very impressive!
You have already built a lot, so what is next for the company?
What are you looking forward to building in the company in the next year?
Felix:
I am very excited about incorporating speech recognition technology into our platform. Right now you first run speech recognition to get the transcription and then apply the NLP model on top of it to do any task like sentiment analysis. However, this approach fails horribly in low-resource languages.
On top of that, speakers of languages like Hindi, Bengali, or Bangladeshi frequently mix English with their language. This makes the speech transcription technology fail because it was trained with only one language in mind.
Solving challenges like this motivates me!
Hal:
Sounds like an exciting challenge that you have been through many times before.
What advice would you give to a version of yourself five or more years ago when you just started working in the NLP space?
Felix: Very good question. There are some regrets that I have.
I regret the most not putting more effort into learning multiple languages. I picked up a bit of Hindi, and a bit of Indonesian Bahasa. But I never picked up a language to the full degree. You learn so much about the culture of the place when you learn their language. I don’t expect to learn and speak 20 languages fluently, but learning more languages would have benefitted my thinking and development of natural language processing technology for such languages.
I also regret not spending enough time working on things that matter beyond accuracy. Latency time and UI of your language technology application are as important as the accuracy of your neural network. An extra 1% gain in accuracy is not worth as much as improvements in latency and creating a pleasant experience for users!
Hal:
Indeed that is what I find myself as well. I hope my blog helps readers get to our joint findings quicker and not repeat our mistakes.
Voila! Hope you enjoyed this interview!
I would really appreciate it if you could recommend this substack to your friends who are interested in this topic!