Poly-glotting your Business with Semantic Understanding

Several AI-driven solution providers are calibrating programs to understand consumers all over the world and helping companies get more patrons

On September 27, 2022, commerce experience cloud Bloomreach launched the Bloomreach Discovery solution to offer French and German semantic understanding for customers in Canada, EMEA, and DACH regions. It will help consumers from these regions to connect with the products they are looking for – without language barriers – and eventually lead to an increase in revenue in the country.

The world comprises numerous languages, and for a business to succeed, it needs to cater to all. What does that mean for your tech?

English computer scientist Tim Burners-Lee, who invented the World Wide Web, has famously said that the computer does not truly understand any of the information it is fed. “[…] but it can now manipulate the terms much more effectively in ways that are useful and meaningful to the human user.”  

A computer perceives every thought in binaries. However, by making the most of a machine’s ability to understand data through patterns, the machine is indirectly learning about the world. It interprets meanings and context. This phenomenon of feeding real-world data to a machine and allowing it to understand it like a human is Semantic Understanding.

Semantic analysis of natural language encapsulates the meaning of the given text while taking into account context, logical structuring of sentences, and grammar roles.

Let us take a look at how it works.

Parts of semantic analysis

• Lexical semantics is a huge part of semantic understanding – it allows machines to comprehend the connection between words. It also works on understanding the dictionary meaning of the utterances.
• Compositional semantic analysis works on understanding the meaning of the whole text as opposed to the meanings of individual words. Let’s take these two sentences, for instance – “John likes Wal-Mart”, and “Wal-Mart likes John”. While both these sentences have the same words, they mean entirely different things. Thus, it becomes essential for a machine to understand the changing semantics.
• In topic classification, the machines sort text according to predefined categories. For instance, semantic analysis at a food aggregator may help the machine understand if the problem is about a “damaged product” or a “payment issue.” 
• Sentiment analysis helps machines understand a text’s basic range of emotions (positive, negative, and neutral). For example, segregating social media mentions to see how customers feel about your brand may help you reach aggrieved people on time. 
• Intent classification involves tagging customers who are interested in your products.
• Depending on the context, a word might mean different things in a sentence. Word sense disambiguation involves understanding the meaning of a word in tandem with the context and where it occurs in the text. For instance, a ring might mean a piece of jewelry; it might also mean the sound of a telephone.
• Relationship extraction involves identifying various components of a sentence and then extracting the relationships between those entities. For instance, let us consider this sentence – Jeff Bezos owns Amazon. In this sentence, we understand that Jeff Bezos is an “owner” of a “company” called Amazon. 

Is it the same for all languages? 

English is not the only language in the world – only 13% of people speak it. For any good business to grow, it needs to be able to hear its customers irrespective of where they are located or what language they speak. Moreover, their feedback – from every nook and corner of the globe – will help a company flourish in the true sense.

In such a scenario, how does semantic analysis work? How well do machines work when it comes to translations? Do they pick up on cultural nuances, contextual knowledge, and colloquial language? Embedding a translation solution into the algorithm skips out on the nuances of language while translating them. It may fail to understand the voice of the customer, which is diverse. 

Translation is already a challenging process. It goes beyond the mere transfer of words—a word-to-word translation will not do. Translations must deal with contrasts between basic linguistic building blocks such as grammar, syntax, semantics, lexicons, morphology, and tonality.

The market for such multi-faceted semantic analysis is emerging quickly.

Bloomreach has more than 12 years of commerce data which it fuses with AI for precise semantic understanding of the texts. With its Bloomreach Discovery solution, it can now tag speech and understand the intent of French and German consumers, thus giving companies an added advantage. Bloomreach is currently used by Bauhaus, FC Bayern Munich, Bosch, Carrefour, Puma, and Sonepar.   

Multilingual semantic technologies

To do these tasks manually will take eons. But at the same time, it seems urgent to understand customers in different languages. This is where a multilingual semantic analysis solution steps in. 

Repusate, an AI-powered sentiment technology solution provider, has built its program to analyze data in 23 languages and dialects through native natural language processing. The company claims its platform can process up to 1,000 reviews per second and provides accuracy in insights upwards of 85% for multilingual sentiment analysis. This software is used by several hospitality facilities in Europe – it sounds like a great idea, given there must be good international footfall. 

Repustate’s semantic technologies calculate sentiment by applying language-specific rules to each piece of text. That means there isn’t one “true” algorithm; what works in English doesn’t necessarily apply to Hindi. It starts by tagging parts of speech in a text. It cross-refers with the existing vocabulary from its pre-tagged text to identify the correct meaning of the word. 

The next step is lemmatization – understanding the word’s root in any language. This helps increase vocabulary and understand the grammatical extensions of every word. For example, in the words – loving, lover, and loved – love is the root, and the rest are conjugations. Repustate will learn terms like these from every language. 

It then learns about the strong words with polarizing sentiments like love, hate, and despise. The Repustate program also studies amplifiers and negations for each language. Negations are mostly placed differently in every language – in some, they come ahead, while in others, they are centered or pushed back.

In its final state, Repustate uses machine learning to calculate a sentiment score that considers all the factors. Brief text with plenty of polarizing terms leads to a score closer to -1 (true negative) and 1 (true positive). A score of 0 or very close to 0 (±0.05) can be interpreted as being neutral – either there was no sentiment, or it was ambiguous.

Austrian AI-solutions provider Cortical.io has developed a software called SemanticPro and has patented technology called Semantic Folding theory. It converts text into a semantic fingerprint – a fine-grained, numeric representation that encodes all contexts of text and is computationally highly efficient.

This technology differentiates between various components of language based on as little as 100 annotated examples. It quickly uses cases in multiple languages.

Many insurance companies have used the program. It has helped them automate and expedite lengthy, labor-intensive workflows around quoting and submissions. 

Going beyond understanding different languages, SemanticPro works on terminologies as well. It has helped financial organizations and large manufacturers comply with changing regulations while delivering fully explainable results.

If you liked reading this, you might like our other stories
Taking the Self-Service Route
BORIS, a Logistical Nightmare?