The Chat API and SDK provide respectively an endpoint and a languageDetector method that makes it possible to detect the language of a user intent. If your Inbenta solution includes multiple Inbenta instances in different languages, this allows you to suggest which one of these instances appears to be the best suited to provide results to this intent. If you only have one instance, it obviously becomes the best and only option.

Inbenta's language detection can only search the languages associated with your instances. If you need more, you may need to use another language detection system from a third-party provider.

Detection Methods

Several methods exist to detect the language of the user intent.

Language detection upon No Results

This is the solution recommended by Inbenta. It involves detecting the language only when the Chat app is unable to find any results for the user intent.

The Chat app follows these steps:

1. Search for results to user input
2. Upon finding no results, the Chat app checks the user intent language.
3. If the intent is in the expected language, the Chat app proceeds with the "no results" process.
4. If the intent is not in the expected language, the Chat app can respond in a variety of ways. For more ideas about how to handle unexpected languages, see the Unexpected Languages section. 

The following diagram explains this flow:

There are two ways to call the language detector after the Chat app returns a No Results:

• Option 1: Use the No Results flag
  1. Disable the No Results flow provided by default in the Chat app instance by deactivating these intents. The Chat API will then return the No Results flag and no-results Direct Answer when it cannot find any results.
  2. When the JS Client receives this flag, it can request the language detection of the user input, and take appropriate action based on that result.
• Option 2: Use the No Results dialog
  1. Use the No Results flow provided by default in the Chat app instance. 
  2. Fill in the Answer Text for the High Number of Unknown Words intent with a text of your choosing (e.g. "There are a lot of words here that I don't recognize") or eliminate this Answer Text if you do not need it.
  3. Add a JS Callback in this intent with the value "customTrigger". This corresponds to the chat SDK action customTrigger. It allows the SDK to intercept the API response using the "onCustomSubscription" inside an adapter, then perform a request to the language detector to check if the user intent uses another language.
  4. The adapter documented here provides a comprehensive example of this process that uses the No Results dialog with a JS Callback in the High Number of Unknown Words intent.

Continuous language detection

You can also evaluate the language of all incoming user intents and then decide to search for results or not, depending on the detected language.

 The following diagram explains this flow:

Using the right solution

There are two factors to consider when you decide to use one solution over the other:

• the number of API requests (and thus the cost of the solution), and
• the number of user intents that you receive in an unexpected language.

Remember that f you use the first option, all user intents, including the ones in unexpected languages, are registered in the system as user inputs.

The solution that performs the lowest number of requests to the API, and thus is more cost effective, depends on the number of user intents that you receive that are in an unexpected language. This is why Inbenta typically recommends the first option: if you get very few of these questions, then the second option becomes very expensive for minimal gains. It only becomes cheaper if you regularly receive a third or more of your user questions in an unexpected language.

Tracking

The Chat API automatically tracks language detection when it occurs. This is done with the log type and data key called LANGUAGE_DETECTION. For more information, see API Tracking Guidelines.

The Chat App's Sessions > Details show when a language was detected in the conversation transcript. How exactly it appears in Sessions > Details may vary depending on your integration method.