The problems of chatbots vary greatly from customer to customer, that is, most of the problems are at the end, so building an accurate system is likely to require a lot of work for each customer. Unfortunately, depending on the solution distribution, these payments and related sales costs may be difficult to reduce through careful design.
To make matters worse, artificial intelligence companies facing long tail problems will actually experience diseconomies of scale. In other words, relative to competitors, economic conditions will get worse over time. Because data has costs of collection, processing and maintenance. Although relative to the amount of data, this cost will decrease over time, but the marginal benefit of additional data decreases faster. In fact, this relationship seems to be exponential. To some extent, developers may need 10 times the data to achieve 2 times the subjective improvement.
Although people are hoping to have an artificial intelligence similar to Moore’s Law to significantly improve processing performance and reduce costs, this does not seem to be achieved (although the algorithm has improved).
Next, we introduce the thinking and the guide to solve these problems obtained from the interviewee.
Two Build a better AI system
Finding a solution
Many artificial intelligence systems are designed to predict complex underlying system interactions, which is the reason for the long tail distribution of input data. Developers usually cannot fully describe the characteristics of the data, so they will model through a series of (supervised) learning experiments. This process requires a lot of work and may touch the asymptote of performance, which in turn triggers or exacerbates many economic challenges faced by artificial intelligence companies.
This is also the crux of the AI business dilemma. If economics is the source of the problem rather than the technology itself, how can we improve them? There is no easy answer.
In a way, the long tail is a measure of the complexity of the problem. In other words, it is the reason we need automation in the first place, and it is directly related to the effort required to solve the problem. However, there are some ways to help us treat the long tail as a first-order focus.
We heard a lot of suggestions on this topic from ML engineers and researchers. Here are some of the best and most innovative guidance.
The simplest case: define the problem
In the simplest case, understanding the problem means figuring out if you are dealing with long tail distributions. If not, for example, the problem can be reasonably described by linear regression or polynomial, then you don’t need to use