The high numbers at the end of the period have been saved!

Editor’s note: This article is from WeChat public account “ 新 智 元 ” (ID: AI_era), edited by Daming

Published by Facebook: The high numbers at the end of the period have been saved, AI will help you solve the equation

Source: Turing TOPIA

Recently, Facebook AI announced the establishment of the first AI system that uses symbolic reasoning to solve advanced mathematical equations, with accuracy crushing Mathematica and Matla.

By developing a new method to represent complex mathematical expressions as a language, and then replacing the solution with a sequence-to-sequence neural network translation problem, the researchers built a solution to the integration problem and the first-, second-, and second-order problems. The first order differential equations are superior to traditional computing systems.

Previously, this type of problem was considered to be beyond the scope of deep learning models because solving complex equations requires precision rather than approximation.

Neural networks are good at learning how to succeed through approximations, such as recognizing that a particular pixel pattern may be an image of a dog, or that the characteristics of a sentence in one language match the characteristics in another language.

Solving complex equations also requires the ability to process symbolic data, such as the letters in the formula b-4ac = 7. These variables cannot be directly added, multiplied or divided, and only traditional pattern matching or statistical analysis can be used. Neural networks are limited to extremely simple mathematical problems.

Facebook AI says that the solution they propose is a completely new way to look at sentences in complex equations, so they can make full use of mature technology in neural machine translation (NMT) training models to Turn problems into solutions.

In order to implement this method, they need to develop a method to decompose existing mathematical expressions into the grammar of two languages, and generate a large-scale training data set containing more than 100M paired equations and solutions.

When faced with thousands of invisible expressions, these equations are not part of the training data. Researchers’ models are better than traditional algebraic equation solving software, such as Maple, Mathematica, and Matlab. Significant improvements in speed and accuracy.

This research not only proves that deep learning can be used for symbolic reasoning, but also shows that neural networks have the potential to handle a wider range of tasks, including those that are not usually related to pattern recognition. Researchers are sharing details of the researchers’ methods and methods that help others generate similar training sets.

A new method for applying neural machine translation (NMT)

People who are particularly good at symbolic mathematics often rely on an intuition. What do they feel about the solution to a given problem, for example, observing that if there is a cosine in the function to be integrated, there may be a sine in its integration, and then the necessary work is done to prove it.

By training a model to detect the pattern of symbolic equations, researchers believe that neural networks can piece together clues to a solution, which is roughly similar to human intuition-based complex problem processing methods.

As a result, researchers have begun to explore symbolic reasoning as an NMT problem, where a model can predict possible solutions based on an instance of the problem and its matching solution.

Published by Facebook: The high numbers at the end of the period have been saved, and AI will help you solve the equation

How the researcher’s method expands an existing equation (on the left) into an expression tree, which can be used as input to a transformation model. For this equation, the preamble sequence input into the researcher model is: (+, multiplied by 3, power, x, 2,-, cos, multiplied by 2, x, 1)

In order to implement this application with neural networks, researchers need a new way to represent mathematical expressions. NMT systems are usually sequence-to-sequence (seq2seq) models that take sequences of words as input and output new sequences, allowing them to translate complete sentences instead of individual words.

Researchers used a two-step approach to apply it to symbolic equations. First, the researchers developed an efficient method for unpacking equations, unfolding the equations into dendritic branches, and then expanding them into sequences compatible with the seq2seq model.

Constants and variables act as leaves, while operators (such as plus and minus signs) and functions are internal nodes that connect branches of the tree.

Although it may not look like a traditional language, organizing expressions in this way provides a language-like syntax for equations—numbers and variables are nouns, and operators are verbs.

The researcher’s method enables the NMT model to learn a solution that matches the pattern of a given tree structure problem (also expressed asTree), similar to matching a sentence in a language to its confirmed translation.

This method allows researchers to use a powerful, ready-to-use seq2seq NMT model to replace word sequences with symbol sequences.

Build a new training data set

While researchers’ expression tree syntax enables NMT models to theoretically effectively turn complex mathematical problems into solutions, training such models requires a large number of examples.

Because of the two types of problems that researchers are concerned about (integral and differential equations), a randomly generated problem does not always have a solution, and researchers cannot simply collect equations and enter them into the system.

Researchers need to generate a completely new training set that contains examples of solved equations that are reconstructed into a model-readable expression tree.

This creates a “problem-solution” pair, similar to a corpus of sentences translated between different languages. The researcher’s data set must also be much larger than the training data previously used, and previous research attempted to train the system on thousands of examples.

Since neural networks generally perform better with more training data, researchers have created a collection of millions of examples.

Building this data set requires researchers to combine a range of data cleansing and generation techniques. For example, for symbolic integral equations, researchers have changed the translation method: instead of generating problems and finding their solutions, it is the problem of generating solutions and finding them (their derivatives) that is a simpler task.

This approach to generating problems from its solutions makes it possible to create millions of integration examples. The researchers’ translation-inspired dataset consists of approximately 1 million pairs of examples, which contains a subset of the integration problem and first- and second-order differential equations.

Researchers use this dataset to train a seq2seq transformer model with 8 focus heads and 6 layers. Transformers are often used for translation tasks, and researchers’ networks are designed to predict solutions to various equations, such as determining the primitives of a given function.

To evaluate the performance of the model, researchers provided 5,000 expressions to the model, forcing the system to identify patterns in equations that did not occur during training.

The accuracy of the researcher’s model when solving the integration problem is 99.7%. For the first and second order differential equations, the accuracy is 94% and 81.2%, respectively. These results exceed those of all three traditional equation solvers tested by researchers.

Mathematica’s results are poor, with an accuracy of 84% on the same integration problem and 77.2% and 61.6% accuracy for differential equation results.

Investigator’s model can also return most predictions in less than 0.5 seconds, while its