Talent and defects coexist.
The Translation Bureau is a subordinate translation team, focusing on science and technology, business, workplace, life and other fields, focusing on introducing new foreign technologies, new perspectives, and new trends.
Editor’s note: This summer, OpenAI launched a new computer system called GPT-3. In terms of natural language processing, GPT-3 has demonstrated amazing capabilities. It can write articles, do translations, generate codes, and even learn a person’s language model and follow this model to talk to people. However, GPT-3 also has certain shortcomings and needs to be gradually improved in the future. This article is the first part, mainly introducing the functions and features of GPT-3, and the next part will introduce the defects of GPT-3 and its future development direction. This article is translated from New York Times, author Cade Metz, the original title is “My Name Is GPT-3 and I Approved This Article”, I hope to inspire you.
This summer, an artificial intelligence laboratory called OpenAI in San Francisco announced a technology that has been in the making for several months, namely a new system called GPT-3. This system learns the details of natural language by analyzing the text in thousands of e-books, Wikipedia, and nearly 1 trillion words posted on blogs, social media, and the Internet.
Mckay Wrigley is a 23-year-old computer programmer in Salt Lake City, USA. He is also one of the few people invited to patch the GPT-3 system. This system uses it from The vastness of all the knowledge learned in the ocean of digital text generates a new language by itself. Wrigley wants to know if it can imitate public figures, write like them, and even chat.
One of his experiments involved popular psychologist Scott Barry Kaufman. GPT-3 is named after Kaufman and discusses the topic of creativity. When asked “How can we become more creative?”, GPT-3 immediately respondedRoad:
“I think creative expression is a natural by-product of growing up in a diverse world. The more diverse the world, the more different people, different opportunities, different places, and different challenges you will encounter. The more these points Diversity, the more likely you are to combine these points to form new things. In many ways, I think if you want to be creative, you must do it. If you want to be a writer, you must write, If you want to be a musician, you have to create music, if you want to be a comedian, you have to create comedies. The more you create, the more likely you are to stumble upon something interesting during these creations. “
Later, when Wrigley posted this on Twitter, someone @the real Scott Barry Kaufman (Scott Barry Kaufman). Kaufman himself said on Twitter: “This sounds like what I want to say.” He later added: “Artificial intelligence is too accurate.”
In the weeks following the advent of GPT-3, dozens of other experiments were conducted in succession, and these experiments were shocking in the same way. It can generate tweets, write poems, summarize e-mails, answer trivial questions, translate languages, and even write computer programs. In this process, GPT-3 can complete these tasks with few hints, some of them Skills caught even experts off guard.
For many artificial intelligence researchers, this is an unexpected advancement towards a machine that can understand the unpredictable human language and may even solve other human skills.
Melanie Mitchell, an artificial intelligence researcher at the Santa Fe Institute in New Mexico, participated in the experiment of the system. She said: “It surprised me and made a lot of People are surprised. It’s hard to figure out what this thing is doing.”
However, GPT-3 is not flawless, it often publishes prejudiced and malicious language. If you ask for 10 paragraphs of Scott Barry Kaufman (Scott Barry Kaufman) style text, it might give you 5 convincing words, while the other 5 are not so convincing. When Kaufman himself actually chatted with AI (“Scott Barry Kaufman”), he was not as impressed as he was at the beginning:
Scott Barry Kaufman (person): What is imagination?
Scott Barry Kaufman (Artificial Intelligence): What is imagination? You should tell me!
Scott Barry Kaufman (person): You should know!
Scott Barry Kaufman (Artificial Intelligence): No, you should know! You are the one who knows what imagination is. What do you think is imagination?
This looks like a real conversation. In addition, it is worth noting that it also demonstrates the ability of the system to complete work independently without additional programming or adjustment.
GPT-3 is the result of years of work by the world’s leading artificial intelligence laboratories, including OpenAI, an independent organization funded by Microsoft with a $1 billion funding, and Google and Facebook laboratories. In Google, there is a similar system to help answer the query results on the company’s search engine.
These systems, called universal language models, can help develop a range of tools, such as services that automatically summarize news articles, and “chatbots” designed for online conversations. So far, their impact on real-world technology has been minimal. However, GPT-3 opens the door to a series of new possibilities, such as software that can accelerate the development of new smartphone applications, or chatbots that enable more humane conversations.
As more and more software designers, entrepreneurs, experts, scholars, and artists begin to explore this system, each new experiment may trigger a fierce debate about how powerful this technology will ultimately be. Although some people say that this may be the path to a truly intelligent machine, others believe that these experiments are fascinating and misleading.
“It’s very fluent and eloquent,” said Mark Riedl, a professor and researcher at the Georgia Institute of Technology. “It’s very good at generating text that sounds reasonable. However, what it doesn’t do is think ahead. It has no plan what to say. It has no real goal.”
a kind of “newness”
Jordan Singer is a product designer for Square, a Silicon Valley mobile payment company. He helped design the company’s smartphone application, creating graphics, menus, buttons, and other widgets that define the appearance and content of the application. When he heard about GPT-3, he wanted to know whether this automated system could do his job.
He entered a brief description of a smartphone application into the system and the computer code needed to create the application. The code description is in simple English. The code is built inside Figma, which is used by professionals such as SingerOf professional design tools.
He did this several times and entered more English descriptions into the system while matching the Figma code. When he completes these operations, GPT-3 can write such code independently.
If he describes a simple application for posting and viewing photos as a user does on Instagram, the system will generate the code needed to build it. This code is sometimes flawed, but under normal circumstances, as long as Singer makes one or two adjustments, he can achieve the effect he wants. “It’s not absolutely perfect,” he said. “But it is very, very close to perfection.”
This behavior is completely new, even the designers of GPT-3 were surprised. They did not build GPT-3 to generate computer code, just as they did not build GPT-3 to write, generate tweets or translate languages like Kaufman did. They built it for only one thing: use a series of words to predict the next word.
GPT-3 is what artificial intelligence researchers call a neural network, which loosely mimics the brainNeural network< /a> The mathematical system. This technology can recognize faces in photos you upload to Facebook and recognize commands you send on your iPhone.
Neural networks learn this skill by finding fixed patterns in large amounts of digital data. For example, by analyzing thousands of photos of cats, it can learn to recognize a cat.
About three years ago, researchers from top laboratories such as Google and OpenAI began to design neural networks that can learn from a huge amount of prose, including unpublished books and thousands of Wikipedia articles. These universal language models are not only suitable for translation tasks, but also for many other tasks.
GPT-3 analyzed digital essays on an unprecedented scale, and spent several months searching for patterns in the massive texts posted on the Internet. In this way, it learns to predict the next word in a sequence. If you enter a few words in GPT-3, it will continue to write automatically and complete your expression with the entire paragraph.
But in the process of acquiring this special skill, it learns more than thismany. During several months of training, GPT-3 has identified more than 175 billion parameters in a large number of books, Wikipedia articles, and other online texts, all of which can have a mathematical representation of a fixed pattern. These patterns are equivalent to maps of human language: mathematical descriptions of the way we put characters together. Whether we write a blog or program a software program, using this map, GPT-3 can perform a variety of tasks, and this may not be its original intention.
Before asking GPT-3 to generate new text, you can focus on the specific patterns that it may learn during training to prepare for the specific tasks of the system. You can enter the description of the smartphone application and the matching Figma code into it. Or you can show a lot of human dialogue. Then, when you start typing, it will complete the sequence in a more specific way. For example, if you guide it with dialogue, it will start chatting with you.
Dario Amodei, vice president of research at OpenAI, said, “It has this emerging quality (emergent quality). It has the ability to recognize the patterns you give it and complete your story. I can give another example at the same time.”
Although the previous language model also works in a similar way, GPT-3 can do things that the previous system could not do, such as writing computer code by yourself. And, perhaps more importantly, you can use a few examples to prepare for a specific task, instead of requiring thousands of examples and hours of extra training like its “predecessors”. Researchers call it “few-shot learning”, and they believe that GPT-3 provides the first real case for humans in this field.
OpenAI chief scientist Ilya Sutskever (Ilya Sutskever) is a key figure in the rise of artificial intelligence technology in the past decade. He said, “It demonstrates an ability that no one thinks can be achieved. Any layman You can use this model to provide these examples in about 5 minutes and get useful behavior from them.”
This is a blessing and a curse.
GPT-3Is it safe?
OpenAI plans to sell GPT-3 access rights via the Internet, turning it into a widely used commercial product. This year, the system was opened to a small number of beta testers through a web browser. Soon after, Jerome Pesenti, the head of Facebook’s artificial intelligence laboratory, called GPT-3 “unsafe” and pointed out that the system was being asked to discuss women, blacks, and Jews.During the war and the Holocaust, harmful language such as sexism and racism will be used.
For systems like GPT-3, this problem is common. Because everyday language itself is prejudiced and often disgusting, especially on the Internet. Because GPT-3 learns from such a language, it also exhibits prejudice and hatred. In the Internet text, GPT-3 learned to associate atheism with the words “cool” and “correct”, and pair Islam with “terrorism”, so it did the same thing.
This may be one of the reasons why OpenAI only shares GPT-3 with a few testers. The laboratory has established a filter to remind people that malicious language may appear, but this approach only treats the symptoms rather than the root cause.
“They not only released GPT-3 publicly, but they are also doing a lot of right things. But there are still many things undecided,” said Allison Koenecke, a researcher at Stanford University.
Liz O’sullivan, vice president of Arthur, which helps companies manage artificial intelligence technology behavior, said OpenAI is ultimately responsible for ensuring that this behavior is controlled. She said that the current situation is that OpenAI is “passing legal and reputation risks to anyone who wants to use the model in consumer-facing applications.”
Other experts worry that these language models may encourage the spread of false information on the Internet, and misleading may help influence online campaigns similar to the 2016 presidential election. GPT-3 points to such a future: we cannot even be sure that what we read is true or false. This applies to Twitter, online conversations, and even long essays.
At the end of July, Liam Porr, a student at the University of California, Berkeley, used GPT-3 to generate several blog posts and posted them on the Internet. 26,000 people read this article. Sixty readers subscribed to this blog after reading it, and only a few suspected that the post was written by a machine.
They are not necessarily easy to be deceived. One of the blog posts argued that if you don’t think too much about everything, you can improve your work efficiency. “In order to accomplish something, maybe we need to think less,” the post began. “This seems counterintuitive, but I believe that sometimes our ideas can hinder the creative process.” This blog post rushed to the top on Hacker News. One. Hacker News is a website where experienced programmers, engineers and entrepreneurs in Silicon Valley rate news articles and other online content.
But like most experiments involving GPT-3, Liam Porr’s experiment is not as powerful as it seems.
The next part will introduce the defects of GPT-3 and its future development direction.
Translator: Jane
Recommended reading: GPT-3, the strongest AI model for natural language processing: How many possibilities are there in the future? (Next)