ChatGPT: why is everyone talking about it?
La ChatGPT released on November 30 marked a new stage in the development of language processing technologies. It is in any case the first time that an AI system, directly resulting from research, has aroused such enthusiasm: numerous articles have appeared in the specialized but also general press. ChatGPT (or OpenAI, the company that develops ChatGPT, or both) are consistently trending on Twitter even today.
Why such an echo? Are ChatGPT's capabilities a departure from previous technologies capable of generating text?
ChatGPT can chat
Clearly, ChatGPT has allowed a qualitative leap: it is able to answer, in an often relevant way, almost any request in natural language. GPT2, then GPT3 - the previous language models developed by OpenAI - were already very good at generating text from a few words or sentences given as a primer (what is called the "prompt"): we often spoke of “parrots” (stochastic parrots), as proposed Emily Bender, and many others researchers reviews of these techniques. Indeed, one could say that these systems produced text from bits of sentences drawn from the immense corpora at their disposal... even if this image of the parrot must be qualified: the systems do not repeat extracts word for word, but produce in general an original text by taking up, paraphrasing, combining elements seen in various corpora.
ChatGPT takes up these characteristics, but the “chat” part (dialogue) adds a completely different dimension, and often seemingly stunning.
The system no longer produces just a paragraph from a sentence given as input. He is now able to answer both simple and complex questions correctly, he can provide elements of reasoning, speak different languages, analyze or produce computer code, among other things.
For example, if asked if two characters were able to meet, the system is able to determine the dates corresponding to their existence, compare these dates and deduce an answer. This is trivial for a human, but no system until now could answer this type of question so precisely without being highly specialized. Beyond the language, ChatGPT can also provide elements of mathematical reasoning (but he is often wrong) and analyze computer code including.
From this point of view, ChatGPT is a step ahead of its competitors.
ChatGPT, a closed system
Regarding the operation of the system, difficult to say more, because OpenAI, despite its name, develops closed systems. That is to say, the computer code (source code) used is not available and the research related to ChatGPT remains for the moment largely a trade secret – even if, obviously, ofother teams are working on similar systems, also based on deep learning. So we have an idea of what is used by OpenAI.
Other companies have kept a more open model, for example Meta with the work carried out at FAIR, usually open and published in major conferences in the field. But, more generally, we are increasingly seeing a closure of research. For example, while Apple has always had a development model private and very secret, Deepmind had an open model and is probably a little less so, now that the company is under the control of Google.
The ChatGPT system itself could go commercial: OpenAI is funded by Microsoft who could decide to close access to the system one day soon, if it is in his interest.
By testing ChatGPT, you contribute (for free) to improve it
In the meantime, OpenAI benefits from the enormous publicity that its tool brings to it, and also from all the user interactions with it. If a user reports that an answer is not good, or asks the tool to reformulate his answer taking into account an additional element, this is as much information as the system stores and can reuse to refine its answer the next time, on the request asked or on a similar request. By testing ChatGPT, we work for free for OpenAI !
To come back to the system itself, the dialogue part is therefore what makes the strength and particularity of ChatGPT (compared to GPT3 for example). It is no longer a "simple" language model capable of generating text "by the kilometer", but a real dialogue system. This one has probably benefited from millions or billions of examples assessed by humans, and the current phase – where hundreds of thousands of users “play” almost free with the system – still allows it to be continuously improved, since all interactions are obviously recorded and used for this.
It is now quite simple to access corpora of billions of words to develop a “GPT” type language model, at least for languages that are widespread on the Internet.
But the data that allowed the development of ChatGPT (dialogues, interactions with humans) are not public, and this is what gives OpenAI a significant advantage over the competition.
For example, Google has different data, but no doubt also usable for this type of system - especially since Google has been developing for several years a knowledge graph which could generate responses with better reliability than ChatGPT. In particular, the analysis of the sequences of queries from Google's search engine could provide valuable information to guide interaction with the user... But, in the meantime, it is OpenAI that has ChatGPT, and not Google: OpenAi has the advantage.
Beyond ChatGPT, what applications in the future?
In fact, although it is possible to bypass ChatGPT limits, the system refuses to explain how to create a bomb, produce erotic tales or give her feelings (ChatGPT frequently responds that she is a machine, thatshe has no feelings or personality). OpenAI has obviously taken care of its communication. The company has also taken extreme care to “armor” the system, to allow it to thwart most of the pitfalls that can ruin this type of application in a few hours, as often happens for systems open to the general public.
We can, for example, cite Meta, which last November had to withdraw its system called “Galactica” two days after putting it online. Galactica had been trained in the scientific field and designed to offer services to researchers. He was first introduced as able to write scientific papers automatically from a prompt…before the company clarified that it was obviously just an editorial aid. But this unfortunate communication strategy sparked a controversy that forced Meta to quickly unplug Galactica.
Conversely, ChatGPT is still online, and still arouses as much passion among a large audience. Questions remain, however: what impact will ChatGPT have? What applications will be derived from it? And what economic model is the OpenAI company (and Microsoft, its main investor) aiming for?
Thierry poibeau, DR CNRS, École normale supérieure (ENS) - PSL
This article is republished from The Conversation under Creative Commons license. Read theoriginal article.
