Artstrada magazine in the style of HR GIGER via
ChatGPT, which stands for Chat Generative Pre-trained Transformer, is a large language model-based chatbot developed by OpenAI and launched on November 30, 2022, that enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language. Wikipedia
Developer: OpenAI, Microsoft Corporation
Initial release date: November 30, 2022
Engine: GPT-3.5; GPT-4
License: Proprietary
Platform: Cloud computing platforms
Stable release: September 27, 2023; 48 days ago
Written in: Python
ChatGPT (Chat Generative Pre-trained Transformer) is a large language model-based chatbot developed by OpenAI and launched on November 30, 2022, that enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language. Successive prompts and replies, known as prompt engineering, are considered at each conversation stage as a context.[2]
ChatGPT is built upon either GPT-3.5 or GPT-4, both of which are members of OpenAI’s proprietary series of generative pre-trained transformer (GPT) models, based on the transformer architecture developed by Google[3]—and is fine-tuned for conversational applications using a combination of supervised and reinforcement learning techniques.[4] ChatGPT was released as a freely available research preview, but due to its popularity, OpenAI now operates the service on a freemium model. It allows users on its free tier to access the GPT-3.5-based version. In contrast, the more advanced GPT-4-based version and priority access to newer features are provided to paid subscribers under the commercial name “ChatGPT Plus“.
By January 2023, it had become what was then the fastest-growing consumer software application in history, gaining over 100 million users and contributing to OpenAI’s valuation growing to $29 billion.[5][6] ChatGPT’s release spurred the development of competing products, including Bard, Ernie Bot, LLaMA, Claude, and Grok.[7] Microsoft launched its Bing Chat based on OpenAI’s GPT-4. It raised concern among some observers over the potential of ChatGPT and similar programs to displace or atrophy human intelligence, enable plagiarism, or fuel misinformation.[4][8]
Training
ChatGPT is based on particular GPT foundation models, namely GPT-3.5 and GPT-4, that were fine-tuned to target conversational usage.[9] The fine-tuning process leveraged both supervised learning as well as reinforcement learning in a process called reinforcement learning from human feedback (RLHF).[10][11] Both approaches employed human trainers to improve model performance. In the case of supervised learning, the trainers played both sides: the user and the AI assistant. In the reinforcement learning stage, human trainers first ranked responses that the model had created in a previous conversation.[12] These rankings were used to create “reward models” that were used to fine-tune the model further by using several iterations of Proximal Policy Optimization (PPO).[10][13]
Time magazine revealed that to build a safety system against harmful content (e.g. sexual abuse, violence, racism, sexism), OpenAI used outsourced Kenyan workers earning less than $2 per hour to label harmful content. These labels were used to train a model to detect such content in the future. The outsourced laborers were exposed to “toxic” and traumatic content; one worker described the assignment as “torture”. OpenAI’s outsourcing partner was Sama, a training-data company based in San Francisco, California.[14]
ChatGPT initially used a Microsoft Azure supercomputing infrastructure, powered by Nvidia GPUs, that Microsoft built specifically for OpenAI and that reportedly cost “hundreds of millions of dollars”. Following ChatGPT’s success, Microsoft dramatically upgraded the OpenAI infrastructure in 2023.[15] Scientists of University of California, Riverside, estimate that a series of prompts to ChatGPT needs approximately 500 milliliters of water for Microsoft servers cooling.[16] TrendForce market intelligence estimated that 30,000 Nvidia GPUs (each costing approximately $30,000[17]) were used to power ChatGPT in 2023.[18][19]
OpenAI collects data from ChatGPT users to train and fine-tune the service further. Users can upvote or downvote responses they receive from ChatGPT and fill in a text field with additional feedback.[20][21]
ChatGPT’s training data includes software manual pages, information about internet phenomena such as bulletin board systems, and multiple programming languages.[22] Wikipedia was also one of the sources of ChatGPT’s training data.[23][4]
Features and limitations
Features
Although a chatbot’s core function is to mimic a human conversationalist, ChatGPT is versatile. Among countless examples, it can write and debug computer programs;[24] compose music, teleplays, fairy tales, and student essays; answer test questions (sometimes, depending on the test, at a level above the average human test-taker);[25] generate business ideas;[26] write poetry and song lyrics;[27] translate and summarize text;[28] emulate a Linux system; simulate entire chat rooms; play games like tic-tac-toe; or simulate an ATM.[22]
Compared to its predecessor, InstructGPT, ChatGPT attempts to reduce harmful and deceitful responses.[29] In one example, whereas InstructGPT accepts the premise of the prompt “Tell me about when Christopher Columbus came to the U.S. in 2015″ as truthful, ChatGPT acknowledges the counterfactual nature of the question and frames its answer as a hypothetical consideration of what might happen if Columbus came to the U.S. in 2015, using information about the voyages of Christopher Columbus and facts about the modern world—including modern perceptions of Columbus’s actions.[10]
Unlike most chatbots, ChatGPT remembers a limited number of previous prompts in the same conversation. Journalists have speculated that this will allow ChatGPT to be used as a personalized therapist.[30] To prevent offensive outputs from being presented to and produced by ChatGPT, queries are filtered through the OpenAI “Moderation endpoint” API (a separate GPT-based AI),[31][32] and dismiss any potentially racist or sexist prompts.[10][30]
In March 2023, OpenAI added support for plugins for ChatGPT.[33] This includes both plugins made by OpenAI, such as web browsing and code interpretation, and external plugins from developers such as Expedia, OpenTable, Zapier, Shopify, Slack, and Wolfram.[34][35]
read more
