Advanced AI solutions like GPT-4 write text that is indistinguishably human-sounding. They do so well in language performing tasks, that it has become increasingly difficult to distinguish whether you are talking with a human or a machine.
Imagine Alan Turing's famous thought experiment. He said that a machine could be described as thinking if it can make us believe we are talking to another human being, just based on their responses.
Testing Modern AI
She and colleagues in the Department of Cognitive Science at UC San Diego set out to test how well AI fared on those tasks. They ran the tests on three AI: ELIZA (an ancient 1960s chatbot), GPT-3, 5, and GPT-4. The system arealistit also uses in a controlled Turing Test) where people talk for five minutes with either human or Assistant. Then they had to guess whether their interlocutor was human.
Setting Up the Test
The test set was GPT-4 and Inference: evaluting it on the few shot examples. This 5 has been called out specifically to act as something. Act as a teenager who was short, didn´t care about the conversation much-used slang and spelt things wrong even sometimes. It also included a little info about the game and what some of we at Rare have been up to during the past week or so. To further the impression of a natural conversation, there was an additional delay between characters sent in response by AI.
Study Results
The study divided 500 participants into five groups. Either through a human-like but essentially automated chat interface that had been programmed in or with one of the AI systems, they each chatted to someone via what looked like your standard messaging app. Five minutes later they guessed whether or not the human was a robot, and provided their reason.
The results were intriguing. GPT-4 was 54% mistaken for human, while GPT-3 The number 5 was falsely identified half the time. ELIZA only fooled people into thinking she was human 22% of the time. We unsuccessfully managed to identify real humans around 67% accurate. It stands to reason that an individual would have hardly been better off getting something wrong when speaking with GPT-4, than hitting the nail on the head; this is a testament as go just how far these AI systems had advanced.
Key Findings
The study revealed that people often relied on the style of language, emotional cues, and knowledge-based questions to decide if they were talking to a human or a machine. These factors played a big role in their guesses.
Conclusion
As AI continues to improve, it’s becoming increasingly difficult to distinguish between human and machine interactions. This development brings us closer to a future where AI can seamlessly integrate into our daily lives, raising important questions about the role of AI in society.