Researchers from Stanford University paid 1,052 people $60 to read the first two lines. The Great Gatsby Go to the app When finished, the AI, which looks like a 2D sprite from a SNES-era Final Fantasy game, asks participants to tell the story of their lives. Scientists took those interviews and created an AI that they say replicated the participants' behavior with 85% accuracy.
study of A representative generation simulation of 1,000 people.It's a joint venture between Stanford and scientists working for Google's DeepMind AI research lab. The point is that creating AI agents from random people can help policymakers and businesspeople better understand the public. Why use focus groups or public opinion surveys? Since you can talk to them once, form an LLM from that conversation. Then there will be their thoughts and opinions forever. Or at least as close to those thoughts and feelings as LLM can recreate.
“This work lays the foundation for new tools. that can help monitor individual and collective behavior,” the report's abstract said.
“For example, Diverse groups of people will respond to new public health policies and messages. Respond to product launches or how to respond to a major shock,” the paper continues. “When simulated individuals are put together into groups, These simulations can help pilot interventions. Develop complex theories to capture causal interactions and appropriate context. and expand our understanding of structures such as institutions and networks in areas such as economics, sociology, organizations, and political science.”
All possibilities are based on a two-hour interview in LLM that answers questions mostly like in real life.
Most of the process is automated. The researchers hired Bovitz, a market research company, to gather participants. The goal was to obtain as broad a sample of the U.S. population as possible when limited to 1,000 people to complete the study. The user registers an account in a specially created interface. Create a 2D sprite avatar and start chatting with the AI interviewer.
The interview questions and format are a modified version used by American Voice Project A joint project of Stanford University and Princeton University that is interviewing people across the country.
Each interview began with the participant reading the first two lines of The Great Gatsby (“In my youth and weakness, Dad gave me some advice that I've kept in mind ever since. 'Whenever you feel like criticizing anyone,' he told me, 'just remember that everyone in The world doesn't have the advantages you have'”) is a way of calibrating sound.
According to the report, “The interview interface displays a 2D avatar sprite representing the interviewer agent at its center. with the participant's avatar displayed at the bottom. Walking towards the goal post to mark progress. When the AI interviewer agent is speaking, it is signaled by a rhythmic animation of the center circle along with the interviewer avatar.”
The average two-hour interview was transcribed 6,491 words, asking questions about race, gender, politics, income, and social media use. work stress and the appearance of the family The researchers published the interview script and the questions the AI asked.
Those transcripts, which are less than 10,000 words each, are fed into another LLM that researchers use to extract generative agents to simulate participants. The researchers then put both the participants and the AI clone through additional economic questions and games to see how they would compare. “When inquiring about the representative All interview recordings are inserted into the model prompt. to instruct the model to imitate people based on their interview data,” the report said.
This part of the process is closest to control. Researchers have used General Social Survey (GSS) and Big Five Personality Inventory (BFI) to test how well the LLM matches their aspirations. Participants and LLM were then put through five economics games to see how they would compare.
The results were mixed. The AI agents answered approximately 85% of the questions as did the real participants in the GSS. They hit 80% of the BFI. However, the numbers dropped as the agents began to play economic games. Researchers offered real-life participants cash prizes for playing games such as Prisoner's Dilemma and Dictator game–
In Prisoner's Dilemma, participants can choose to work together and either succeed or defeat their partner for the chance to win the big prize. In the dictator game Participants must choose how to allocate resources to other participants. Real-life subjects received $60 more for playing these things.
When faced with these economic games, human AI clones do not replicate their real-world counterparts either. “On average The generative agent achieves a normal correlation of 0.66” or approximately 60%.
The entire document is worth reading if you're interested in what academics think about AI agents and the public. It didn't take long for researchers to distill human personalities into LLMs that behave similarly. with time and energy They might bring the two closer together.
This makes me worried. It's not because I don't want to see the indescribable human spirit reduced to a spreadsheet. But because I know this type of technology will be used for sickness, we've seen even dumber LLMs practiced in public records tricking Grandma into giving her banking information to an AI relative after a quick phone call. What happens when those machines have scripts? What happens when they have access to purpose-built personas based on social media activity and other publicly available data?
What happens when a company or politician decides that people want and need something that is not based on their stated will? But it depends on that estimation.