A new study from Harvard Business School shows that GPT-4 can improve the quality of an employee’s work by more than 40%. In contrast, poorly used AI can degrade performance by 19 percentage points. Employees treat AI either like “centaurs” or like “cyborgs”: like a sidekick or an integrated ally.
According to a new study from Harvard Business School, consultants who use OpenAI’s GPT-4 language model are “significantly more productive and deliver significantly higher quality results” than those who don’t use it. The study compared the performance of BCG advisors to those who used AI and those who did not. Researchers found that consultants who used GPT-4 produced 40 percent better quality of work, according to the paper “Navigating the Jagged Technological Frontier” by researchers from Harvard, MIT, Wharton, BCG and Warwick Business School.
Those who used GPT-4 saw a 25% increase in speed and a 12% increase in task completion rate. According to the study, consultants of all skill levels benefited from the use of AI, but the lowest performers saw the largest increase: 43% compared to 17% for the highest performers.
Although the quality of the work has improved significantly, the results have also become less variable. “Although GPT-4 helps generate superior content, it could lead to more consistent results,” the researchers said. These results are tasks “at the limit” of GPT-4, meaning they can easily be completed by the AI. As part of the experiment, consultants were tasked with innovating and developing creative products – that is, coming up with new beverage concepts, selecting the most viable product and creating a plan for market launch.
An abrupt border
The researchers also tested “out-of-bounds” tasks, i.e. those that the AI model would have difficulty handling. The article is about the ability to recognize subtle clues in interview notes that accompany tabular data.
As part of the experiment, consultants were asked to develop concrete strategies to boost a company’s activities. Her job was to analyze the performance of the company’s sales channels through interviews and financial data and then make recommendations to the CEO. The performance measurement was to suggest the right strategy. In this case, advisors who used AI were 19 percentage points less likely to succeed. “Professionals who experienced negative results when using AI tended to blindly accept the results and analyze them less,” observe the authors.
Why is that important? Professionals who use AI need to know how to use this technology skillfully so that it bears fruit. Otherwise, using AI for tasks it is not suited for can actually harm employee performance. “Beyond the limit, AI results are inaccurate, less useful, and detrimental to human performance,” the authors write. They call this unequal performance the “jagged technological frontier.”
The Harvard study involved 758 consultants, or about 7% of BCG’s workforce. The tasks analyzed cover the daily work of a consultant and include creativity, analytical thinking, writing skills and persuasion.
Centaurs and cyborgs
The study authors classified AI users into two groups: “centaurs,” who distribute and delegate tasks between themselves and the AI, and “cyborgs,” who integrate their workflow into the AI. “AI.”
The method centaur takes its name from the mythical half-human, half-horse creature, although researchers believe AI could also be a mix of humans and machines. “Users adopting this strategy switch between human and AI tasks, distributing responsibilities based on the strengths and capabilities of each unit,” the report said. “They identify which tasks are best suited for human intervention and which can be effectively completed by AI.”
On the other hand, for them Cyborgs, named after the human-machine hybrids featured in science fiction literature, this approach was more about integrations, such as asking the AI to complete a job it started. “Cyborg users don’t just delegate tasks; they combine their efforts with those of AI, at the very edge of possibility,” they write. “This strategy can manifest itself by alternating responsibilities at the sub-task level, for example by throwing away a sentence for the AI to complete, or by collaborating with the AI.”
The concept of centaurs and cyborgs shows that there are multiple paths to effective AI collaboration and that adopting the right approach for the right use case can help spread the benefits of AI more broadly. AI. Different use cases require different requirements, and adopting a centaur or cyborg approach can be part of individual integration strategies that companies can tailor to their specific needs and capabilities.
“It is clear that the best approaches to leveraging AI are not fully understood and need to be studied in depth by scientists and practitioners,” the researchers write. But the immediate risk is that employers will stop delegating tasks that AI is good at from junior employees, “leading to long-term training gaps,” they added. Expertise must be developed through formal education, on-the-job training, and workforce development.
The release of Large Language Models (LLM) has sparked great interest in how humans will use artificial intelligence (AI) to accomplish a variety of tasks. In our study with Boston Consulting Group, a global management consulting firm, we examine the performance impact of AI for realistic, complex, and knowledge-intensive tasks. The previously recorded experiment involved 758 consultants, about 7% of the company’s individual consultants. After establishing baseline performance on a similar task, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-AI access. 4 with an overview of the technology.
We propose that AI capabilities create a “see-saw technological frontier” where some tasks can be easily accomplished by AI, while others, although seemingly of a similar level of difficulty, are beyond the reach of current AI capabilities. In each of the 18 realistic consulting tasks at the edge of AI capabilities, consultants who used AI were significantly more productive (completing 12.2% more tasks on average and 25.1% fewer tasks) and delivered significantly higher quality results (more than 40%). higher quality compared to a control group). Consultants of all skill groups benefited significantly from the AI increase: those below the average performance threshold increased by 43% and those above the threshold by 43%, i.e. 17% compared to their own values. For a task selected as out of bounds, consultants who used AI were 19 percentage points less likely to provide correct solutions than consultants who did not use AI.
Furthermore, our analysis shows the emergence of two different patterns of successful AI use by humans along a spectrum of human-AI integration. One group of consultants behaved like “centaurs,” like the mythical half-horse, half-human creature, dividing their solution-creation activities and delegating them to the AI or themselves. Another group of consultants behaved more like “cyborgs.” , by fully integrating their workflows with AI and continuously interacting with the technology.
Source: Harvard Business School
And you ?
Do you think this study is credible or relevant?
How do you assess the impact of AI on work performance?
See also:
According to a KPMG survey, generative artificial intelligence will revolutionize workplace efficiency. 72% of respondents said it will significantly improve productivity
It has almost doubled our workload: AI is supposed to make tasks easier. These workers disagree. When AI complicates work instead of simplifying it
According to an Adobe study, 57% of respondents say AI can increase creativity