AI Beats Average Humans in Creativity Tests, Study Shows

A groundbreaking study has revealed that advanced generative AI systems can outperform the average human in specific creativity tests. Conducted by a team at the Université de Montréal and led by Professor Karim Jerbi, the research assessed over 100,000 participants against several large language models, including GPT-4, ChatGPT, and Claude. The results indicate that while AI has made significant strides, it still cannot match the peak creativity of the most skilled human creators.

The study represents the largest direct comparison between human and AI creativity to date. It highlights a notable transition in the capabilities of AI, with generative models like GPT-4 demonstrating superior performance on tasks designed to evaluate original thinking and idea generation. According to Jerbi, “Our study shows that some AI systems based on large language models can now outperform average human creativity on well-defined tasks.”

Methodology and Findings

To evaluate creativity effectively, the research team employed the Divergent Association Task (DAT). This psychological test measures divergent creativity, focusing on an individual’s ability to generate diverse ideas from a single prompt. Participants were asked to list ten nouns that are as unrelated as possible, providing a clear measure of creative thought.

The findings were striking; while AI systems occasionally surpassed the performance of average humans, particularly on simpler tasks, the most creative 10% of humans consistently outperformed even the most advanced AI. This gap was particularly evident in richer creative endeavors, such as poetry and storytelling. The study underscores that while generative AI has reached a level where it can compete with the average human, the highest echelons of creativity remain distinctly human.

Exploring the Depth of AI Creativity

The researchers also sought to determine whether AI’s success on simpler tasks translates to more complex creative activities. They tested AI systems against human participants on creative writing challenges, including composing haikus and writing movie plot summaries. The results mirrored earlier findings; while AI could match or exceed the average human’s output, the top human creators delivered more nuanced and original work.

An important aspect of the study involved assessing whether AI creativity is fixed or can be modified. The researchers concluded that creativity in AI can be adapted by adjusting technical settings, such as the model’s temperature, which influences the predictability of generated responses. Higher temperature settings produced more varied and adventurous outputs, while lower settings resulted in more conservative responses. The study also found that the way instructions are formulated plays a significant role in shaping AI creativity, emphasizing the necessity of human guidance in the creative process.

The implications of this research raise questions about the role of AI in creative sectors. While generative AI has shown it can reach human-level creativity in specific contexts, it relies heavily on human input and direction. Jerbi cautions against viewing AI as a competitor in creative fields, stating, “Even though AI can now reach human-level creativity on certain tests, we need to move beyond this misleading sense of competition.”

Rather than threatening creative professions, the study suggests a future in which AI acts as a collaborator, enhancing human creativity rather than replacing it. By providing new perspectives and ideas, AI could amplify imagination and open up new avenues for creative exploration.

The detailed research findings are published in the journal Scientific Reports under the title “Divergent creativity in humans and large language models.” This study marks a significant step in understanding the relationship between human creativity and artificial intelligence, positioning AI as an invaluable tool for creative professionals.