More-powerful AI is coming. Academia and industry must oversee it — together

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
You have full access to this article via your institution.

Sam Altman of OpenAI, the company that developed ChatGPT, says that machines will create superintelligence. Research is needed to verify these and other claims.Credit: Jason Redmond/AFP/Getty
“It is possible that we will have superintelligence in a few thousand days (!); it may take longer, but I’m confident we’ll get there.”
So wrote Sam Altman, the chief executive of OpenAI, a technology firm based in San Francisco, California, on 23 September. This was less than two weeks after the company behind ChatGPT released o1, its most advanced large language model (LLM) yet. Once confined to the realms of science fiction, the rise of LLMs in the past few years has renewed the relevance of the question of when we might create artificial general intelligence (AGI). Although it lacks a precise definition, AGI broadly refers to an AI system capable of human-level reasoning, generalization, planning and autonomy.

How close is AI to human-level intelligence?

How close is AI to human-level intelligence?
Policymakers everywhere have questions about AGI, including what its benefits and risks will be. These are not easy questions to answer, especially given that much of the work is happening in the private sector, in which studies are not always published openly. But what is clear is that AI companies are laser-focused on giving their systems the whole range of cognitive abilities enjoyed by humans. Companies developing AI models have a strong incentive to maintain the idea that AGI is nigh, to attract interest and therefore investment.
There was a consensus among researchers who spoke to Nature for a News Feature published this week (see Nature 636, 22–25; 2024) that large language models (LLMs), such as o1, Google’s Gemini and Claude, made by Anthropic, based in San Francisco, have not yet delivered AGI. And, informed by insights from neuroscience, many say that there are good reasons to think that LLMs never will, and that another technology will be needed for AI to achieve human-level intelligence.
Despite their breadth of ability — from generating computer code to summarizing academic articles and answering mathematics questions — there are fundamental limitations in the way the most powerful LLMs work, which involves essentially devouring a mass of data and using that to predict the next ‘token’ in a series. That generates plausible answers to a problem, rather than actually solving it.

AI and science: what 1,600 researchers think

AI and science: what 1,600 researchers think
François Chollet, a software engineer formerly at Google, based in Mountain View, California, and Subbarao Kambhampati, a computer scientist at Arizona State University in Tempe, have tested o1’s performance on tasks that require abstract reasoning and planning, and found that it comes up short of AGI. If AGI is to happen, some researchers think that AI systems would need coherent ‘world models’, or representations of their surroundings that they can use to test hypotheses, reason, plan and generalize knowledge learnt in one domain to potentially limitless other situations.
This is where ideas from neuroscience and cognitive science could propel the next breakthroughs. Yoshua Bengio’s team at the University of Montreal, Canada, for example, is exploring alternative AI architectures that would better support the building of coherent world models and the ability to reason using such models.
Some researchers argue that the next breakthroughs in AI might come not from the biggest systems, but from smaller, more-energy-efficient AI. Smarter systems in the future could also require less data to train if they had the ability to decide which aspects of their environment to sample, rather than simply ingesting everything they are fed, says Karl Friston, a theoretical neuroscientist at University College London.
Such work demonstrates that researchers from an array of fields need to be involved in AI development. This will be necessary to verify what systems are truly capable of, ensure that they live up to the technology companies’ claims and identify the breakthroughs needed for development. However, right now, accessing the leading AI systems can be difficult for researchers who don’t work at the companies that can afford the vast quantity of graphical processing units (GPUs) needed to train the systems (A. Khandelwal et al. Preprint at arXiv https://doi.org/nt67; 2024).

ChatGPT broke the Turing test — the race is on for new ways to assess AI

ChatGPT broke the Turing test — the race is on for new ways to assess AI
To give a sense of the scale of activity, in 2021, US government agencies (excluding the Department of Defense) allocated US$1.5 billion to AI research and development, and the European Commission spends around €1 billion (US$1.05 billion) annually. By contrast, companies worldwide spent more than $340 billion on AI research in 2021 (N. Ahmed et al. Science 379, 884–886; 2023). There are ways that governments could fund AI research on a bigger scale, for example by pooling resources. The Confederation of Laboratories for Artificial Intelligence Research in Europe, a non-profit organization based in The Hague, the Netherlands, has suggested building a ‘CERN for AI’ that can attract the same level of talent as AI companies do and so create a cutting-edge research environment.
It is hard to predict when AGI might arrive — estimates range from a few years from now to a decade or longer. But more huge advances in AI will certainly happen, and many of them will probably come from industry, considering the scale of investment. To ensure that these advances are beneficial, the research from technology companies needs to be verified using the best current understanding of what constitutes human intelligence, according to neuroscience, cognitive science, social science and other relevant fields. This publicly funded research needs to have a key role in AGI’s development.
Humanity needs to bring all knowledge to bear so that applications of AI research are robust and its risks are mitigated as much as possible. Governments, companies, research funders and researchers need to recognize their complementary strengths. If they do not, then insights that could help to improve AI will be missed — and the resulting systems risk being unpredictable and therefore unsafe.
Nature 636, 273 (2024)
doi: https://doi.org/10.1038/d41586-024-03911-3
Reprints and permissions
How close is AI to human-level intelligence?
AI and science: what 1,600 researchers think
AI firms must play fair when they use academic data in training
What the EU’s tough AI law means for research and ChatGPT
ChatGPT broke the Turing test — the race is on for new ways to assess AI
Better data sets won’t solve the problem — we need AI for Africa to be developed in Africa
World View 11 DEC 24
The AI revolution is running out of data. What can researchers do?
News Feature 11 DEC 24
What should we do if AI becomes conscious? These scientists say it’s time for a plan
News 10 DEC 24
How AI is unlocking ancient texts — and could rewrite history
News Feature 30 DEC 24
Spatial transcriptomic clocks reveal cell proximity effects in brain ageing
Article 18 DEC 24
Will artificial intelligence help or hinder progress on the SDGs?
Nature Index 11 DEC 24
Better data sets won’t solve the problem — we need AI for Africa to be developed in Africa
World View 11 DEC 24
The AI revolution is running out of data. What can researchers do?
News Feature 11 DEC 24
Stop digital platforms treating people as lab rats
Correspondence 10 DEC 24
Human Technopole (HT) is an interdisciplinary research institute in Milan (Italy) created and supported by the Italian government to develop innova…
Milan (IT)
Human Technopole
Postdoctor in tumor immunity Reference number PAR 2024/1129 The University of Gothenburg tackles society’s challenges with diverse knowledge. 56 00…
Gothenburg (Stad), Västra Götaland (SE)
University of Gothenburg
Founded in 2018, Westlake University is a new type of non-profit research-oriented university in Hangzhou, China, supported by public a…
Hangzhou, Zhejiang, China
Westlake University
Job advertisement The University of Veterinary Medicine Hannover (TiHo) is inviting applications for a W3 Professorship in "Buiatrics" (m/f/d), whi…
30559, Hannover (DE)
Stiftung Tierärztliche Hochschule Hannover
Developing methods for low sample inputs for proteomics in high resolution mass spectrometry, with applications in neuromuscular diseases
Dortmund, Nordrhein-Westfalen (DE)
Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V.
You have full access to this article via your institution.