WhenWillAIExceedHumanPerformance
EvidencefromAIExperts
KatjaGrace1,2,JohnSalvatier2,AllanDafoe1,3,BaobaoZhang3,andOwainEvans1
1FutureofHumanityInstitute,OxfordUniversity
2AIImpacts
3DepartmentofPoliticalScience,YaleUniversity
Abstract
Advancesinarticialintelligence(AI)willtransformmodernlifebyreshapingtransportation,
health,science,nance,andthemilitary[1,2,3].Toadaptpublicpolicy,weneedtobetter
anticipatetheseadvances[4,5].Herewereporttheresultsfromalargesurveyofmachine
learningresearchersontheirbeliefsaboutprogressinAI.ResearcherspredictAIwilloutper-
formhumansinmanyactivitiesinthenexttenyears,suchastranslatinglanguages(by2024),
writinghigh-schoolessays(by2026),drivingatruck(by2027),workinginretail(by2031),
writingabestsellingbook(by2049),andworkingasasurgeon(by2053).Researchersbelieve
thereisa50%chanceofAIoutperforminghumansinalltasksin45yearsandofautomating
allhumanjobsin120years,withAsianrespondentsexpectingthesedatesmuchsoonerthan
NorthAmericans.Theseresultswillinformdiscussionamongstresearchersandpolicymakers
aboutanticipatingandmanagingtrendsinAI.
Introduction
Advancesinarticialintelligence(AI)willhavemassivesocialconsequences.Self-drivingtech-
nologymightreplacemillionsofdrivingjobsoverthecomingdecade.Inadditiontopossible
unemployment,thetransitionwillbringnewchallenges,suchasrebuildinginfrastructure,pro-
tectingvehiclecyber-security,andadaptinglawsandregulations[5].Newchallenges,bothforAI
developersandpolicy-makers,willalsoarisefromapplicationsinlawenforcement,militarytech-
nology,andmarketing[6]repareforthesechallenges,accurateforecastingoftransformative
AIwouldbeinvaluable.
SeveralsourcesprovideobjectiveevidenceaboutfutureAIadvances:trendsincomputing
hardware[7],taskperformance[8],andtheautomationoflabor[9].ThepredictionsofAIexperts
providecrucialadditionalinformation.WesurveyalargerandmorerepresentativesampleofAI
expertsthananystudytodate[10,11].OurquestionscoverthetimingofAIadvances(including
bothpracticalapplicationsofAIandtheautomationofvarioushumanjobs),aswellasthesocial
andethicalimpactsofAI.
SurveyMethod
Oursurveypopulationwasallresearcherswhopublishedatthe2015NIPSandICMLconfer-
ences(twoofthepremiervenuesforpeer-reviewedresearchinmachinelearning).Atotalof352
researchersrespondedtooursurveyinvitation(21%ofthe1634authorswecontacted).Ourques-
tionsconcernedthetimingofspecicAIcapabilities(e.g.foldinglaundry,languagetranslation),
superiorityatspecicoccupations(e.g.truckdriver,surgeon),superiorityoverhumansatalltasks,
andthesocialimpactsofadvancedAI.SeeSurveyContentfordetails.
TimeUntilMachinesOutperformHumans
AIwouldhaveprofoundsocialconsequencesifalltasksweremorecosteectivelyaccomplishedby
machines.Oursurveyusedthefollowingdenition:
“High-levelmachineintelligence”(HLMI)isachievedwhenunaidedmachinescanac-
complisheverytaskbetterandmorecheaplythanhumanworkers.a
rX
iv
:1
75
.087
v[c
s
.A
I]
2M
a
y
21EachindividualrespondentestimatedtheprobabilityofHLMIarrivinginfutureyears.Takingthe
meanovereachindividual,theaggregateforecastgavea50%chanceofHLMIoccurringwithin
45yearsanda10%chanceofitoccurringwithin9years.Figure1displaystheprobabilistic
predictionsforarandomsubsetofindividuals,aswellasthemeanpredictions.Thereislarge
inter-subjectvariation:Figure3showsthatAsianrespondentsexpectHLMIin30years,whereas
NorthAmericansexpectitin74years.
0.00
0.25
0.50
0.75
1.00
0255075100
Years from 2016
P
ro
b
ab
ili
ty
o
f
H
L
M
I
Aggregate Forecast (with 95% Confidence Interval)
Random Subset of Individual Forecasts
LOESS
Figure1:Aggregatesubjectiveprobabilityof‘high-levelmachineintelligence’arrivalby
futureyears.Eachrespondentprovidedthreedatapointsfortheirforecastandthesewerettothe
GammaCDFbyleastsquarestoproducethegreyCDFs.The“AggregateForecast”isthemeandistribution
overallindividualCDFs(alsocalledthe“mixture”distribution).Thecondenceintervalwasgenerated
bybootstrapping(clusteringonrespondents)andplottingthe95%intervalforestimatedprobabilitiesat
eachyear.TheLOESScurveisanon-parametricregressiononalldatapoints.
WhilemostparticipantswereaskedaboutHLMI,asubsetwereaskedalogicallysimilarquestion
thatemphasizedconsequencesforemployment.Thequestiondenedfullautomationoflaboras:
whenalloccupationsarefullyautomatable.Thatis,whenforanyoccupation,machines
couldbebuilttocarryoutthetaskbetterandmorecheaplythanhumanworkers.
ForecastsforfullautomationoflaborweremuchlaterthanforHLMI:themeanoftheindividual
beliefsassigneda50%probabilityin122yearsfromnowanda10%probabilityin20years.Figure2:TimelineofMedianEstimates(with50%intervals)forAIAchievingHumanPer-
formance.Timelinesshowing50%probabilityintervalsforachievingselectedAImilestones.Specically,
intervalsrepresentthedaterangefromthe25%to75%probabilityoftheeventoccurring,calculatedfrom
themeanofindividualCDFsasinFig.1.Circlesdenotethe50%-probabilityyear.Eachmilestoneisfor
AItoachieveorsurpasshumanexpert/professionalperformance(fulldescriptionsinTableS5).Notethat
theseintervalsrepresenttheuncertaintyofsurveyrespondents,notestimationuncertainty.
Respondentswerealsoaskedwhen32“milestones”forAIwouldbecomefeasible.Thefullde-
scriptionsofthemilestoneareinTableS5.Eachmilestonewasconsideredbyarandomsubsetof
respondents(n≥24).Respondentsexpected(meanprobabilityof50%)20ofthe32AImilestones
tobereachedwithintenyears.Fig.2displaystimelinesforasubsetofmilestones.
IntelligenceExplosion,Outcomes,AISafety
TheprospectofadvancesinAIraisesimportantquestions.WillprogressinAIbecomeexplosively
fastonceAIresearchanddevelopmentitselfcanbeautomatedHowwillhigh-levelmachineintel-
ligence(HLMI)aecteconomicgrowthWhatarethechancesthiswillleadtoextremeoutcomes
(eitherpositiveornegative)WhatshouldbedonetohelpensureAIprogressisbenecialTableS4displaysresultsforquestionsweaskedonthesetopics.Herearesomekeyndings:
1.Researchersbelievetheeldofmachinelearninghasacceleratedinrecentyears.
Weaskedresearcherswhethertherateofprogressinmachinelearningwasfasterinthe
secondhalfoftheircareerandonly10%saidprogresswasfasterinthersthalf.Themedian
careerlengthamongrespondentswas6years.
2.ExplosiveprogressinAIafterHLMIisseenaspossiblebutimprobable.Some
authorshavearguedthatonceHLMIisachieved,AIsystemswillquicklybecomevastly
superiortohumansinalltasks[3,12].Thisaccelerationhasbeencalledthe“intelligence
explosion.”WeaskedrespondentsfortheprobabilitythatAIwouldperformvastlybetter
thanhumansinalltaskstwoyearsafterHLMIisachieved.Themedianprobabilitywas
10%(interquartilerange:1-25%).Wealsoaskedrespondentsfortheprobabilityofexplosive
globaltechnologicalimprovementtwoyearsafterHLMI.Herethemedianprobabilitywas
20%(interquartilerange5-50%).
3.HLMIisseenaslikelytohavepositiveoutcomesbutcatastrophicrisksare
possible.RespondentswereaskedwhetherHLMIwouldhaveapositiveornegativeimpact
onhumanityoverthelongrun.Theyassignedprobabilitiestooutcomesonave-point
scale.Themedianprobabilitywas25%fora“good”outcomeand20%foran“extremely
good”outcome.Bycontrast,theprobabilitywas10%forabadoutcomeand5%foran
outcomedescribedas“ExtremelyBad(e.g.,humanextinction).”
4.SocietyshouldprioritizeresearchaimedatminimizingthepotentialrisksofAI.
Forty-eightpercentofrespondentsthinkthatresearchonminimizingtherisksofAIshould
beprioritizedbysocietymorethanthestatusquo(withonly12%wishingforless).
Asia (n=68)
Europe (n=58)
North America (n=64)
Other Regions (n=21)
0.00
0.25
0.50
0.75
1.00
0255075100
Years from 2016
Pr
ob
ab
ili
ty
o
f H
LM
I
Undergrad Region HLMI CDFs
Figure3:AggregateForecast(computedasinFigure1)forHLMI,groupedbyregionin
whichrespondentwasanundergraduate.Additionalregions(MiddleEast,S.America,Africa,
Oceania)hadmuchsmallernumbersan