Topics
late
AI
Amazon
Image Credits:Gabby Jones / Bloomberg / Getty Images
Apps
Biotech & Health
mood
Image Credits:Gabby Jones / Bloomberg / Getty Images
Cloud Computing
Commerce
Crypto
endeavour
EVs
Fintech
Fundraising
contraption
Gaming
Government & Policy
ironware
Layoffs
Media & Entertainment
Meta
Microsoft
seclusion
Robotics
Security
societal
Space
Startups
TikTok
Transportation
Venture
More from TechCrunch
Events
Startup Battlefield
StrictlyVC
Podcasts
telecasting
Partner Content
TechCrunch Brand Studio
Crunchboard
reach Us
declarer work to improve Google ’s Gemini AI are compare its answers against output grow by Anthropic ’s competitor modelling Claude , according to internal correspondence seen by TechCrunch .
Google would not say , when reached by TechCrunch for comment , if it had receive license for its use of Claude in testing against Gemini .
As tech company race to build up good AI models , the performance of these mannikin is often evaluated against competition , typically by running theirown models through diligence benchmarksrather than having contractor painstakingly pass judgment their competitors ’ AI responses .
The contractors working on Gemini tasked with rate the accuracy of the model ’s outputs must score each response that they see according to multiple criteria , like truthfulness and verboseness . The contractors are given up to 30 proceedings per prompting to make up one’s mind whose answer is better , Gemini ’s or Claude ’s , harmonise to the correspondence seen by TechCrunch .
The contractors latterly began comment references to Anthropic ’s Claude appearing in the intragroup Google platform they employ to compare Gemini to other unnamed AI models , the correspondence show . At least one of the outputs presented to Gemini contractile organ , see by TechCrunch , explicitly stated : “ I am Claude , created by Anthropic . ”
One national chat show the declarer noticing Claude ’s response appearing to emphasize safety more than Gemini . “ Claude ’s safety setting are the strict ” among AI models , one contractor write . In certain case , Claude would n’t respond to prompts that it considered unsafe , such as role - playing a different AI assistant . In another , Claude forfend answer a command prompt , while Gemini ’s response was droop as a “ immense safety violation ” for admit “ nudity and bondage . ”
Anthropic’scommercial terms of serviceforbid customer from accessing Claude “ to build a contend intersection or service ” or “ discipline contend AI models ” without approval from Anthropic . Google is a majorinvestorin Anthropic .
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
Shira McNamara , a spokesperson for Google DeepMind , which flow Gemini , would not say — when asked by TechCrunch — whether Google has obtained Anthropic ’s approving to get at Claude . When reached prior to publishing , an Anthropic spokesperson did not comment by military press time .
McNamara said that DeepMind does “ equate model outputs ” for evaluations but that it does n’t train Gemini on Anthropic model .
“ Of course , in line with standard industry praxis , in some cases we compare poser end product as part of our evaluation process , ” McNamara said . “ However , any trace that we have used Anthropic model to take Gemini is inaccurate . ”
Last week , TechCrunch exclusively reportedthat Google contractors working on the company ’s AI intersection are now being made to grade Gemini ’s AI responses in areas outside of their expertness . Internal commensurateness expressed concerns by contractile organ that Gemini could beget inaccurate information on highly tender subject like health care .
you may send tips securely to this newsperson on Signal at +1 628 - 282 - 2811 .
