Topics
Latest
AI
Amazon
Image Credits:Frederic Lardinois/TechCrunch
Apps
Biotech & Health
Climate
Cloud Computing
Commerce
Crypto
Enterprise
EVs
Fintech
fund-raise
Gadgets
Gaming
Government & Policy
Hardware
Layoffs
Media & Entertainment
Meta
Microsoft
concealment
Robotics
Security
Social
Space
inauguration
TikTok
Transportation
speculation
More from TechCrunch
Events
Startup Battlefield
StrictlyVC
Podcasts
video
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
In a typical twelvemonth , Cloud Next — one of Google ’s two major annual developer conferences , the other beingI / O — almost exclusively feature managed and otherwise unsympathetic source , gate - behind - shut away - down - APIs products and service . But this class , whether to foster developer goodwill or advance its ecosystem aspiration ( or both ) , Google debuted a number of open reference tools primarily aimed at supporting reproductive AI projects and substructure .
The first , MaxDiffusion , which Google actually quietly released in February , is a collection of quotation implementations of various diffusion models — modelling like the image generatorStable dissemination — that play on XLA gadget . “ XLA ” stand for Accelerated Linear Algebra , an admittedly awkward acronym referring to a technique that optimize and speeds up specific types of AI workloads , including fine - tuning and serving .
Google ’s owntensor processing units ( TPUs)are XLA gadget , as are recent Nvidia GPUs .
Beyond MaxDiffusion , Google ’s launchingJetStream , a new engine to run generative AI model — specifically textual matter - yield models ( sonotStable Diffusion ) . presently trammel to support TPUs with GPU compatibility supposedly coming in the futurity , JetStream bid up to 3x higher “ performance per dollar sign ” for model like Google ’s ownGemma 7Band Meta’sLlama 2 , Google exact .
“ As customers bring their AI workloads to product , there ’s an increasing demand for a cost - efficient illation stack that delivers eminent performance , ” Mark Lohmeyer , Google Cloud ’s GM of compute and automobile learning infrastructure , wrote in ablog postshared with TechCrunch . “ JetStream facilitate with this pauperism … and includes optimizations for popular heart-to-heart models such as Llama 2 and Gemma . ”
Now , “ 3x ” melioration is quite a claim to make , and it ’s not exactly clear how Google make it at that digit . Using which generation of TPU ? equate to which baseline locomotive engine ? And how ’s “ performance ” being fix here , anyway ?
I ’ve call for Google all these questions and will update this post if I hear back .
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
Second - to - last on the list of Google ’s opened root contributions are new gain toMaxText , Google ’s collection of textual matter - generate AI models place TPUs and Nvidia GPUs in the cloud . MaxText now let in Gemma 7B , OpenAI’sGPT-3(the predecessor toGPT-4),Llama 2and models from AI startupMistral — all of which Google say can be customise and delicately - tuned to developer ’ needs .
“ We ’ve intemperately optimized [ the models ’ ] operation on TPUs and also partnered intimately with Nvidia to optimize performance on great GPU clustering , ” Lohmeyer pronounce . “ Theseimprovements maximize GPU and TPU utilization , leading to high energy efficiency and monetary value optimization . ”
Finally , Google ’s collaborate with Hugging Face , the AI startup , to createOptimum TPU , which provides tooling to impart sure AI workloads to TPUs . The end is to reduce the roadblock to entranceway for getting generative AI model onto TPU hardware , according to Google — in particular text - generating models .
But at present , Optimum TPU is a minute bare - finger cymbals . The only poser it work with is Gemma 7B. And Optimum TPU does n’t yet patronize train generative model on TPUs — only running them .
Google ’s promising improvements down the line .
