Dr Andrew Ng, The Artificial Intelligence Innovator and Pioneer Helping The Planet Embrace AI
Google Cloud Teams Up With NLP Startup Cohere On Multiyear Partnership ...
Google Cloud announced a multiyear partnership with Cohere, an early-stage startup that is building a natural language processing platform to make it easier for developers to build natural language processing models into applications. The solution requires a fair amount of infrastructure resources to pull off, and Google Cloud Platform (GCP) is going to provide them under the terms of this deal.
The two companies are also planning a go-to-market effort together, giving Cohere a big lift as a startup to help jumpstart usage and sales with the power of the GCP sales team.
Google Cloud CEO Thomas Kurian said that Cohere offers a great use case for Google Cloud Tensor Processing Unit (TPUs) chips that builds on work Google has been doing in-house.
"First of all, this is a perfect example of a technology that we built at Google for our own use, these high-scale TPUs. We're now making them available in the cloud to other platform companies," Kurian told me. "But here is an example [with Cohere] where they find that the ability to use and build models and train them on TPUs provides them very differentiated capability."
Aidan Gomez, co-founder and CEO at Cohere, who previously worked at Google Brain, said his company is trying to build an NLP solution that makes all of this advanced technology available to any developer. "We scrape the data to train these big models, we train them on massive TPU pods, and we optimize them because they're extremely large, and want them to fit the latency tolerances of pretty much any production system," Gomez explained.
He said that by optimizing the workloads, Cohere can open up access to all of this advanced technology, enabling developers to access the models and start building NLP-based solutions based on the models that Cohere is providing. Kurian said that he is starting to see this shift from the text-based UI to one that's more driven by natural language interactions, and Cohere is a good example of how to make this happen.
"If you look at the state of the art, the way that the vast majority of people use computers is through graphical user interfaces or screens. I don't think that people want to experience computers through just one sense, which is the sense of sight," Kurian said. "They want to interact with computers in more ways and they want to interact in more natural ways. So language is the next big phase of evolution of the way that people interact with systems."
Techcrunch event
San FranciscoOctober 27-29, 2025
It's not every day that you see the CEO of one of the big three cloud platforms get on a call to discuss a partnership with a startup, but Kurian believes this is a particularly compelling and creative example of TPU usage. "If you actually use the Cohere technology, you will find that it works very elegantly because of the combination of the software that Aidan's team has built and the computational infrastructure that TPUs provide."
Cohere was founded in 2019 by Gomez, Nick Frosst and Ivan Zhang in Toronto. The company has raised $40 million from Index Ventures, Radical Ventures, Section 32, and a who's who of industry AI angels including Geoffrey Hinton and Fei-Fei Li, among others.
Ron Miller was an enterprise reporter at TechCrunch.
Previously, he was a long-time Contributing Editor at EContent Magazine. Past regular gigs included CITEworld, DaniWeb, TechTarget, Internet Evolution and FierceContentManagement.
Disclosures:
Ron was formerly corporate blogger for Intronis where he wrote once weekly on IT issues. He has contributed to various corporate blogs in the past including Ness, Novell and the IBM Mid-market Blogger Program.
Google Open-Sources BERT: A Natural Language Processing Training ...
In a recent blog post, Google announced they have open-sourced BERT, their state-of-the-art training technique for natural language processing (NLP) applications. Google has decided to do this, in part, due to a lack of public data sets that are available to developers. BERT also includes a new bidirectional technique which improves its effectiveness in NLP. To reduce the amount of time required for developers and researchers to train their NLP models, Google has made optimizations in Cloud Tensor Processing Units (TPUs) which reduces the amount of time it takes to train a model to 30 minutes vs a few hours using a single GPU.
Google feels there is a shortage of NLP training data available to developers. Jacob Devlin and Ming-Wei Change, research scientists at Google, explain why it was important to share their datasets:
One of the biggest challenges in natural language processing (NLP) is the shortage of training data. Because NLP is a diversified field with many distinct tasks, most task-specific datasets contain only a few thousand or a few hundred thousand human-labeled training examples. However, modern deep learning-based NLP models see benefits from much larger amounts of data, improving when trained on millions, or billions, of annotated training examples.
Researchers at Google have also developed a new technique, called Bidirectional Encoder Representations from Transformers (BERT), for training general purpose language representation models using a very large data set that includes generic text from the web, also referred to as pre-training. Devlin explains why this pre-training is important:
The pre-trained model can then be fine-tuned on small-data NLP tasks like question answering and sentiment analysis, resulting in substantial accuracy improvements compared to training on these datasets from scratch.
BERT includes source code that is built upon TensorFlow, an open-source machine learning framework, and a series of pre-trained language representation models. Google has published an associated paper where their state-or-the-art results on 11 NLP tasks are demonstrated, including how it performed against the Stanford Question Answering Dataset (SQuAD v1.1).
The difference between BERT and other approaches is based upon how it works in pre-training contextual representations, including Semi-supervised Sequence Learning, Generative Pre-Training, ELMo, and ULMFit. But, what really sets BERT apart is the first deeply bidirectional, unsupervised language representation which is pre-trained using only a plain text corpus. Devlin explains why this approach is unique:
Pre-trained representations can either be context-free or contextual, and contextual representations can further be unidirectional or bidirectional. Context-free models such as word2vec or GloVe generate a single word embedding representation for each word in the vocabulary. For example, the word "bank" would have the same context-free representation in "bank account" and "bank of the river."
Contextual models instead generate a representation of each word that is based on the other words in the sentence. For example, in the sentence "I accessed the bank account," a unidirectional contextual model would represent "bank" based on "I accessed the" but not "account." However, BERT represents "bank" using both its previous and next context — "I accessed the ... Account" — starting from the very bottom of a deep neural network, making it deeply bidirectional.
Unidirectional models work by predicting each word based upon previous words within a sentence. Historically, bidirectional training was difficult as a word would inevitably be able to see itself in multi-layer models when the 'next' predicted word looks back to the 'previous' predicted word. BERT addresses this challenge through the use of masked words.
/filters:no_upscale()/news/2018/11/google-bert-nlp/en/resources/1f1-1542570561648.Png)
Image Source: https://ai.Googleblog.Com/2018/11/open-sourcing-bert-state-of-art-pre.Html
In addition to new techniques included in BERT, Google has made enhancements to Cloud TPUs which allow developers and researchers to quickly experiment, debug and tweak models. These investments allowed Google to exceed the capabilities of existing pre-training models.
BERT is available from GitHub in addition to the tensor2tensor library.
Google Deploys New NLP Models, Cloud TPUs To Make Its Search Engine ...
Google LLC is augmenting its search engine with natural-language processing features that it said represent the most significant update of the past five years.
The company detailed the changes in a blog post published this morning. Google is adding new NLP models to its search engine that use a technique called Bidirectional Encoder Representations from Transformers, or BERT, to analyze user queries. The method allows artificial intelligence algorithms to interpret text more accurately by analyzing how the words in a sentence relate to one another.
The concept is not entirely new. NLP algorithms capable of deducing sentence context have been around for a while and even boast their own term of art: transformers. But whereas traditional NLP software can look at a sentence only from left to right or right to left, a BERT-based model does both at the same time, which allows it to gain a much deeper understanding of the meaning behind words. The technique was hailed as a breakthrough when Google researchers first detailed it in an academic paper last year.
The company said the update will allow it to return more relevant results for about one in every 10 English-language requests made from the U.S. The biggest improvement will be for the queries that were previously most likely to trip up Google: those that contain prepositions such as "for" and "to" or are written in a conversational style.
Google is also using the technology to improve the featured snippets that show up above some search results. And over time, the company plans to expand availability to additional languages and regions, which should open the door to even more search improvements. That's because on top of their ability to understand complex sentences, BERT-based AI models can transfer knowledge between languages.
"It's not just advancements in software that can make this possible: we needed new hardware too," Pandu Nayak, Google's vice president of search, detailed in the post announcing the changes. "Some of the models we can build with BERT are so complex that they push the limits of what we can do using traditional hardware, so for the first time we're using the latest Cloud TPUs to serve search results." TPUs, or Tensor Processing Units, are the internally designed machine learning chips that Google offers through its cloud platform.
Search is just one of the areas where the company is working to apply BERT. Last month, Google researchers revealed VideoBERT, an experimental algorithm that harnesses the technique to predict events likely to happen a few seconds into the future.
Photo: UnsplashSupport our mission to keep content open and free by engaging with theCUBE community. Join theCUBE's Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
About SiliconANGLE Media
SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.Com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
Comments
Post a Comment