BERT was a significant step forward in teaching machines how to understand the complexity of human language; back in 2018, when it was first deployed, its innovation was in the ability to read words and sentences in context. However, it is not perfect. According to Google, an average user still needs eight different queries to find the relevant answers to more complex questions.
Then, in 2021, Prabhakar Raghavan, Senior Vice President at Google, announced the existence of MUM. MUM stands for Multitask Unified Model and picks up where BERT left off. It aims to reduce the number of searches needed to get useful information from the search engine.
While BERT was trained to understand context, MUM goes a step further – it does not only understand, it’s capable of generating (with understanding) as well; plus, as stated by Google, it’s 1,000 times more powerful than BERT.
Key Takeaways
- MUM is a machine learning algorithm that is 1,000 times more powerful than BERT, Google's previous algorithm.
- MUM is trained in 75 different languages and can understand information across media other than text.
- The benefits of MUM include faster, easier access to information, less bias, and better machine learning models.
- MUM is a part of Google's goal to make information universally accessible and useful.
- MUM is designed to provide users with the most relevant and accurate results possible.
What Is Google’s MUM?
MUM is based on natural language processing, natural language understanding and artificial intelligence in a way more advanced way than BERT.
It is a technology that uses T5 text-to-text Transfer Transformer framework to develop a deeper understanding of words, phrases, sentences, as well as context and everything around it – overall language and world knowledge in general.
It’s trained in 75 different languages and a wide variety of tasks at once (contrary to BERT, whose downside is its speed as, for example, each language is trained on its own model). What’s more, MUM is multimodal, meaning it can understand information across media other than text, too.
Pandu Nayak, Google Fellow and Vice President, Search, promises that thanks to MUM, ‘in the future, you’ll need fewer searches to get things done,’ and provides an example:
Take this scenario:
You’ve hiked Mt. Adams. Now you want to hike Mt. Fuji next fall, and you want to know what to do differently to prepare.
Today, Google could help you with this, but it would take many thoughtfully considered searches — you’d have to search for the elevation of each mountain, the average temperature in the fall, difficulty of the hiking trails, the right gear to use, and more.
After a number of searches, you’d eventually be able to get the answer you need.
[...] MUM could understand you’re comparing two mountains so elevation and trail information may be relevant. It could also understand that, in the context of hiking, to “prepare” could include things like fitness training as well as finding the right gear.
Google’s GIF: https://storage.googleapis.com/gweb-uniblog-publish-prod/original_images/MUM_GIF02_02_1.gif
MUM Across Languages
One of the greatest promises of MUM is removing the language barrier. As it can work across different languages (it’s trained in 75), it’s said to be capable of transferring information from one language to another.
For example, it can learn from a source written in French and provide you with an answer if you look for something related to it in English.
As per the previously mentioned example about Mt. Fuji:
Say there’s really helpful information about Mt. Fuji written in Japanese; today, you probably won’t find it if you don’t search in Japanese. But MUM could transfer knowledge from sources across languages, and use those insights to find the most relevant results in your preferred language.
So in the future, when you’re searching for information about visiting Mt. Fuji, you might see results like where to enjoy the best views of the mountain, onsen in the area and popular souvenir shops — all information more commonly found when searching in Japanese.
Multimodality – MUM Across Formats
As we’ve already mentioned, MUM is also multimodal, meaning it can learn information from all kinds of file formats, e.g., websites, images, videos, or audios. Going back to the example of Mt. Fuji, you may one day be able to share the photo of your gear and determine if it’s suitable for this particular trip.
Machine learning is all about gathering, analyzing and combining information from various sources to provide accurate responses. MUM is capable of accumulating data across different media, creating a more comprehensive, in-depth data collection for machine learning models.
Given this broad understanding of information across media and languages, MUM will be able to provide relevant answers even to complex queries without the need for multiple searches.
T5 Text-to-Text Transfer Transformer Framework
Natural language processing (NLP) has been flourishing, with new results and solutions appearing regularly for a while now. It refers to pre-training a model on a vast range of different text data with various tasks, e.g., guessing missing words, predicting sentences, and so on.
Then, the model can be adjusted and improved, resulting in quite a comprehensive understanding of a given language. That’s how BERT works, the MUM’s precedessing algorithm.
We’ve seen the rise of multiple transfer learning methods; their abundance made it difficult to keep track of innovations, their upsides and downsides.
In “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer” paper, Google tries to determine which transfer learning techniques work better than others; next, they use the data to create a brand-new model called the Text-To-Text Transfer Transformer (T5), which is pre-trained on the Colossal Clean Crawled Corpus(C4, also introduced by Google as an open-source pre-training dataset).
The T5 framework excels on a variety of NLP benchmarks and can also be fine-tuned to multiple downstream tasks and combine them into a text-to-text format, making it much more flexible and versatile.
The Goals and Benefits of MUM
MUM is a step towards the company’s goal of making information universally accessible and useful, regardless of the language or format it’s in.
It’s also a part of Google’s AI Principles, which aim to ensure that their products and technologies ‘are socially beneficial, avoid creating or reinforcing unfair bias, and are built and tested for safety.’
The benefits of the new technology include:
- Faster, easier access to information – you won’t need to go through multiple searches to find what you’re looking for.
- Greater understanding of the world – MUM can learn from different languages and media, providing you with a more comprehensive understanding of the topic you’re interested in.
- Less bias – as MUM is trained on a great variety of data, the risk of biased results is lower.
- Better machine learning models – thanks to the ability to understand different languages and media, MUM can create more accurate models.
MUM and SEO: All About E-A-T
With MUM, Google is aiming to provide users with more relevant and accurate results, which ultimately leads to a better user experience. And we all know that a good user experience is essential for SEO.
Google’s Search Quality Evaluator Guidelines state that one of the main goals of their search engine is to ‘deliver a great user experience.’ In order to do that, they need to provide users with ‘high-quality’ results that are ‘relevant’ and ‘trustworthy.’ This is where E-A-T comes in.
E-A-T stands for Expertise, Authoritativeness, and Trustworthiness, and it’s a set of quality criteria that Google’s search algorithm looks for when ranking websites. It was first introduced in 2014 and has been constantly evolving ever since.
Expertise refers to the content’s quality, i.e., whether it’s well researched, comprehensive, and up-to-date.
Authoritativeness is all about the author – whether they’re an expert on the topic and whether they can be trusted.
Trustworthiness is self-explanatory – it’s about whether the website can be trusted, e.g., whether it’s reliable, safe, and secure.
All three of these factors are essential for SEO, and Google is constantly working on ways to improve their algorithms so they can better identify websites that meet these criteria.
MUM is yet another step in that direction. The technology focuses on the user's intention and understanding the user's needs. It's designed to provide users with the most relevant and accurate results possible, which is precisely what E-A-T is all about.
Final Note
In the light of all the above, it’s safe to say that MUM will be a game-changer for Google Search.
It is a huge leap in the field of machine learning and teaching AIs the complexities of language. It certainly looks promising, though we need to remember that nothing is perfect, especially in its early stages.
There will be issues and problems to solve along the way, but if Google manages to keep their promises, MUM may one day change the way we use search engines and interact with machines.
While it’s still too early to tell what kind of an impact MUM will have on the digital world, we can be certain it will be significant. Google has been making great strides in the field of NLP, AI, and machine learning in general, and MUM is yet another example of it. We can only wait and see what the future holds.
