Aldo's Notes #22
The One with Transformers; Open AI APIs; Snapchat launches chatbot; Google's Magic Eraser and Meta drops Quest prices
1. Transformers, explained: Understand the model behind GPT and BERT
Dale Markowitz, Applied AI Engineer at Google, explains ‘transformers’, the key ingredient behind GPT and Bert.
Transformers are a type of neural network that can translate text, write poems and op-eds, and even generate computer code.
They are like a magical machine learning hammer that seems to make every problem into a nail.
Neural networks are a very effective type of model for analyzing complicated data types, but there are different types of neural networks optimized for different types of data.
Until transformers came around, the way we used deep learning to understand text was with a type of model called a Recurrent Neural Network (RNN).
RNNs had a lot of problems, including difficulty in handling large sequences of text and being hard to train.
Transformers were developed in 2017 by researchers at Google and the University of Toronto to do translation, but unlike RNNs, they can be efficiently parallelized.
This means that with the right hardware, really big transformer models can be trained on massive amounts of data, such as almost the entire public web.
Why it’s important: if you want to keep up with natural language processing, you have to know about transformers.
2. Open AI introduces APIs
OpenAI has added ChatGPT and Whisper models to its API.
The initiative provides developers with advanced language and speech-to-text capabilities.
System-wide optimizations have reduced ChatGPT costs by 90% since December, and these savings are being passed on to API users.
The open-source Whisper large-v2 model is now available on the API, offering faster and more cost-effective results.
ChatGPT API users can expect continuous model improvements and the option to choose dedicated capacity for greater control.
OpenAI has refined its API terms of service based on developer feedback to better meet their needs.
Why it matters: tighter integration with developers’ work means a higher chance of long-term success.
3. Snapchat launches AI chatbot
My AI is a new chatbot launched for Snapchat+ subscribers
It uses the latest version of OpenAI's GPT technology customized for Snapchat
My AI can recommend birthday gift ideas, plan a hiking trip, suggest recipes, and write haikus
Users can customize the name and wallpaper for their Chat
My AI is prone to hallucination and can be tricked into saying anything, so users should not rely on it for advice or share any secrets
All conversations with My AI will be stored and reviewed to improve the product experience
Users can submit feedback by pressing and holding on any message from My AI
My AI is designed to avoid biased, incorrect, harmful, or misleading information, but mistakes may occur.
Why it matters: one of the first uses if new-generation AI-powered chatbots. Will it work? Time will tell.
4. Magic Eraser coming to a phone near you
9to5Google reports on Magic Eraser availability
Google has expanded the availability of its Magic Eraser tool to all Pixel phones and to any Google One subscriber, even those using iOS devices.
Magic Eraser is a feature in Google Photos that can automatically remove distractions from photos or let users manually select areas to remove or change the color of.
The tool is available in the "Suggestions" or "Tools" tabs of the Google Photos editor and is accessible to Google One subscribers on non-Pixel and iOS devices.
Google has also made the HDR effect for videos widely available, allowing users to enhance the brightness and contrast of their videos.
The expanded availability of Magic Eraser and HDR for videos is seen as a way to make Google One more appealing to users, particularly with its low monthly subscription cost of $1.99.
Why it matters: better tools in the hands of creators!
5. Meta drops Quest prices
Meta has announced a price drop for two of their models, the Meta Quest 2 and Meta Quest Pro.
Starting from March 5, the Meta Quest 2 with 256GB of storage will cost $429.99 USD and the Meta Quest Pro will cost $999.99 USD.
The aim of this price drop is to make VR more accessible to people by creating hardware that is affordable for everyone.
The Meta Quest 2 was the first standalone headset with inside-out tracking, allowing people to experience VR without a PC or game console.
The company hopes that by lowering the price of their headsets, more people will be able to experience fully immersive games and experiences.
The price drop will take effect in various countries, including the US, UK, Canada, and Australia.
The company is committed to building a successful VR market and making VR accessible to more people.
Why it matters: lower prices means more people can get into VR