On December 7th 2023, Google finally decided to respond to the threat of Open AI and ChatGPT by announcing Gemini 1.0, its most capable LLM (large language model).
As per reports, Gemini 1.0 is being pre-trained and fine-tuned on various modalities, making it good at understanding and dealing with different inputs, and outperforming other models that handle multiple types of data.
What makes Gemini 1.0 different?
Firstly, Gemini is not just one language model, but three models with each model having its own use-case.
Gemini Nano
Gemini Nano is the most lightweight and efficient model of the three, designed to run directly on mobile devices. Consider it a smarter version of your Google Assistant that can unlock a plethora of features for your mobile device.
Gemini Pro
This is the mid-tier model released by Google and is currently powering Google Bard. Based on Google’s claims, its said that Bard should now be able to better compete with GPT4 since it runs on the new Gemini Pro model (let’s see how true this holds).
Gemini Ultra
The highest-tier Model amongst the 3 and also the one to give GPT4 a run for its money, Gemini Ultra is built for extremely complex tasks and is said to have outperformed GPT4 in almost every category.
To know more about the three models, check out Google’s official page.
Second, let’s check Gemini’s published on-paper stats
Google says Gemini AI, their strongest version, beats ChatGPT in about 90% of academic benchmarks, including popular tests in text, reasoning, image understanding, video understanding, speech recognition, and translation. Overall, Gemini is better in most areas, except for text, where it's a tie with ChatGPT.
In an MMLU (Massive Multitask Language Understanding) test with 57 subjects including Math, Physics, and Law, Gemini AI scored 90% while ChatGPT scored 86.4%.
As per the reports, Gemini Pro (Google’s less powerful model) was easily able to outperform ChatGPT 3.5 (OpenAI’s free version) in 6 out of 8 tests.
Does this mean Gemini AI is already better?
Well, not exactly!
While Gemini AI looks great on paper, Open AI still has an upper hand with their model for these reasons -
Gemini AI is still under development. Compared to ChatGPT, it has limited real-world data and experience. There’s still a long way to go for Google.
While Google’s focus on Multimodal applications looks great, ChatGPT is still a better option for folks primarily requiring text generation.
You can’t ignore the pre-existing userbase ChatGPT has built for itself which also allows access to its capabilities for a broader range of applications.
Lastly, Open AI is a fairly newer company with more flexibility to experiment when compared to Google. This just means that Open AI can spend more time and resources on experimentation, research, and development i.e faster update cycles.
Conclusion
So, is Gemini AI better or is ChatGPT still in the lead?
Honestly, it’s tough to tell at the moment. We are yet to see Gemini AI in real-time action and clearly, Open AI isn’t going to go down without putting up a solid fight.
All we can do at the moment is grab our popcorn and watch these behemoths go toe-to-toe against each other. It’s been a while since we saw an all-out tech war like this one and honestly, it keeps getting fun!
Tune back to this newsletter in a few months when we have a more conclusive report on which Gen AI model is a better one.
Until next time!