How do you determine the most powerful AI models currently available? Check out the rankings to find out.
Community compiled leaderboards for AI models have surged in popularity online in recent months, providing a real-time window into the jockeying of major tech giants in the AI space.
Various leaderboards document which AI models are the most advanced at performing certain tasks.An AI model is essentially a set of mathematical formulas wrapped in code designed to achieve a specific purpose.
Startups like Google's Gemini (previously Bard) and Parisian Mistral AI New entrants like Mistral-Medium have galvanized the AI community and are jockeying for position at the top of the charts.
However, OpenAI's GPT-4 still dominates.
People care about the cutting edge of technology," says Ying Sheng, a PhD student in computer science at Stanford University and co-creator of the Chatbot Arena list. I think people actually like to see that the rankings continue to change. It shows that the game is still going on and there is still room for improvement."
The rankings are based on tests of the capabilities of AI models, which are designed to figure out what AI is generally capable of, and which models might be most at home in specific applications, such as speech recognition. These tests, sometimes called benchmarking tests, measure AI performance through metrics such as how close to a human voice AI vocalizations sound, or how humanized an AI chatbot's responses are.
As AI continues to evolve, continuous improvement of these tests is equally critical.
Vanessa Parli, director of research at the Institute for Artificial Intelligence at Stanford University's Center for the Human Dimension, said, "These benchmarks aren't perfect, but as of now, it's the only way we can evaluate the system."
The Institute's annual report on the Stanford Artificial Intelligence Index tracks the technical performance of AI models over time under various metrics. According to Parli, last year's report researched 50 benchmarks, but only included 20. This year, the report will eliminate some outdated benchmarks in order to focus on newer, more comprehensive ones.
The leaderboard also gives us a peek at the number of models under development. open LLM built by Hugging Face [large language model] leaderboard, an open source machine learning platform, has evaluated and ranked more than 4,200 models as of early February, all submitted by community members.
The models participate in seven key benchmark tests designed to assess their ability in various categories, such as reading comprehension and math problem solving. The evaluation process includes elementary school math and science questions that test the models' common-sense reasoning and measure their tendency to disseminate misinformation. Some of the tests provide a multiple-choice format, while others require the models to generate their own answers based on cues.
OpenAI's ChatGPT-4 can be seen at the top of the LMSYS Chatbot Arena Leaderboard, followed closely by Google's Geminivia. LMSYS
Visitors can view the specific performance of each model on a particular benchmark test, as well as their average total score. So far, no model has achieved a perfect score of 100 on any benchmark. Smaug-72B, a newly developed AI model by San Francisco startup Abacus.AI, became the first model to break 80 points on average.
Many large-scale language models have already surpassed human benchmarks on such tests, a phenomenon researchers call "saturation," says Thomas Wolf, co-founder and chief scientific officer of Hugging Face. It usually occurs when the model's ability increases beyond a specific test, as when a student moves from middle school to high school and progressively outgrows the previous stage of learning, or when the model has memorized how to answer certain test questions, a concept known as "overfitting.
