{"id":24384,"date":"2025-11-03T09:48:29","date_gmt":"2025-11-03T09:48:29","guid":{"rendered":"https:\/\/www.gsma.com\/newsroom\/?p=24384"},"modified":"2025-11-03T12:08:01","modified_gmt":"2025-11-03T12:08:01","slug":"gsma-open-telco-llm-benchmarks-ranks-frontier-ai-models-for-telco-ai-revealing-critical-gap-in-network-automation-readiness","status":"publish","type":"post","link":"https:\/\/www.gsma.com\/newsroom\/article\/gsma-open-telco-llm-benchmarks-ranks-frontier-ai-models-for-telco-ai-revealing-critical-gap-in-network-automation-readiness\/","title":{"rendered":"GSMA Open-Telco LLM Benchmarks ranks frontier AI models\u00a0for Telco AI, revealing critical gap in network automation readiness\u00a0\u00a0"},"content":{"rendered":"\n

GSMA\u2019s Louis Powell highlights the findings from the second phase of the GSMA Open-Telco LLM Benchmarks, expanding the focus from general domain knowledge to real-world challenges in network management, configuration, and troubleshooting \u2013 concluding that hybrid architectures are the way to move the industry forward<\/em> <\/p>\n\n\n\n

We kicked off the GSMA Open-Telco LLM Benchmarks initiative in Barcelona<\/a> this year to bring together a passionate community determined to tackle the unique challenges of telco AI. The community\u2019s early work \u2013 under GSMA Open-Telco LLM Benchmarks 1.0<\/a> \u2013 quickly showed us that \u2018out-of-the-box\u2019 AI models just aren\u2019t up to the task for telco-specific needs. The results spoke for themselves. <\/p>\n\n\n\n

Now, as we move to GSMA Open-Telco LLM Benchmarks 2.0<\/a>, the initiative has grown from a benchmarking exercise to become a thriving, community-driven hub of open-source innovation. And the first findings lead us to the conclusion that a hybrid architecture strategy is the most effective path forward to unlock both scale and precision in telco AI; combining the broad reasoning of foundation models with the precision of specialised components.  <\/p>\n\n\n\n

Why? Because the results reveal two critical realities: first, targeted fine-tuning delivers operational accuracy. And second, automation remains a shared challenge as current LLMs struggle to translate natural-language intents into schema-compliant configurations for zero-touch orchestration.  <\/p>\n\n\n\n

Delivering the Data and Core Findings: The Hybrid Architecture Imperative<\/strong> <\/p>\n\n\n\n

The collective input of our community led to the construction of five high-stakes, operationally focused evaluation datasets to look at a range of scenarios: <\/p>\n\n\n\n