{"id":25707,"date":"2026-04-28T08:59:10","date_gmt":"2026-04-28T07:59:10","guid":{"rendered":"https:\/\/www.gsma.com\/newsroom\/?p=25707"},"modified":"2026-04-28T10:00:51","modified_gmt":"2026-04-28T09:00:51","slug":"pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages","status":"publish","type":"post","link":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/","title":{"rendered":"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0"},"content":{"rendered":"\n

CommonLingua, the first joint release under the GSMA\u2019s \u201cAI Language Models in Africa, by Africa, for Africa\u201d initiative, is a compact 2-million-parameter open-source model covering 334 languages \u2013 including 61 African languages \u2013 and outperforming systems up to 300 times large<\/em> <\/p>\n\n\n\n

28 April 2026, London:<\/strong> Pleias<\/a> and the GSMA<\/a> today announced the release of CommonLingua, an open-source language identification (LID) model purpose-built to unlock African language data at scale. It is delivered under the GSMA’s AI Language Models in Africa, by Africa, for Africa<\/em> initiative, a coalition dedicated to closing the African language gap in AI. <\/p>\n\n\n\n

Africa is home to more than 2,000 living languages, many of which remain underrepresented in AI training data. As a result, language identification systems often perform less reliably on African-language content, particularly when distinguishing between closely related or code-mixed text. Before a Swahili, Yoruba, or Wolof language model can be built, the underlying text must first be correctly identified by language \u2013 a step where existing tools currently often fail on African content.  <\/p>\n\n\n\n

This is because leading LID systems such as fastText, GlotLID, and OpenLID were built around European and Asian high-resource languages and frequently mislabel African-language text as English or French. Even state-of-the-art frontier models drop roughly 30 points in accuracy on African languages compared to major world languages. <\/p>\n\n\n\n

CommonLingua is designed to fix this first step of the pipeline. On the new CommonLID benchmark, CommonLingua achieves 83% accuracy and a macro score F1 of 0.79, outperforming leading LID models by more than 10 percentage points under comparable evaluation conditions, while using roughly one three-hundredth of the parameters. The model is lightweight at 2 million parameters and shipping as an 8 MB checkpoint, and is designed for efficient deployment, running approximately 20 texts per second on CPU and up to 3,000 texts per second on a single GPU. <\/p>\n\n\n\n

CommonLingua covers 334 languages in total, including 61 African languages across eight language families: Bantu (21), Niger-Congo \/ West African (18), Afro-Asiatic and Semitic (7), Cushitic and Chadic (4), Berber (3), Nilo-Saharan (3), and pidgins, creoles, and other (5). The model operates directly on UTF-8 byte sequences rather than relying on a language-specific tokenizer, enabling consistent handling across scripts including Latin, Arabic, Ethiopic, N\u2019Ko, and Tifinagh. <\/p>\n\n\n\n

“African languages are not an edge case. They are the working languages of hundreds of millions of people, and they deserve AI infrastructure built with the same care as any other language. CommonLingua is deliberately the first brick we are laying: you cannot curate what you cannot identify”<\/em> said Pierre-Carl Langlais, Co-founder and Chief Technology Officer, Pleias.<\/strong> <\/p>\n\n\n\n

The model is trained exclusively on open-licensed and public domain content aggregated through the Common Corpus project, including Wikipedia, Scientific publications in OpenAlex, VOA Africa, WaxalNLP, Cultural Heritage, and Pralekha. All datasets are released under permissive licenses. <\/p>\n\n\n\n

Louis Powell, Director of AI Initiatives at GSMA<\/strong> added: <\/strong>\u201cClosing the gap in African-language AI is is fundamental to digital inclusion and unlocking economic opportunity. Progress has long been held back by the lack of foundational infrastructure, beginning with something as essential as language identification. CommonLingua addresses this critical gap, enabling the development of richer datasets and more representative AI systems at scale. Through our initiative, the GSMA is bringing partners together to move beyond fragmented efforts towards shared infrastructure that can power Africa\u2019s digital ecosystem.<\/em>\u201d <\/p>\n\n\n\n

This conversation will continue at MWC26 Kigali<\/a>, where GSMA and partners will bring together industry leaders to accelerate progress on African-language AI. Register now <\/strong>to be part of the discussion. <\/p>\n\n\n\n

-ENDS-<\/em><\/strong> <\/p>\n\n\n\n

About Pleias<\/strong> <\/p>\n\n\n\n

Pleias is a research lab and AI company specialising in open, auditable language models trained exclusively on permissively licensed data. Pleias develops the Common Corpus, the largest fully open multilingual pretraining dataset, and the Pleias family of small language models optimised for retrieval, reasoning, and low-resource languages. <\/p>\n\n\n\n

About the GSMA <\/strong> 
The GSMA is a global organisation unifying the mobile ecosystem to discover, develop, and deliver innovation foundational to positive business environments and societal change. Our vision is to unlock the full power of connectivity so that people, industry, and society thrive. Representing mobile operators and organisations across the mobile ecosystem and adjacent industries, the GSMA delivers for its members across three broad pillars: Connectivity for Good, Industry Services and Solutions, and Outreach. This activity includes advancing policy; tackling today\u2019s biggest societal challenges; underpinning the technology and interoperability that make mobile work; and providing the world\u2019s largest platform to convene the mobile ecosystem at the MWC and M360 series of events. <\/p>\n\n\n\n

Media Contacts<\/strong> <\/p>\n\n\n\n

Pleias<\/strong>: contact@pleias.fr<\/a> <\/p>\n\n\n\n

GSMA<\/strong>: pressoffice@gsma.com<\/a> <\/p>\n\n\n\n

<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

CommonLingua, the first joint release under the GSMA\u2019s \u201cAI Language Models in Africa, by Africa, for Africa\u201d initiative, is a compact 2-million-parameter open-source model covering 334 languages \u2013 including 61 African languages \u2013 and outperforming systems up to 300 times large  28 April 2026, London: Pleias and the GSMA today announced the release of CommonLingua, an open-source language identification (LID) model purpose-built to unlock African language […]<\/p>\n","protected":false},"author":116,"featured_media":25711,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_jetpack_memberships_contains_paid_content":false,"jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[3],"tags":[],"algolia_discover_type":[],"jetpack_publicize_connections":[],"yoast_head":"\nPleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0 - Newsroom<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0\" \/>\n<meta property=\"og:description\" content=\"CommonLingua, the first joint release under the GSMA\u2019s \u201cAI Language Models in Africa, by Africa, for Africa\u201d initiative, is a compact 2-million-parameter open-source model covering 334 languages \u2013 including 61 African languages \u2013 and outperforming systems up to 300 times large  28 April 2026, London: Pleias and the GSMA today announced the release of CommonLingua, an open-source language identification (LID) model purpose-built to unlock African language […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/\" \/>\n<meta property=\"og:site_name\" content=\"Newsroom\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/gsma\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-04-28T07:59:10+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-04-28T09:00:51+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"ssharma@gsma.com\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@GSMA\" \/>\n<meta name=\"twitter:site\" content=\"@GSMA\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"ssharma@gsma.com\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0 - Newsroom","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/","og_locale":"en_GB","og_type":"article","og_title":"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0","og_description":"CommonLingua, the first joint release under the GSMA\u2019s \u201cAI Language Models in Africa, by Africa, for Africa\u201d initiative, is a compact 2-million-parameter open-source model covering 334 languages \u2013 including 61 African languages \u2013 and outperforming systems up to 300 times large  28 April 2026, London: Pleias and the GSMA today announced the release of CommonLingua, an open-source language identification (LID) model purpose-built to unlock African language […]","og_url":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/","og_site_name":"Newsroom","article_publisher":"https:\/\/www.facebook.com\/gsma\/","article_published_time":"2026-04-28T07:59:10+00:00","article_modified_time":"2026-04-28T09:00:51+00:00","og_image":[{"width":600,"height":400,"url":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","type":"image\/png"}],"author":"ssharma@gsma.com","twitter_card":"summary_large_image","twitter_creator":"@GSMA","twitter_site":"@GSMA","twitter_misc":{"Written by":"ssharma@gsma.com","Estimated reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#article","isPartOf":{"@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/"},"author":{"name":"ssharma@gsma.com","@id":"https:\/\/www.gsma.com\/newsroom\/#\/schema\/person\/e724eba4846a50206db8ca64bace5ceb"},"headline":"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0","datePublished":"2026-04-28T07:59:10+00:00","dateModified":"2026-04-28T09:00:51+00:00","mainEntityOfPage":{"@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/"},"wordCount":902,"publisher":{"@id":"https:\/\/www.gsma.com\/newsroom\/#organization"},"image":{"@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#primaryimage"},"thumbnailUrl":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","articleSection":["Press Release"],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/","url":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/","name":"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0 - Newsroom","isPartOf":{"@id":"https:\/\/www.gsma.com\/newsroom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#primaryimage"},"image":{"@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#primaryimage"},"thumbnailUrl":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","datePublished":"2026-04-28T07:59:10+00:00","dateModified":"2026-04-28T09:00:51+00:00","breadcrumb":{"@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#primaryimage","url":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","contentUrl":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","width":600,"height":400,"caption":"Three women smiling and laughing together whilst looking at their mobile phones indoors."},{"@type":"BreadcrumbList","@id":"https:\/\/www.gsma.com\/newsroom\/press-release\/pleias-andgsma-launch-commonlingua-an-open-source-language-identification-model-supporting-61-african-languages\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.gsma.com\/newsroom\/"},{"@type":"ListItem","position":2,"name":"Pleias\u00a0and GSMA\u00a0Launch \u2018CommonLingua\u2019,\u00a0 an Open\u00a0Source\u00a0Language Identification Model supporting 61 African\u00a0Languages\u00a0"}]},{"@type":"WebSite","@id":"https:\/\/www.gsma.com\/newsroom\/#website","url":"https:\/\/www.gsma.com\/newsroom\/","name":"Newsroom","description":"","publisher":{"@id":"https:\/\/www.gsma.com\/newsroom\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.gsma.com\/newsroom\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/www.gsma.com\/newsroom\/#organization","name":"GSMA","url":"https:\/\/www.gsma.com\/newsroom\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/www.gsma.com\/newsroom\/#\/schema\/logo\/image\/","url":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/GSMA_logo_colour_web.jpg","contentUrl":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/GSMA_logo_colour_web.jpg","width":1000,"height":1000,"caption":"GSMA"},"image":{"@id":"https:\/\/www.gsma.com\/newsroom\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/gsma\/","https:\/\/x.com\/GSMA","https:\/\/www.instagram.com\/gsmaonline\/","https:\/\/www.linkedin.com\/company\/12380","https:\/\/www.youtube.com\/user\/GSMAOnline"]},{"@type":"Person","@id":"https:\/\/www.gsma.com\/newsroom\/#\/schema\/person\/e724eba4846a50206db8ca64bace5ceb","name":"ssharma@gsma.com"}]}},"jetpack_featured_media_url":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","jetpack_sharing_enabled":true,"featured_image_url":"https:\/\/www.gsma.com\/newsroom\/wp-content\/uploads\/\/women_Phone_Connectivity.png","_links":{"self":[{"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/posts\/25707"}],"collection":[{"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/users\/116"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/comments?post=25707"}],"version-history":[{"count":4,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/posts\/25707\/revisions"}],"predecessor-version":[{"id":25713,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/posts\/25707\/revisions\/25713"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/media\/25711"}],"wp:attachment":[{"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/media?parent=25707"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/categories?post=25707"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/tags?post=25707"},{"taxonomy":"algolia_discover_type","embeddable":true,"href":"https:\/\/www.gsma.com\/newsroom\/wp-json\/wp\/v2\/algolia_discover_type?post=25707"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}