MillID leaderboard

MillID (MultiLingual Language IDentifier) against GlotLID and Meta's fastText language identifier (NLLB). F1 is per language, then averaged with every language weighted equally (macro).

The models: MillID tiny v1.5, MillID medium v1, MillID large v1.5. Code, and running them in Python or JavaScript without a server: baobab-tech/millid.

Every score is on the test split of the build the models were trained on. The public baobabtech/millid-dataset is that build without Save the Children's and Nal'ibali's text, which is not ours to redistribute: the same splits, and a test split 1.3% smaller. udhr is the same everywhere.

MillID knows 1,641 labels, GlotLID over 2,000, fastText 218. Each external model is shown restricted to MillID's labels (its best answer among them), the like-for-like comparison; its as-is scores are below.
On test and udhr, fastText is mostly scored on languages it cannot answer. test-common and udhr-common keep the passages whose language all three know (about 210 languages): the head-to-head on equal terms.
Read GlotLID's scores with their bias in mind: the dataset's FinePDFs and FineWeb-2 passages were labelled by GlotLID, and GlotLID was trained on UDHR, so both favour it.
fastText's labels are mapped onto MillID's spelling: Simplified and Traditional Han as Hani, pes/prs as fas, est as ekk, tgl as fil.

Macro F1, every set

The three MillID models, and GlotLID and fastText restricted to MillID's labels. The best score on each set is in bold.

Set	Languages	tiny v1.5	medium v1	large v1.5	GlotLID	fastText
`test`	1,621	0.9554	0.9576	0.9646	0.9755	0.0659
`test-common`	213	0.9513	0.9527	0.9594	0.9846	0.9173
`udhr`	358	0.8764	0.8780	0.8812	0.8873	0.3499
`udhr-common`	170	0.9513	0.9480	0.9529	0.9659	0.9329
`udhr-all`	429	0.7149	0.7184	0.7201	0.7245	0.2797

test: every language, passages from documents never seen in training
test-common: the languages all three systems know
udhr: the Declaration of Human Rights, languages MillID knows
udhr-common: the Declaration, languages all three know
udhr-all: the Declaration, every language in it

test

Every language, passages from documents never seen in training. F1 ≥ 0.9: how many of its 1,621 languages a system reaches that F1 on. Top-5 and median F1 are in leaderboard_overall.csv.

System	Size	Top-1	Top-3	Macro P	Macro R	Macro F1	F1 ≥ 0.9
GlotLID	1.7 GB	0.9783	0.9978	0.9778	0.9765	0.9755	1,490
GlotLID, as is	1.7 GB	0.9776	0.9977	0.9783	0.9752	0.9748	1,488
large v1.5	192.6M	0.9674	0.9924	0.9655	0.9663	0.9646	1,478
medium v1	115.0M	0.9616	0.9910	0.9564	0.9614	0.9576	1,451
tiny v1.5	2.9M	0.9587	0.9890	0.9550	0.9588	0.9554	1,446
fastText	1.2 GB	0.2515	0.2611	0.0521	0.1209	0.0659	26
fastText, as is	1.2 GB	0.2511	0.2610	0.0520	0.1205	0.0657	25

test-common

The languages all three systems know. F1 ≥ 0.9: how many of its 213 languages a system reaches that F1 on. Top-5 and median F1 are in leaderboard_overall.csv.

System	Size	Top-1	Top-3	Macro P	Macro R	Macro F1	F1 ≥ 0.9
GlotLID	1.7 GB	0.9749	0.9927	0.9923	0.9790	0.9846	203
GlotLID, as is	1.7 GB	0.9738	0.9925	0.9924	0.9771	0.9835	201
large v1.5	192.6M	0.9525	0.9898	0.9757	0.9460	0.9594	193
medium v1	115.0M	0.9425	0.9882	0.9732	0.9361	0.9527	191
tiny v1.5	2.9M	0.9396	0.9857	0.9709	0.9353	0.9513	188
fastText	1.2 GB	0.9352	0.9710	0.9354	0.9201	0.9173	166
fastText, as is	1.2 GB	0.9339	0.9707	0.9357	0.9169	0.9149	165

udhr

The Declaration of Human Rights, languages MillID knows. F1 ≥ 0.9: how many of its 358 languages a system reaches that F1 on. Top-5 and median F1 are in leaderboard_overall.csv.

System	Size	Top-1	Top-3	Macro P	Macro R	Macro F1	F1 ≥ 0.9
GlotLID	1.7 GB	0.8614	0.9294	0.9258	0.8844	0.8873	291
GlotLID, as is	1.7 GB	0.8598	0.9280	0.9261	0.8826	0.8865	287
large v1.5	192.6M	0.8579	0.9183	0.9171	0.8825	0.8812	283
medium v1	115.0M	0.8524	0.9194	0.9187	0.8769	0.8780	282
tiny v1.5	2.9M	0.8492	0.9117	0.9148	0.8736	0.8764	274
fastText	1.2 GB	0.4629	0.4857	0.3152	0.4483	0.3499	61
fastText, as is	1.2 GB	0.4625	0.4857	0.3152	0.4479	0.3497	60

udhr-common

The Declaration, languages all three know. F1 ≥ 0.9: how many of its 170 languages a system reaches that F1 on. Top-5 and median F1 are in leaderboard_overall.csv.

System	Size	Top-1	Top-3	Macro P	Macro R	Macro F1	F1 ≥ 0.9
GlotLID	1.7 GB	0.9338	0.9683	0.9888	0.9579	0.9659	156
GlotLID, as is	1.7 GB	0.9336	0.9678	0.9888	0.9576	0.9657	155
large v1.5	192.6M	0.9168	0.9630	0.9800	0.9436	0.9529	152
tiny v1.5	2.9M	0.9123	0.9606	0.9794	0.9393	0.9513	149
medium v1	115.0M	0.9121	0.9633	0.9770	0.9382	0.9480	153
fastText	1.2 GB	0.9195	0.9647	0.9474	0.9441	0.9329	142
fastText, as is	1.2 GB	0.9187	0.9647	0.9474	0.9431	0.9323	141

udhr-all

The Declaration, every language in it. F1 ≥ 0.9: how many of its 429 languages a system reaches that F1 on. Top-5 and median F1 are in leaderboard_overall.csv.

System	Size	Top-1	Top-3	Macro P	Macro R	Macro F1	F1 ≥ 0.9
GlotLID, as is	1.7 GB	0.7502	0.8145	0.7695	0.7529	0.7454	277
GlotLID	1.7 GB	0.7378	0.7960	0.7425	0.7381	0.7245	272
large v1.5	192.6M	0.7348	0.7865	0.7392	0.7364	0.7201	266
medium v1	115.0M	0.7301	0.7875	0.7415	0.7318	0.7184	266
tiny v1.5	2.9M	0.7274	0.7809	0.7340	0.7290	0.7149	254
fastText	1.2 GB	0.3964	0.4160	0.2479	0.3741	0.2797	55
fastText, as is	1.2 GB	0.3961	0.4160	0.2479	0.3737	0.2795	54

Per language, test

Against GlotLID restricted, the best MillID is better on 165 languages and worse on 498; against fastText restricted, the best MillID is better on 1616 languages and worse on 0 (F1 differing by more than 0.01), of 1621. F1 per language: tiny is MillID tiny v1.5, medium is MillID medium v1, large is MillID large v1.5; GlotLID and fastText are restricted to MillID's labels. Every language, with their unrestricted F1 too, is in leaderboard_{set_name}.csv.

The 30 languages with the most passages:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`fra_Latn`	French	1652	0.929	0.952	0.960	0.928	0.808
`eng_Latn`	English	1444	0.917	0.916	0.934	0.859	0.648
`spa_Latn`	Spanish	1136	0.930	0.942	0.955	0.951	0.574
`tam_Taml`	Tamil	754	0.997	0.997	0.999	0.993	0.965
`arb_Arab`	Arabic (Modern Standard)	745	0.814	0.829	0.833	0.785	0.512
`por_Latn`	Portuguese	657	0.947	0.955	0.965	0.949	0.755
`kan_Knda`	Kannada	601	0.969	0.982	0.965	0.993	0.922
`zul_Latn`	Zulu	594	0.846	0.833	0.878	0.896	0.539
`swh_Latn`	Swahili	592	0.848	0.858	0.870	0.916	0.346
`mar_Deva`	Marathi	589	0.965	0.981	0.976	0.990	0.926
`tso_Latn`	Tsonga	581	0.965	0.970	0.978	0.956	0.425
`hun_Latn`	Hungarian	576	0.987	0.990	0.991	0.999	0.232
`afr_Latn`	Afrikaans	569	0.967	0.974	0.979	0.985	0.860
`tel_Telu`	Telugu	555	0.991	0.994	0.988	0.994	0.951
`rus_Cyrl`	Russian	528	0.932	0.925	0.949	0.949	0.462
`nbl_Latn`	South Ndebele	528	0.903	0.895	0.927	0.949	0.000
`ory_Orya`	Odia (individual language)	511	0.988	0.993	0.993	0.994	0.976
`sot_Latn`	Southern Sotho	491	0.966	0.966	0.972	0.978	0.660
`nso_Latn`	Northern Sotho	479	0.956	0.960	0.962	0.969	0.914
`kua_Latn`	Kuanyama	476	0.962	0.979	0.975	0.986	0.000
`asm_Beng`	Assamese	467	0.997	0.994	0.996	0.999	0.990
`ben_Beng`	Bangla	466	0.977	0.980	0.979	0.997	0.718
`pan_Guru`	Punjabi	460	1.000	1.000	1.000	1.000	0.990
`amh_Ethi`	Amharic	458	0.981	0.984	0.986	1.000	0.807
`tsn_Latn`	Tswana	456	0.945	0.939	0.947	0.961	0.652
`vro_Latn`	Võro	448	0.947	0.958	0.973	0.990	0.000
`gaz_Latn`	Oromo (West Central)	441	0.910	0.943	0.928	0.938	0.407
`som_Latn`	Somali	439	0.982	0.988	0.990	0.994	0.487
`uzn_Latn`	Northern Uzbek	436	0.976	0.978	0.982	0.994	0.449
`tha_Thai`	Thai	431	0.979	0.970	0.981	0.980	0.650

Where MillID gains most over GlotLID:

Label	Language	Passages	tiny	medium	large	GlotLID
`lbj_Tibt`	Ladakhi	66	0.943	0.943	0.923	0.000
`ain_Latn`	Ainu	1	0.667	0.667	0.667	0.000
`ajp_Arab`	South Levantine Arabic	28	0.946	0.902	0.902	0.581
`und_Laoo`	und	20	0.923	0.857	0.788	0.621
`sgh_Cyrl`	Shughni	9	0.857	0.900	0.889	0.667
`und_Thaa`	und	29	0.881	0.873	0.863	0.651
`icr_Latn`	Islander Creole English	24	0.698	0.870	0.700	0.684
`und_Geor`	und	32	0.877	0.706	0.792	0.694
`tdt_Latn`	Tetun Dili	57	0.693	0.656	0.782	0.603
`cek_Latn`	Eastern Khumi Chin	67	0.785	0.853	0.825	0.684
`und_Syrc`	und	43	0.966	0.977	0.989	0.822
`neb_Latn`	Toura (Côte d'Ivoire)	11	0.957	0.957	1.000	0.842
`dag_Latn`	Dagbani	149	0.767	0.817	0.814	0.660
`tuk_Arab`	Turkmen	135	0.905	0.903	0.917	0.761
`orv_Cyrl`	Old Russian	239	0.931	0.935	0.938	0.800

Where MillID loses most to GlotLID:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`mus_Latn`	Muscogee	2	0.333	0.667	0.667	1.000	0.000
`ttj_Latn`	Tooro	28	0.485	0.490	0.622	0.946	0.000
`knc_Arab`	Central Kanuri	68	0.462	0.509	0.532	0.825	0.007
`hrv_Latn`	Croatian	168	0.622	0.687	0.707	0.944	0.178
`esi_Latn`	North Alaskan Inupiatun	9	0.455	0.500	0.556	0.783	0.000
`hak_Hani`	Hakka Chinese	29	0.621	0.415	0.581	0.824	0.000
`bos_Latn`	Bosnian	198	0.649	0.673	0.700	0.902	0.181
`plw_Latn`	Brooke's Point Palawano	8	0.727	0.800	0.762	1.000	0.000
`ccp_Latn`	Chakma	3	0.800	0.333	0.750	1.000	0.000
`mbd_Latn`	Dibabawon Manobo	14	0.596	0.757	0.800	1.000	0.000
`taq_Tfng`	Tamasheq	104	0.529	0.535	0.527	0.718	0.000
`zsm_Latn`	Standard Malay	244	0.770	0.746	0.781	0.957	0.319
`biu_Latn`	Biete	7	0.667	0.667	0.833	1.000	0.000
`ldi_Latn`	Laari	62	0.777	0.831	0.765	0.992	0.000
`tzm_Tfng`	Central Atlas Tamazight	157	0.659	0.620	0.614	0.816	0.569

Per language, test-common

Against GlotLID restricted, the best MillID is better on 13 languages and worse on 114; against fastText restricted, the best MillID is better on 111 languages and worse on 31 (F1 differing by more than 0.01), of 213. F1 per language: tiny is MillID tiny v1.5, medium is MillID medium v1, large is MillID large v1.5; GlotLID and fastText are restricted to MillID's labels. Every language, with their unrestricted F1 too, is in leaderboard_{set_name}.csv.

The 30 languages with the most passages:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`fra_Latn`	French	1652	0.943	0.974	0.983	0.955	0.963
`eng_Latn`	English	1444	0.932	0.941	0.953	0.877	0.857
`spa_Latn`	Spanish	1136	0.937	0.953	0.972	0.958	0.941
`tam_Taml`	Tamil	754	0.997	0.998	0.999	1.000	0.997
`arb_Arab`	Arabic (Modern Standard)	745	0.871	0.905	0.905	0.886	0.954
`por_Latn`	Portuguese	657	0.959	0.963	0.973	0.956	0.949
`kan_Knda`	Kannada	601	0.973	0.987	0.967	0.994	0.999
`zul_Latn`	Zulu	594	0.883	0.871	0.909	0.904	0.970
`swh_Latn`	Swahili	592	0.893	0.897	0.909	0.953	0.945
`mar_Deva`	Marathi	589	0.968	0.981	0.979	0.990	0.987
`tso_Latn`	Tsonga	581	0.979	0.984	0.985	0.956	0.986
`hun_Latn`	Hungarian	576	0.989	0.991	0.993	1.000	0.980
`afr_Latn`	Afrikaans	569	0.972	0.976	0.983	0.985	0.986
`tel_Telu`	Telugu	555	0.997	1.000	0.999	0.998	0.999
`rus_Cyrl`	Russian	528	0.945	0.929	0.956	0.985	0.927
`ory_Orya`	Odia (individual language)	511	0.994	0.996	0.995	1.000	0.996
`sot_Latn`	Southern Sotho	491	0.969	0.967	0.974	0.978	0.856
`nso_Latn`	Northern Sotho	479	0.958	0.961	0.964	0.970	0.967
`asm_Beng`	Assamese	467	0.997	0.995	0.997	0.999	0.997
`ben_Beng`	Bangla	466	0.988	0.987	0.993	0.997	0.989
`pan_Guru`	Punjabi	460	1.000	1.000	1.000	1.000	0.990
`amh_Ethi`	Amharic	458	0.984	0.987	0.986	1.000	0.998
`tsn_Latn`	Tswana	456	0.948	0.940	0.947	0.961	0.870
`gaz_Latn`	Oromo (West Central)	441	0.935	0.955	0.941	0.974	0.973
`som_Latn`	Somali	439	0.984	0.992	0.991	0.995	0.991
`uzn_Latn`	Northern Uzbek	436	0.984	0.984	0.986	0.999	0.985
`tha_Thai`	Thai	431	0.980	0.970	0.981	0.980	0.981
`ydd_Hebr`	Eastern Yiddish	430	0.993	0.998	0.994	1.000	0.999
`tur_Latn`	Turkish	430	0.968	0.974	0.973	0.999	0.980
`nno_Latn`	Norwegian Nynorsk	427	0.913	0.919	0.927	0.985	0.954

Where MillID gains most over GlotLID:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`ewo_Latn`	Ewondo	134	0.959	0.965	0.978	0.883	0.860
`eng_Latn`	English	1444	0.932	0.941	0.953	0.877	0.857
`arb_Latn`	Standard Arabic	163	0.942	0.944	0.949	0.913	0.852
`tso_Latn`	Tsonga	581	0.979	0.984	0.985	0.956	0.986
`fra_Latn`	French	1652	0.943	0.974	0.983	0.955	0.963
`arb_Arab`	Arabic (Modern Standard)	745	0.871	0.905	0.905	0.886	0.954
`por_Latn`	Portuguese	657	0.959	0.963	0.973	0.956	0.949
`hin_Deva`	Hindi	392	0.855	0.817	0.893	0.877	0.863
`fuv_Latn`	Nigerian Fulfulde	164	0.907	0.797	0.844	0.892	0.859
`kam_Latn`	Kamba	272	0.980	0.980	0.989	0.974	0.863
`knc_Latn`	Central Kanuri	293	0.967	0.959	0.967	0.952	0.981
`spa_Latn`	Spanish	1136	0.937	0.953	0.972	0.958	0.941
`bug_Latn`	Buginese	78	0.981	0.987	0.994	0.981	0.938
`bjn_Arab`	Banjar	116	0.992	0.983	0.996	0.987	0.960
`san_Deva`	Sanskrit	239	0.990	0.996	0.998	0.990	0.971

Where MillID loses most to GlotLID:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`knc_Arab`	Central Kanuri	68	0.522	0.562	0.598	0.860	0.029
`bos_Latn`	Bosnian	198	0.685	0.727	0.762	1.000	0.897
`tzm_Tfng`	Central Atlas Tamazight	157	0.697	0.645	0.649	0.934	0.751
`hrv_Latn`	Croatian	168	0.659	0.717	0.757	0.994	0.856
`zsm_Latn`	Standard Malay	244	0.789	0.773	0.809	0.998	0.908
`taq_Tfng`	Tamasheq	104	0.654	0.613	0.604	0.805	0.000
`ind_Latn`	Indonesian	335	0.842	0.807	0.848	0.991	0.887
`nob_Latn`	Norwegian Bokmål	265	0.821	0.820	0.832	0.959	0.894
`kin_Latn`	Kinyarwanda	210	0.849	0.818	0.864	0.981	0.854
`run_Latn`	Rundi	291	0.883	0.843	0.879	0.985	0.855
`mag_Deva`	Magahi	97	0.890	0.905	0.897	1.000	0.772
`cjk_Latn`	Chokwe	24	0.857	0.909	0.837	1.000	0.807
`taq_Latn`	Tamasheq	106	0.902	0.909	0.922	1.000	0.838
`dzo_Tibt`	Dzongkha	346	0.906	0.926	0.916	0.999	0.696
`lim_Latn`	Limburgish	196	0.896	0.915	0.930	1.000	0.956

Per language, udhr

Against GlotLID restricted, the best MillID is better on 75 languages and worse on 57; against fastText restricted, the best MillID is better on 313 languages and worse on 8 (F1 differing by more than 0.01), of 358. F1 per language: tiny is MillID tiny v1.5, medium is MillID medium v1, large is MillID large v1.5; GlotLID and fastText are restricted to MillID's labels. Every language, with their unrestricted F1 too, is in leaderboard_{set_name}.csv.

The 30 languages with the most passages:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`roh_Latn`	Romansh	399	0.994	0.991	0.998	0.999	0.946
`oci_Latn`	Occitan	398	0.277	0.331	0.338	0.376	0.435
`cmn_Hani`	Chinese (Mandarin)	319	0.892	0.886	0.875	0.868	0.468
`krl_Latn`	Karelian	183	0.000	0.000	0.000	0.011	0.000
`qug_Latn`	Chimborazo Highland Quichua	182	0.011	0.000	0.000	0.000	0.000
`acu_Latn`	Achuar-Shiwiar	180	0.828	0.743	0.801	0.854	0.000
`hus_Latn`	Huastec	175	0.977	0.991	0.988	0.986	0.000
`hau_Latn`	Hausa	155	0.990	0.997	1.000	1.000	0.876
`auc_Latn`	Waorani	154	0.000	0.050	0.025	0.000	0.000
`kwi_Latn`	Awa-Cuaiquer	145	0.880	0.775	0.863	0.877	0.000
`jpn_Jpan`	Japanese	140	1.000	1.000	1.000	1.000	0.944
`snn_Latn`	Siona	136	0.630	0.773	0.467	0.514	0.000
`ktu_Latn`	Kituba (Democratic Republic of Congo)	118	0.765	0.769	0.773	0.680	0.000
`lat_Latn`	Latin	118	0.996	0.996	0.996	0.992	0.000
`hat_Latn`	Haitian Creole	117	0.992	0.991	0.987	0.979	0.699
`nya_Latn`	Nyanja	116	0.991	0.987	0.978	1.000	0.806
`kmb_Latn`	Kimbundu	116	0.970	0.979	0.991	1.000	0.877
`rmn_Latn`	Balkan Romani	116	0.678	0.667	0.704	0.857	0.000
`twi_Latn`	Twi	116	0.983	0.987	0.991	0.996	0.396
`chr_Cher`	Cherokee	115	0.670	0.715	0.945	0.663	0.000
`por_Latn`	Portuguese	115	0.955	0.996	1.000	0.996	0.793
`lvs_Latn`	Standard Latvian	115	0.987	0.996	1.000	1.000	0.991
`pov_Latn`	Upper Guinea Crioulo	115	0.821	0.838	0.827	0.833	0.000
`kng_Latn`	Koongo	115	0.076	0.047	0.047	0.193	0.457
`ell_Grek`	Greek	115	0.670	0.655	0.670	1.000	1.000
`tha_Thai`	Thai	114	1.000	1.000	1.000	1.000	0.996
`tso_Latn`	Tsonga	114	0.860	0.882	0.774	0.781	0.691
`cjk_Latn`	Chokwe	114	0.871	0.949	0.959	1.000	0.533
`fas_Arab`	Persian	114	0.991	0.991	0.982	1.000	0.934
`lin_Latn`	Lingala	110	0.991	1.000	0.996	1.000	0.986

Where MillID gains most over GlotLID:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`mic_Latn`	Mi'kmaw	59	0.992	1.000	1.000	0.427	0.000
`tet_Latn`	Tetum	58	0.638	0.699	0.708	0.167	0.000
`fuf_Latn`	Pular	58	0.906	0.816	0.936	0.483	0.000
`skr_Arab`	Saraiki	63	0.867	0.923	0.875	0.500	0.000
`sag_Latn`	Sango	59	0.992	0.992	0.992	0.578	0.189
`zgh_Tfng`	Standard Moroccan Tamazight	60	0.750	0.824	0.652	0.462	0.000
`qvh_Latn`	Huamalíes-Dos de Mayo Huánuco Quechua	58	0.468	0.215	0.215	0.154	0.000
`chr_Cher`	Cherokee	115	0.670	0.715	0.945	0.663	0.000
`tzm_Tfng`	Central Atlas Tamazight	1	0.167	0.333	0.100	0.065	0.033
`snn_Latn`	Siona	136	0.630	0.773	0.467	0.514	0.000
`fkv_Latn`	Kven Finnish	58	0.635	0.697	0.906	0.667	0.000
`bum_Latn`	Bulu	58	0.215	0.319	0.188	0.098	0.000
`mxv_Latn`	Metlatónoc Mixtec	58	0.129	0.215	0.034	0.000	0.000
`pnb_Arab`	Western Panjabi	61	0.882	0.931	0.884	0.739	0.000
`lld_Latn`	Ladin	59	0.909	0.899	0.881	0.731	0.000

Where MillID loses most to GlotLID:

Label	Language	Passages	tiny	medium	large	GlotLID	fastText
`gyr_Latn`	Guarayu	49	0.537	0.339	0.393	0.990	0.000
`dyu_Latn`	Dyula	58	0.000	0.000	0.000	0.434	0.015
`ell_Grek`	Greek	115	0.670	0.655	0.670	1.000	1.000
`cbu_Latn`	Candoshi-Shapra	57	0.500	0.371	0.557	0.874	0.000
`shp_Latn`	Shipibo-Conibo	62	0.469	0.410	0.607	0.912	0.000
`cpu_Latn`	Pichis Ashéninka	60	0.729	0.571	0.571	0.974	0.000
`qvn_Latn`	North Junín Quechua	57	0.219	0.219	0.034	0.460	0.000
`hin_Deva`	Hindi	59	0.723	0.607	0.727	0.957	0.975
`cni_Latn`	Asháninka	52	0.773	0.708	0.717	0.963	0.000
`hrv_Latn`	Croatian	58	0.500	0.537	0.559	0.727	0.474
`rmn_Latn`	Balkan Romani	116	0.678	0.667	0.704	0.857	0.000
`koi_Cyrl`	Komi-Permyak	57	0.837	0.837	0.849	1.000	0.000
`qvm_Latn`	Margos-Yarowilca-Lauricocha Quechua	60	0.356	0.094	0.356	0.500	0.000
`kea_Latn`	Kabuverdianu	58	0.792	0.794	0.773	0.935	0.124
`ind_Latn`	Indonesian	60	0.768	0.674	0.645	0.903	0.565

baobabtech/millid

MillID leaderboard

Macro F1, every set

test

test-common

udhr

udhr-common

udhr-all

Per language, test

Per language, test-common

Per language, udhr

为此模型做种