Loading…
Loading…
Jarbas/m2v-256-distiluse-base-multilingual-cased-v2
encoder/model.safetensors 122 MBmodel.safetensors 122 MBhead.safetensors 1 MBA static-embedding intent classifier for the Thalovant hub. It maps an
utterance to one of 360 labels of the form <skill_id>:<intent>,
exactly as ovos-m2v-pipeline registers them. The labels are the fleet's
skills and the other intents the hubs register (the persona and reading
pipelines, wikihow), from the fleet corpus and nothing else. There is no
"none of these" label: a sentence about something no skill does still gets
the nearest skill's label, often with high confidence.
Base: Jarbas/m2v-256-distiluse-base-multilingual-cased-v2, fine-tuned with model2vec's classifier trainer.
Measured on sentences the classifier never saw: 102049 held-out rows whose text, in any language, and whose intent line are both absent from the 406478 rows an evaluation model was trained on. A split per row would score the en-GB twin of an en-US training sentence as new.
thalovant-skillkit check
would report about a sentence that is in fact new and the skill's own.ar-SA 0.98, ast-ES 0.96, az-AZ 0.99, bg-BG 0.96, cs-CZ 0.94, el-GR 0.97, et-EE 0.99, eu-ES 0.97, fi-FI 0.94, he-IL 0.94, hi-IN 0.995, hu-HU 0.94, id-ID 0.92, lt-LT 0.96, lv-LV 0.98, nb-NO 0.94, nl-BE 0.92, nl-NL 0.92, pl-PL 0.97, pt-AO 0.92, pt-BR 0.92, pt-MZ 0.92, pt-PT 0.92, ro-RO 0.94, ru-RU 0.92, sk-SK 0.96, sl-SI 0.98, sw-KE 0.97, th-TH 0.92, uk-UA 0.99, vi-VN 0.95 (thresholds in
training.json).The published weights are then trained with the same recipe on all
508527 rows, so a skill's existing sentences are the
classifier's training data. At most 512 sentences per label and language
are trained on (the reading pipeline's English intents alone expand to
thousands); index.json keeps all 553440.
from model2vec.inference import StaticModelPipeline
model = StaticModelPipeline.from_pretrained("thalovant/thalovant-m2v-intents")
model.predict(["will it rain tomorrow"])
Configure the pipeline with "mode": "classifier" and this model's path;
labels.json beside the weights lists the labels the hub may route to.
index.json names every sentence in the training corpus by a digest of its
language and text, with the labels that publish it. thalovant-skillkit check
uses it to fail a skill that publishes a sentence another skill already owns,
and asks this classifier what it makes of the rest -- without the fleet's
sentences leaving their private repositories.
False warnings: held-out sentences given to another skill at 0.9 or more. Threshold: the lowest that keeps them under 1.0 %.
| Language | Rows | Intent | Skill | False warnings | Threshold |
|---|---|---|---|---|---|
an-ES |
895 | 94.0 % | 98.0 % | 0.4 % | 0.9 |
ar-SA |
797 | 82.8 % | 92.6 % | 1.5 % | 0.98 |
ast-ES |
714 | 94.7 % | 96.4 % | 1.3 % | 0.96 |
az-AZ |
488 | 77.7 % | 89.5 % | 2.2 % | 0.99 |
bg-BG |
613 | 91.3 % | 96.6 % | 1.3 % | 0.96 |
ca-ES |
3484 | 93.2 % | 96.7 % | 0.5 % | 0.9 |
cs-CZ |
963 | 81.9 % | 91.9 % | 1.1 % | 0.94 |
da-DK |
1153 | 90.2 % | 94.2 % | 0.4 % | 0.9 |
de-AT |
2558 | 91.5 % | 96.3 % | 0.6 % | 0.9 |
de-CH |
2513 | 91.4 % | 96.2 % | 0.6 % | 0.9 |
de-DE |
2935 | 91.0 % | 96.4 % | 0.6 % | 0.9 |
el-GR |
1144 | 79.5 % | 89.8 % | 2.5 % | 0.97 |
en-AU |
2927 | 95.4 % | 98.3 % | 0.2 % | 0.9 |
en-CA |
2868 | 95.3 % | 98.2 % | 0.2 % | 0.9 |
en-GB |
2927 | 95.4 % | 98.3 % | 0.2 % | 0.9 |
en-IE |
45 | 91.1 % | 93.3 % | 2.2 % | 0.9 |
en-NZ |
2927 | 95.4 % | 98.3 % | 0.2 % | 0.9 |
en-US |
3738 | 95.2 % | 98.1 % | 0.3 % | 0.9 |
es-AR |
2236 | 96.1 % | 97.6 % | 0.1 % | 0.9 |
es-CO |
2326 | 96.2 % | 97.6 % | 0.1 % | 0.9 |
es-ES |
2770 | 95.4 % | 97.2 % | 0.2 % | 0.9 |
es-MX |
2309 | 96.2 % | 97.6 % | 0.1 % | 0.9 |
es-US |
2309 | 96.2 % | 97.6 % | 0.1 % | 0.9 |
et-EE |
523 | 79.3 % | 87.6 % | 2.5 % | 0.99 |
eu-ES |
1211 | 84.5 % | 91.2 % | 2.2 % | 0.97 |
fa-IR |
937 | 83.9 % | 95.9 % | 0.5 % | 0.9 |
fi-FI |
679 | 83.4 % | 90.3 % | 1.5 % | 0.94 |
fr-BE |
3004 | 91.6 % | 96.5 % | 0.9 % | 0.9 |
fr-CA |
3043 | 91.8 % | 96.5 % | 0.9 % | 0.9 |
fr-CH |
3004 | 91.6 % | 96.5 % | 0.9 % | 0.9 |
fr-FR |
3543 | 92.5 % | 96.9 % | 0.8 % | 0.9 |
gl-ES |
3114 | 94.3 % | 96.2 % | 0.8 % | 0.9 |
he-IL |
498 | 81.5 % | 90.4 % | 1.4 % | 0.94 |
hi-IN |
1830 | 82.4 % | 90.6 % | 3.4 % | 0.995 |
hr-HR |
581 | 89.3 % | 95.5 % | 0.9 % | 0.9 |
hu-HU |
969 | 79.9 % | 88.0 % | 1.0 % | 0.94 |
id-ID |
674 | 84.4 % | 96.0 % | 1.0 % | 0.92 |
it-CH |
61 | 88.5 % | 90.2 % | 4.9 % | 0.9 |
it-IT |
2511 | 92.3 % | 97.0 % | 0.8 % | 0.9 |
ja-JP |
706 | 75.6 % | 92.6 % | 0.7 % | 0.9 |
kab |
51 | 72.5 % | 94.1 % | 0.0 % | 0.9 |
ko-KR |
884 | 69.3 % | 89.9 % | 0.7 % | 0.9 |
lt-LT |
515 | 79.0 % | 88.0 % | 2.7 % | 0.96 |
lv-LV |
578 | 80.1 % | 87.0 % | 2.2 % | 0.98 |
ms-MY |
584 | 90.1 % | 94.3 % | 0.7 % | 0.9 |
nb-NO |
645 | 91.0 % | 95.3 % | 1.2 % | 0.94 |
nl-BE |
1266 | 90.1 % | 94.7 % | 1.1 % | 0.92 |
nl-NL |
2084 | 92.1 % | 95.6 % | 1.0 % | 0.92 |
oc-FR |
749 | 83.9 % | 95.5 % | 0.7 % | 0.9 |
pl-PL |
1098 | 84.1 % | 90.1 % | 2.1 % | 0.97 |
pt-AO |
2649 | 94.4 % | 96.2 % | 1.2 % | 0.92 |
pt-BR |
1538 | 92.6 % | 94.9 % | 1.0 % | 0.92 |
pt-MZ |
2649 | 94.4 % | 96.2 % | 1.2 % | 0.92 |
pt-PT |
3163 | 94.7 % | 96.3 % | 1.1 % | 0.92 |
ro-RO |
607 | 86.8 % | 92.6 % | 1.3 % | 0.94 |
ru-RU |
1014 | 87.9 % | 96.0 % | 1.1 % | 0.92 |
sk-SK |
815 | 76.8 % | 88.7 % | 2.6 % | 0.96 |
sl-SI |
605 | 88.4 % | 92.4 % | 2.0 % | 0.98 |
sv-FI |
724 | 90.6 % | 95.0 % | 1.0 % | 0.9 |
sv-SE |
807 | 90.2 % | 94.9 % | 1.0 % | 0.9 |
sw-KE |
460 | 79.6 % | 91.1 % | 2.2 % | 0.97 |
th-TH |
884 | 82.5 % | 91.4 % | 1.0 % | 0.92 |
tr-TR |
629 | 83.5 % | 92.8 % | 0.6 % | 0.9 |
uk-UA |
1035 | 68.3 % | 82.1 % | 5.4 % | 0.99 |
vi-VN |
606 | 85.0 % | 91.9 % | 1.8 % | 0.95 |
zh-CN |
761 | 96.2 % | 98.6 % | 0.3 % | 0.9 |
zh-SG |
761 | 96.2 % | 98.6 % | 0.3 % | 0.9 |
zh-TW |
1361 | 97.0 % | 99.0 % | 0.1 % | 0.9 |
| Skill | Rows | Intent | Skill | False warnings |
|---|---|---|---|---|
ovos-common-reading-pipeline-plugin |
695 | 98.3 % | 98.9 % | 0.1 % |
ovos-persona-pipeline-plugin |
1576 | 94.5 % | 94.9 % | 0.4 % |
ovos-skill-wikihow.openvoiceos |
387 | 95.6 % | 95.6 % | 1.0 % |
thalovant-skill-alarm.thalovant |
1975 | 92.7 % | 96.2 % | 0.9 % |
thalovant-skill-camera.thalovant |
693 | 75.3 % | 82.1 % | 3.9 % |
thalovant-skill-christmas-companion.thalovant |
457 | 81.0 % | 90.6 % | 1.8 % |
thalovant-skill-christmas-games.thalovant |
455 | 96.9 % | 97.4 % | 1.1 % |
thalovant-skill-christmas-protocol.thalovant |
169 | 85.2 % | 100.0 % | 0.0 % |
thalovant-skill-coming-soon.thalovant |
2973 | 98.5 % | 98.7 % | 0.3 % |
thalovant-skill-custos-query.thalovant |
340 | 84.1 % | 88.2 % | 2.1 % |
thalovant-skill-custos-shadow.thalovant |
291 | 95.9 % | 97.2 % | 0.0 % |
thalovant-skill-date-time.thalovant |
14071 | 95.7 % | 98.5 % | 0.4 % |
thalovant-skill-easter-eggs.thalovant |
1542 | 86.8 % | 88.7 % | 2.4 % |
thalovant-skill-fart.thalovant |
2130 | 82.4 % | 93.8 % | 0.5 % |
thalovant-skill-guide.thalovant |
2944 | 85.2 % | 92.1 % | 1.5 % |
thalovant-skill-home.thalovant |
18230 | 87.5 % | 94.2 % | 1.5 % |
thalovant-skill-interaction-modes.thalovant |
468 | 84.6 % | 94.7 % | 0.4 % |
thalovant-skill-joke-garden.thalovant |
2685 | 89.2 % | 91.8 % | 1.4 % |
thalovant-skill-language-buddy.thalovant |
421 | 85.0 % | 85.0 % | 3.6 % |
thalovant-skill-learning-lounge.thalovant |
452 | 86.7 % | 86.7 % | 3.1 % |
thalovant-skill-lists.thalovant |
85 | 74.1 % | 95.3 % | 0.0 % |
thalovant-skill-local-pulse.thalovant |
1783 | 94.0 % | 94.0 % | 2.2 % |
thalovant-skill-media.thalovant |
7212 | 97.0 % | 98.5 % | 0.2 % |
thalovant-skill-music-assistant.thalovant |
2660 | 80.8 % | 94.9 % | 0.4 % |
thalovant-skill-news.thalovant |
1613 | 94.7 % | 96.8 % | 0.6 % |
thalovant-skill-on-this-day.thalovant |
1092 | 96.2 % | 96.6 % | 0.2 % |
thalovant-skill-ops-copilot.thalovant |
492 | 95.1 % | 95.1 % | 0.8 % |
thalovant-skill-pop-facts.thalovant |
3190 | 93.0 % | 96.8 % | 0.8 % |
thalovant-skill-pop-quiz.thalovant |
3533 | 99.3 % | 99.3 % | 0.2 % |
thalovant-skill-recall.thalovant |
1604 | 79.3 % | 90.6 % | 1.4 % |
thalovant-skill-reminder.thalovant |
1733 | 90.0 % | 95.2 % | 0.5 % |
thalovant-skill-rewrite-forge.thalovant |
2259 | 96.1 % | 96.1 % | 0.3 % |
thalovant-skill-routines.thalovant |
1137 | 90.9 % | 98.9 % | 0.1 % |
thalovant-skill-safety-guide.thalovant |
703 | 90.9 % | 90.9 % | 1.8 % |
thalovant-skill-source-scout.thalovant |
840 | 85.5 % | 85.5 % | 1.9 % |
thalovant-skill-stopwatch.thalovant |
1012 | 81.4 % | 93.9 % | 1.3 % |
thalovant-skill-timer.thalovant |
1771 | 95.1 % | 98.4 % | 0.2 % |
thalovant-skill-twenty-questions.thalovant |
301 | 93.7 % | 93.7 % | 0.3 % |
thalovant-skill-volume.thalovant |
1195 | 87.5 % | 97.7 % | 0.6 % |
thalovant-skill-weather.thalovant |
12274 | 87.4 % | 95.3 % | 0.8 % |
thalovant-skill-what-next.thalovant |
2606 | 98.2 % | 98.2 % | 0.4 % |
Each line is a held-out sentence whose true intent the classifier read as another intent. Two skills on one line is a collision the fleet should resolve.
thalovant-skill-home.thalovant:home.turn.on read as thalovant-skill-home.thalovant:home.turn.off (222): "ممكن تشغل الضوء في something" (ar-SA)thalovant-skill-home.thalovant:home.turn.on read as thalovant-skill-media.thalovant:media.play.title (215): "شغل ضوء something" (ar-SA)thalovant-skill-weather.thalovant:high_low.weather read as thalovant-skill-weather.thalovant:temperature.weather (191): "qué fredo ferá?" (an-ES)thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.turn.off (158): "son as luzes de something amortadas" (an-ES)thalovant-skill-home.thalovant:home.state read as thalovant-skill-weather.thalovant:air_quality.weather (147): "ランプはオンですか" (ja-JP)thalovant-skill-recall.thalovant:what.did.i.ask read as thalovant-skill-recall.thalovant:did.i.ask (139): "was hab ich dich über something gefragt" (de-DE)thalovant-skill-home.thalovant:home.brightness read as thalovant-skill-home.thalovant:home.state (105): "dimm das licht im something" (de-DE)thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.turn.on (103): "something kondisioner işləyir mi" (az-AZ)thalovant-skill-date-time.thalovant:weekday.for.date read as thalovant-skill-date-time.thalovant:holiday.when (97): "quin dia de la setmana és cap d'any" (ca-ES)thalovant-skill-home.thalovant:home.state read as thalovant-skill-weather.thalovant:temperature.weather (92): "hvad er temperaturen i stuen" (da-DK)thalovant-skill-weather.thalovant:condition.weather read as thalovant-skill-weather.thalovant:forecast.weather (90): "هل هو غائم" (ar-SA)thalovant-skill-weather.thalovant:condition.weather read as thalovant-skill-weather.thalovant:temperature.weather (82): "ist es draußen bewölkt" (de-AT)thalovant-skill-home.thalovant:home.turn.off read as thalovant-skill-home.thalovant:home.state (69): "قهوهساز something را خاموش کن" (fa-IR)thalovant-skill-date-time.thalovant:days.until read as thalovant-skill-timer.thalovant:timer.remaining (65): "cuánto tiempo queda para navidad" (es-AR)thalovant-skill-home.thalovant:home.state read as thalovant-skill-home.thalovant:home.cover.open (57): "are the blinds open" (en-AU)Trained from the corpus built 2026-11-13T21:14:58Z covering 41 skills and sources, expanded by thalovant-skillkit 0.37.0.