Skip to main content.

LAW-MWE-CxG 2018 (COLING): Shared task results

Results of the PARSEME shared task - edition 1.1 (2018)

Last updated: Oct 05, 2020

• NEW! [Sep 15, 2020] We have changed the definition of seen VMWEs. We now consider unseen VMWEs with respect to train+dev instead of train only (see the reasons for this change). This page contains the results conforming to the previous definition. We will soon proceed to updating them.

Participants

For the 2018 edition of the PARSEME shared task, 12 teams submitted 17 system results to the shared task: 13 to the closed track and 4 to the open track. No team submitted system results for the 20 languages of the shared task, but 11 teams submitted system results covering 19 languages (all except Arabic). The teams are listed below (with links to description papers/pages when available):

• CRF-DepTree-categs and CRF-Seq-nocategs (closed)
  • Paper
  • Code
• GBD-NER-resplit and GBD-NER-standard (closed)
  • Paper
  • Code
  • Unofficial evaluation results
• MWETreeC (closed)
• Polirem-basic and Polirem-rich (closed)
• TRAPACC and TRAPACC_S (closed)
  • Paper
  • Code
• TRAVERSAL (closed)
  • Paper
  • Code
• varIDE (closed)
  • Paper
  • Code
• Veyn (closed)
  • Paper
  • Code
• mumpitz and mumpitz-preinit (closed and open)
  • Paper
• Deep-BGT (open)
  • Paper
• Milos (open)
  • Paper
• SHOMA (open)
  • Paper
  • Code

The full set of VMWE predictions for the test corpora provided by each system is available on our gitlab repository.

Results description

The evaluation results are given below. First, we present cross-lingual rankings averaged over languages, both for the general metrics and for metrics focusing on specific phenomena (discontinuity, variability, etc). Second, we present rankings for each language separately. Third, we present non-ranked results of systems per language and per VMWE category.

1. Cross-lingual macro-averages
   • General ranking
   • Continuity: Continuous VMWEs | Discontinuous VMWEs
   • Number of tokens: Multi-token VMWEs | Single-token VMWEs
   • Novelty: Seen-in-train VMWEs | Unseen-in-train VMWEs
   • Variability: Variant-of-train VMWEs | Identical-to-train VMWEs
2. Language-specific system rankings
   • BG | DE | EL | EN | ES | EU | FA | FR | HE | HI | HR | HU | IT | LT | PL | PT | RO | SL | TR
3. Results per VMWE category (not ranked)
   • BG | DE | EL | EN | ES | EU | FA | FR | HE | HI | HR | HU | IT | LT | PL | PT | RO | SL | TR

Open-track and closed-track systems are shown in the same table, separated by a thick line. We report MWE-based and Token-based precision (P), recall (R) and F-measure (F1), as described on the evaluation metrics page.

Macro-average calculation ranges over all 19 non-optional languages. If a system did not submit results for a given language, this language is included in macro-average calculation as having P=R=F1=0. Macro-averaged ranking tables indicate the number of languages (#Langs) for which the system provided results. For phenomenon-specific macro-averaged rankings, only MWE-based measures are reported, and languages not containing a given phenomenon are ignored (only relevant for Single-token VMWEs).

Remember that the aim of this shared task is not to rank systems/participants, but to encourage a cross-lingual discussion on VMWE identification models. The participation policy may have introduced bias to the annotated corpora and the participating systems. Therefore, the rankings should be considered with caution.

Cross-lingual macro-averages

General ranking

System	Track	#Langs	MWE-based				Token-based
			P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	19/19	67.58	44.97	54	1	77.41	48.55	59.67	1
TRAPACC_S	closed	19/19	62.28	41.4	49.74	2	68.54	42.06	52.13	4
TRAPACC	closed	19/19	55.68	44.67	49.57	3	62.1	46.37	53.09	3
CRF-Seq-nocategs	closed	19/19	56.13	39.12	46.11	4	73.44	43.49	54.63	2
varIDE	closed	19/19	61.49	36.71	45.97	5	64.13	37.63	47.43	6
CRF-DepTree-categs	closed	19/19	52.33	37.83	43.91	6	64.65	41.56	50.6	5
GBD-NER-standard	closed	19/19	36.56	48.3	41.62	7	41.11	52.21	46	7
GBD-NER-resplit	closed	19/19	30.26	52.95	38.51	8	33.83	58.03	42.74	9
Veyn	closed	19/19	42.76	32.51	36.94	9	58.13	36.57	44.9	8
mumpitz	closed	19/7	17.14	13.03	14.81	10	24.95	15.5	19.12	11
Polirem-rich	closed	19/3	10.9	2.87	4.54	11	13.07	3.89	6	12
Polirem-basic	closed	19/3	10.78	0.65	1.23	12	11.33	0.68	1.28	13
MWETreeC	closed	19/19	0.21	3.72	0.4	13	23.5	24.78	24.12	10
SHOMA	open	19/19	66.08	51.82	58.09	1	76.22	54.27	63.4	1
Deep-BGT	open	19/10	33.41	25.29	28.79	2	39.77	26.47	31.78	2
Milos	open	19/4	9.17	7.87	8.47	3	11.5	8.25	9.61	3
mumpitz-preinit	open	19/1	2.28	1.9	2.07	4	3.71	2.35	2.88	4

↑ Back to top

Continuous VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
TRAVERSAL	closed	19/19	68.19	49.78	57.55	1
TRAPACC_S	closed	19/19	65.12	48.18	55.38	2
TRAPACC	closed	19/19	59.09	51.99	55.31	3
CRF-Seq-nocategs	closed	19/19	54.99	49.84	52.29	4
varIDE	closed	19/19	78.03	37.98	51.09	5
CRF-DepTree-categs	closed	19/19	52.8	42.44	47.06	6
GBD-NER-standard	closed	19/19	38.76	55.2	45.54	7
GBD-NER-resplit	closed	19/19	33.5	57.92	42.45	8
Veyn	closed	19/19	41.76	37.76	39.66	9
mumpitz	closed	7/19	16.83	15.32	16.04	10
Polirem-rich	closed	3/19	10.9	4.78	6.65	11
Polirem-basic	closed	3/19	10.78	1.09	1.98	12
MWETreeC	closed	19/19	0.21	4.21	0.4	13
SHOMA	open	19/19	66.07	59.73	62.74	1
Deep-BGT	open	10/19	36.05	27.54	31.23	2
Milos	open	4/19	9.42	9.49	9.45	3
mumpitz-preinit	open	1/19	1.97	2.31	2.13	4

↑ Back to top

Discontinuous VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
TRAVERSAL	closed	19/19	61.14	34.81	44.36	1
varIDE	closed	19/19	44.53	32.24	37.4	2
CRF-DepTree-categs	closed	19/19	48.8	26.4	34.26	3
TRAPACC_S	closed	19/19	53.23	24.88	33.91	4
TRAPACC	closed	19/19	43.29	27.3	33.48	5
GBD-NER-standard	closed	19/19	29.32	33.69	31.35	6
GBD-NER-resplit	closed	19/19	23.22	41.41	29.76	7
Veyn	closed	19/19	40.53	19.07	25.94	8
CRF-Seq-nocategs	closed	19/19	54.2	15.48	24.08	9
mumpitz	closed	7/19	18.34	8.71	11.81	10
Polirem-rich	closed	3/19	3.51	0.06	0.12	11
Polirem-basic	closed	3/19	0	0	0	n/a
MWETreeC	closed	19/19	0	0	0	n/a
SHOMA	open	19/19	62.95	32.87	43.19	1
Deep-BGT	open	10/19	28.83	19.4	23.19	2
Milos	open	4/19	9.37	5.79	7.16	3
mumpitz-preinit	open	1/19	3.25	1.44	2	4

↑ Back to top

Multi-token VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
TRAVERSAL	closed	19/19	74.66	44.59	55.83	1
TRAPACC	closed	19/19	57.23	43.42	49.38	2
TRAPACC_S	closed	19/19	63.6	39.97	49.09	3
CRF-Seq-nocategs	closed	19/19	66.16	38.23	48.46	4
CRF-DepTree-categs	closed	19/19	60.52	37.21	46.09	5
varIDE	closed	19/19	61.49	36.18	45.56	6
GBD-NER-standard	closed	19/19	36.56	51.05	42.61	7
GBD-NER-resplit	closed	19/19	30.26	55.86	39.26	8
Veyn	closed	19/19	52.16	30.33	38.36	9
mumpitz	closed	7/19	22.23	12.47	15.98	10
Polirem-rich	closed	3/19	10.9	2.87	4.54	11
Polirem-basic	closed	3/19	10.78	0.65	1.23	12
MWETreeC	closed	19/19	0	0	0	n/a
SHOMA	open	19/19	73.37	50.65	59.93	1
Deep-BGT	open	10/19	35.04	25.09	29.24	2
Milos	open	4/19	10.37	6.89	8.28	3
mumpitz-preinit	open	1/19	3	1.61	2.1	4

↑ Back to top

Single-token VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
TRAPACC	closed	5/5	35.13	30.8	32.82	1
TRAPACC_S	closed	5/5	34.64	30.49	32.43	2
TRAVERSAL	closed	5/5	30.7	22.49	25.96	3
CRF-DepTree-categs	closed	5/5	28.49	21.81	24.71	4
Veyn	closed	5/5	22.91	25.76	24.25	5
CRF-Seq-nocategs	closed	5/5	24.95	22.69	23.77	6
varIDE	closed	5/5	36.47	6.43	10.93	7
mumpitz	closed	1/5	4.87	12.62	7.03	8
MWETreeC	closed	5/5	0.79	61.8	1.56	9
Polirem-rich	closed	0/5	0	0	0	n/a
Polirem-basic	closed	0/5	0	0	0	n/a
GBD-NER-standard	closed	5/5	0	0	0	n/a
GBD-NER-resplit	closed	5/5	0	0	0	n/a
SHOMA	open	5/5	27.77	28.9	28.32	1
Deep-BGT	open	3/5	27.61	24.33	25.87	2
Milos	open	2/5	12.23	15.84	13.8	3
mumpitz-preinit	open	1/5	6.43	9.8	7.77	4

↑ Back to top

Seen-in-train VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
TRAVERSAL	closed	19/19	86.54	63	72.92	1
GBD-NER-resplit	closed	19/19	82.76	63.82	72.07	2
TRAPACC	closed	19/19	82.72	61.41	70.49	3
GBD-NER-standard	closed	19/19	82.92	60.74	70.12	4
TRAPACC_S	closed	19/19	82.04	57.06	67.31	5
CRF-Seq-nocategs	closed	19/19	78.27	52.71	63	6
CRF-DepTree-categs	closed	19/19	83.23	50.03	62.49	7
varIDE	closed	19/19	62.8	56.2	59.32	8
Veyn	closed	19/19	76.6	43.7	55.65	9
mumpitz	closed	7/19	30.69	17.81	22.54	10
Polirem-rich	closed	3/19	14.98	4.44	6.85	11
MWETreeC	closed	19/19	13.16	3.99	6.12	12
Polirem-basic	closed	3/19	15.79	1.16	2.16	13
SHOMA	open	19/19	89	66.78	76.31	1
Deep-BGT	open	10/19	46.25	30.36	36.66	2
Milos	open	4/19	16.46	10.4	12.75	3
mumpitz-preinit	open	1/19	4.34	3.07	3.6	4

↑ Back to top

Unseen-in-train VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
GBD-NER-standard	closed	19/19	14.33	31.54	19.71	1
GBD-NER-resplit	closed	19/19	12.74	37.66	19.04	2
TRAVERSAL	closed	19/19	23.94	13.61	17.35	3
CRF-DepTree-categs	closed	19/19	18.71	15.58	17	4
TRAPACC	closed	19/19	19.19	14.52	16.53	5
TRAPACC_S	closed	19/19	24.07	12.47	16.43	6
CRF-Seq-nocategs	closed	19/19	20.49	13.63	16.37	7
Veyn	closed	19/19	11.57	10.58	11.05	8
mumpitz	closed	7/19	5.5	5.92	5.7	9
varIDE	closed	19/19	14.61	3.31	5.4	10
Polirem-rich	closed	3/19	1.76	0.36	0.6	11
MWETreeC	closed	19/19	0.02	1.99	0.04	12
Polirem-basic	closed	3/19	0	0	0	n/a
SHOMA	open	19/19	31.73	25.8	28.46	1
Deep-BGT	open	10/19	12.99	13	12.99	2
Milos	open	4/19	5.56	5.89	5.72	3
mumpitz-preinit	open	1/19	0.75	0.72	0.73	4

↑ Back to top

Variant-of-train VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
GBD-NER-resplit	closed	19/19	76.6	56.48	65.02	1
TRAVERSAL	closed	19/19	83.22	50.82	63.1	2
GBD-NER-standard	closed	19/19	76.63	52.16	62.07	3
TRAPACC	closed	19/19	74.73	47.22	57.87	4
TRAPACC_S	closed	19/19	76.22	43.11	55.07	5
varIDE	closed	19/19	52.7	53.97	53.33	6
CRF-DepTree-categs	closed	19/19	78.29	39.01	52.07	7
CRF-Seq-nocategs	closed	19/19	73.17	35.48	47.79	8
Veyn	closed	19/19	70.65	34.62	46.47	9
mumpitz	closed	7/19	29.96	13.77	18.87	10
Polirem-rich	closed	3/19	14.51	3.21	5.26	11
MWETreeC	closed	19/19	7.89	2.16	3.39	12
Polirem-basic	closed	3/19	10.53	0.48	0.92	13
SHOMA	open	19/19	85.95	50.03	63.25	1
Deep-BGT	open	10/19	45.04	22.42	29.94	2
Milos	open	4/19	15.96	9.97	12.27	3
mumpitz-preinit	open	1/19	4.44	2.67	3.33	4

↑ Back to top

Identical-to-train VMWEs

System	Track	#Langs	MWE-based
			P	R	F1	Rank
TRAPACC	closed	19/19	90.44	77.94	83.73	1
TRAVERSAL	closed	19/19	89.15	75.71	81.88	2
TRAPACC_S	closed	19/19	85.56	73.04	78.81	3
GBD-NER-resplit	closed	19/19	87.27	71.18	78.41	4
GBD-NER-standard	closed	19/19	87.39	69.44	77.39	5
CRF-Seq-nocategs	closed	19/19	80.32	70.54	75.11	6
CRF-DepTree-categs	closed	19/19	85.85	60.25	70.81	7
varIDE	closed	19/19	82.23	57.52	67.69	8
Veyn	closed	19/19	81.15	53.37	64.39	9
mumpitz	closed	7/19	31.57	22.25	26.1	10
Polirem-rich	closed	3/19	15.31	5.99	8.61	11
MWETreeC	closed	19/19	13.16	4.69	6.92	12
Polirem-basic	closed	3/19	15.79	2.03	3.6	13
SHOMA	open	19/19	90.26	85.15	87.63	1
Deep-BGT	open	10/19	46.45	36.71	41.01	2
Milos	open	4/19	17.2	11	13.42	3
mumpitz-preinit	open	1/19	4.25	3.66	3.93	4

↑ Back to top

Language-specific system rankings

BG

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
varIDE	closed	63.58	61.49	62.52	1	66.82	61.16	63.86	1
TRAPACC	closed	70.61	53.43	60.83	2	72.94	54.45	62.35	2
CRF-Seq-nocategs	closed	76.3	48.06	58.97	3	82.67	49.51	61.93	5
TRAVERSAL	closed	75.59	47.61	58.42	4	82.36	49.79	62.06	3
GBD-NER-standard	closed	50.49	69.1	58.35	5	52.94	70.48	60.47	6
mumpitz	closed	75.12	46.42	57.38	6	86.99	48.16	62	4
CRF-DepTree-categs	closed	73.92	41.04	52.78	7	77.9	41.31	53.99	7
TRAPACC_S	closed	80.8	38.96	52.57	8	84.45	39.12	53.47	8
GBD-NER-resplit	closed	39.01	71.79	50.55	9	41.23	73.87	52.92	9
Veyn	closed	0	0	0	n/a	0	0	0	n/a
MWETreeC	closed	0	0	0	n/a	5.05	3.04	3.79	10
Deep-BGT	open	85.96	52.99	65.56	1	91	52.82	66.85	1
SHOMA	open	75.6	55.97	64.32	2	79.98	56.43	66.17	2

↑ Back to top

DE

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	53.26	39.36	45.27	1	61.13	42.26	49.97	3
TRAPACC	closed	53.28	37.55	44.05	2	62.4	39.49	48.37	4
TRAVERSAL	closed	62.93	32.73	43.06	3	76.17	38.36	51.02	2
CRF-DepTree-categs	closed	52.71	29.32	37.68	4	61.87	33.95	43.84	6
mumpitz	closed	32.15	38.35	34.98	5	55.91	48	51.66	1
CRF-Seq-nocategs	closed	40.21	23.09	29.34	6	69.68	33.95	45.66	5
GBD-NER-resplit	closed	27.16	29.72	28.38	7	34.33	41.33	37.51	9
GBD-NER-standard	closed	26.43	27.91	27.15	8	35.97	40.62	38.15	8
Veyn	closed	32.65	22.29	26.49	9	57.52	31.38	40.61	7
varIDE	closed	82.35	8.43	15.3	10	82.35	5.74	10.74	10
MWETreeC	closed	0.76	2.81	1.2	11	4.07	7.69	5.32	11
Deep-BGT	open	60.94	36.35	45.53	1	77.92	37.64	50.76	3
Milos	open	63.74	34.94	45.14	2	67.71	24.51	35.99	4
SHOMA	open	54.15	37.95	44.63	3	69.7	40.82	51.49	2
mumpitz-preinit	open	43.37	36.14	39.43	4	70.5	44.62	54.65	1

↑ Back to top

EL

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	61.65	41.72	49.76	1	70.25	42.68	53.1	3
TRAVERSAL	closed	65.7	36.33	46.79	2	82.16	42.01	55.59	1
TRAPACC	closed	64.31	36.33	46.43	3	72.86	37.07	49.14	5
CRF-DepTree-categs	closed	50.8	38.12	43.56	4	61.44	41.34	49.42	4
CRF-Seq-nocategs	closed	54.41	35.73	43.13	5	74.66	41.92	53.7	2
GBD-NER-resplit	closed	29.55	67.07	41.03	6	36.12	74.56	48.66	7
GBD-NER-standard	closed	41.68	39.52	40.57	7	51.95	44.52	47.95	8
mumpitz	closed	45	30.54	36.39	8	73.21	36.82	49	6
varIDE	closed	86.51	21.76	34.77	9	87.73	20.33	33.02	10
Veyn	closed	39.01	21.96	28.1	10	70.02	28.54	40.55	9
MWETreeC	closed	0	0	0	n/a	41.13	24.44	30.66	11
SHOMA	open	62.79	53.89	58	1	78.6	58.08	66.79	1
Milos	open	42.76	37.72	40.08	2	64.16	47.03	54.27	2

↑ Back to top

EN

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC	closed	38.4	28.74	32.88	1	42.23	28.98	34.37	1
TRAVERSAL	closed	55.5	21.16	30.64	2	58.31	20.33	30.15	3
TRAPACC_S	closed	49.77	21.76	30.28	3	53.5	21.07	30.23	2
CRF-Seq-nocategs	closed	49.76	20.36	28.9	4	55.95	20.33	29.82	4
CRF-DepTree-categs	closed	40.87	18.76	25.72	5	47.2	18.58	26.67	5
varIDE	closed	59.38	15.17	24.17	6	61.09	14.44	23.36	6
GBD-NER-standard	closed	13.69	32.53	19.27	7	14.94	32.84	20.54	7
GBD-NER-resplit	closed	9.55	42.32	15.59	8	11	45.17	17.69	8
Veyn	closed	27.78	2.99	5.41	9	40.66	3.4	6.28	10
MWETreeC	closed	0.61	0.6	0.6	10	22.4	10.12	13.94	9
Milos	open	33.81	32.73	33.27	1	37.32	31.83	34.36	1
SHOMA	open	45.67	11.58	18.47	2	56.09	11.87	19.59	2

↑ Back to top

ES

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	29.54	40	33.98	1	35.96	44.43	39.75	2
CRF-Seq-nocategs	closed	30.87	36	33.24	2	37.47	41.74	39.49	3
TRAPACC	closed	25.69	41.2	31.64	3	32.04	46.78	38.03	4
TRAVERSAL	closed	28.84	33.4	30.95	4	39.91	40.26	40.09	1
varIDE	closed	17.02	49.2	25.3	5	20.15	53.74	29.31	7
CRF-DepTree-categs	closed	15.79	28.2	20.24	6	26.59	35.22	30.3	5
GBD-NER-standard	closed	11.83	44.6	18.7	7	15.64	53.48	24.2	9
Veyn	closed	17.28	16.8	17.04	8	31.68	23.83	27.2	8
GBD-NER-resplit	closed	9.1	42.8	15.01	9	12.4	53.48	20.13	11
mumpitz	closed	9.66	13	11.08	10	31.83	28.87	30.28	6
MWETreeC	closed	0	0	0	n/a	31.03	17.57	22.43	10
SHOMA	open	31.65	48.8	38.39	1	38.33	53.57	44.69	1
Deep-BGT	open	24.5	34.2	28.55	2	33.13	38.61	35.66	2

↑ Back to top

EU

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	84.69	68.6	75.8	1	86.07	69.38	76.83	1
TRAPACC	closed	85.56	64	73.23	2	86.97	65.01	74.4	3
CRF-Seq-nocategs	closed	77.67	62.6	69.32	3	86.42	65.81	74.72	2
TRAVERSAL	closed	78.28	58.4	66.9	4	83.42	65.01	73.07	4
CRF-DepTree-categs	closed	69.29	58.2	63.26	5	77.04	61.03	68.11	5
Veyn	closed	60.15	64	62.02	6	66.6	68.19	67.39	6
varIDE	closed	39.87	76	52.31	7	41.61	78.83	54.46	7
GBD-NER-standard	closed	45.05	58.2	50.79	8	45.67	58.65	51.35	8
GBD-NER-resplit	closed	29.18	69.8	41.16	9	29.97	71.27	42.2	9
MWETreeC	closed	0	0	0	n/a	3.07	4.67	3.71	10
SHOMA	open	81.8	72.8	77.04	1	86.27	74.95	80.21	1

↑ Back to top

FA

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GBD-NER-resplit	closed	78.23	77.45	77.83	1	84.13	78.62	81.28	1
GBD-NER-standard	closed	78.11	74.05	76.02	2	84.54	75.65	79.85	2
TRAPACC	closed	86.98	66.67	75.48	3	92.07	67.83	78.12	4
TRAPACC_S	closed	87.1	64.67	74.23	4	93.12	65.68	77.03	5
CRF-Seq-nocategs	closed	81.22	63.87	71.51	5	93.05	67.39	78.17	3
CRF-DepTree-categs	closed	77.69	57.68	66.21	6	87.98	59.84	71.23	7
TRAVERSAL	closed	73.8	58.48	65.26	7	90.19	65.23	75.7	6
Veyn	closed	80.44	50.9	62.35	8	92.47	54.09	68.25	8
varIDE	closed	91.98	29.74	44.95	9	96.6	28.12	43.56	9
MWETreeC	closed	0	0	0	n/a	55.49	35.85	43.56	9
SHOMA	open	86.12	71.86	78.35	1	93.87	74.3	82.95	1

↑ Back to top

FR

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	77.19	44.18	56.19	1	84.72	48.76	61.9	1
varIDE	closed	55.24	46.59	50.54	2	63.4	49.7	55.72	2
TRAPACC	closed	61.09	38.15	46.97	3	70.55	42.36	52.93	4
CRF-DepTree-categs	closed	61.17	37.95	46.84	4	68.15	40.56	50.86	6
CRF-Seq-nocategs	closed	63.6	36.14	46.09	5	82.66	41.93	55.64	3
TRAPACC_S	closed	72.1	33.73	45.96	6	82.26	36.04	50.12	7
mumpitz	closed	56.8	33.53	42.17	7	81.25	38.86	52.57	5
GBD-NER-standard	closed	36.93	34.34	35.59	8	46.02	38.51	41.93	9
Veyn	closed	39.95	31.12	34.99	9	62.26	39.45	48.3	8
GBD-NER-resplit	closed	22.14	44.38	29.55	10	27.62	51.15	35.87	10
Polirem-rich	closed	68.82	12.85	21.66	11	82.59	17.42	28.77	12
Polirem-basic	closed	78.05	6.43	11.87	12	83.51	6.92	12.78	13
MWETreeC	closed	0	0	0	n/a	34.68	26.47	30.02	11
SHOMA	open	71.86	52.81	60.88	1	79.04	56.02	65.57	2
Deep-BGT	open	57.81	49.8	53.51	2	78.88	56.45	65.8	1
Milos	open	34	44.18	38.43	3	49.29	53.46	51.29	3

↑ Back to top

HE

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	50.33	15.14	23.28	1	74.64	18.1	29.13	2
GBD-NER-standard	closed	15.89	39.64	22.69	2	19.83	44.94	27.52	3
TRAPACC	closed	36.41	14.14	20.37	3	47.59	16.28	24.26	4
Veyn	closed	23.39	17.33	19.91	4	39.29	23.81	29.65	1
varIDE	closed	69.23	10.76	18.62	5	70.3	10.04	17.58	8
TRAPACC_S	closed	56.82	9.96	16.95	6	69.02	11	18.97	6
GBD-NER-resplit	closed	8.55	48.21	14.52	7	10.8	56.1	18.11	7
CRF-DepTree-categs	closed	22.43	9.56	13.41	8	37.82	14.11	20.55	5
MWETreeC	closed	0	0	0	n/a	36.62	9.96	15.66	9
CRF-Seq-nocategs	closed	0	0	0	n/a	26.98	1.47	2.79	10
SHOMA	open	61.37	28.49	38.91	1	74.43	31.26	44.02	1

↑ Back to top

HI

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
CRF-DepTree-categs	closed	77.95	68.6	72.98	1	81.12	66.94	73.35	1
CRF-Seq-nocategs	closed	76.79	64.2	69.93	2	81.8	64.31	72.01	2
TRAPACC	closed	72.39	66.6	69.38	3	77.17	66.12	71.22	3
TRAPACC_S	closed	82.91	58.2	68.39	4	86.53	56.43	68.31	5
TRAVERSAL	closed	66.3	60.6	63.32	5	73.15	67.12	70	4
GBD-NER-standard	closed	52.97	74.8	62.02	6	55.95	73.28	63.45	6
GBD-NER-resplit	closed	52.97	74.8	62.02	6	55.95	73.28	63.45	6
varIDE	closed	85.2	42.6	56.8	7	86.77	41.58	56.22	7
Veyn	closed	0	0	0	n/a	0	0	0	n/a
MWETreeC	closed	0	0	0	n/a	44.82	29.8	35.8	8
SHOMA	open	76.84	69	72.71	1	84.3	68.57	75.62	1

↑ Back to top

HR

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	68.04	46.59	55.3	1	78.14	50.73	61.52	1
TRAPACC_S	closed	60.42	34.94	44.27	2	68.42	36.9	47.95	4
TRAPACC	closed	51.37	37.55	43.39	3	61.42	42.12	49.97	2
GBD-NER-resplit	closed	36.69	44.58	40.25	4	45.29	51.1	48.02	3
GBD-NER-standard	closed	36.53	43.57	39.74	5	44.02	48.53	46.17	6
CRF-DepTree-categs	closed	40.13	24.1	30.11	6	65.93	32.97	43.96	7
CRF-Seq-nocategs	closed	36.98	23.09	28.43	7	75.05	34.71	47.46	5
Veyn	closed	34.07	18.67	24.12	8	71.67	27.11	39.34	8
varIDE	closed	79.07	6.83	12.57	9	83.7	7.05	13.01	10
MWETreeC	closed	0	0	0	n/a	55.52	15.66	24.43	9
SHOMA	open	59.58	39.96	47.84	1	79.52	45.88	58.19	1

↑ Back to top

HU

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC	closed	92.44	88.27	90.31	1	90.72	85.44	88	1
TRAPACC_S	closed	94.87	85.82	90.12	2	93.56	81.75	87.25	2
CRF-Seq-nocategs	closed	93.94	77.96	85.21	3	95.16	76.31	84.7	3
TRAVERSAL	closed	88.01	74.74	80.84	4	89.91	79.61	84.45	4
CRF-DepTree-categs	closed	88	73.71	80.22	5	90.2	73.3	80.88	6
Veyn	closed	83.55	75.26	79.19	6	93.81	75.05	83.39	5
GBD-NER-resplit	closed	67.52	20.36	31.29	7	69.66	31.65	43.52	7
GBD-NER-standard	closed	67.12	19.2	29.86	8	68.92	29.71	41.52	8
varIDE	closed	100	10.31	18.69	9	100	8.74	16.07	9
MWETreeC	closed	2.58	67.14	4.96	10	3.81	74.85	7.25	10
SHOMA	open	90.08	81.96	85.83	1	93.49	80.87	86.73	1
Deep-BGT	open	78	71.26	74.48	2	80.71	73.11	76.72	2

↑ Back to top

IT

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	63.09	40.32	49.2	1	74.42	42.11	53.78	1
TRAPACC	closed	52.43	30.44	38.52	2	61.54	30.34	40.64	4
CRF-Seq-nocategs	closed	55.14	27.02	36.27	3	78.49	33.05	46.51	2
TRAPACC_S	closed	55.66	23.79	33.33	4	65.42	22.99	34.02	8
CRF-DepTree-categs	closed	44.76	25.81	32.74	5	58.78	29.8	39.55	5
varIDE	closed	31.07	34.07	32.5	6	39.22	35.06	37.02	6
Veyn	closed	34.01	30.44	32.13	7	58.41	38.16	46.16	3
Polirem-rich	closed	72.36	17.94	28.76	8	86.54	21.9	34.96	7
GBD-NER-standard	closed	15.45	29.84	20.36	9	22.68	35.45	27.67	9
GBD-NER-resplit	closed	10.69	28.83	15.59	10	16.63	38.31	23.19	10
Polirem-basic	closed	83.33	4.03	7.69	11	81.82	3.48	6.68	11
MWETreeC	closed	0	0	0	n/a	1.45	6.58	2.38	12
SHOMA	open	50.37	41.33	45.4	1	67.49	46.59	55.13	1
Deep-BGT	open	45.52	25.6	32.77	2	70	27.63	39.62	2

↑ Back to top

LT

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC_S	closed	45.76	24.8	32.17	1	53.52	25.02	34.1	2
TRAPACC	closed	33.98	28.2	30.82	2	42.48	28.94	34.43	1
TRAVERSAL	closed	29.61	13.8	18.83	3	55.56	16.92	25.94	3
Veyn	closed	18.89	6.8	10	4	48.85	11.31	18.37	4
GBD-NER-standard	closed	3.76	39.6	6.87	5	4.76	45.95	8.62	5
GBD-NER-resplit	closed	3.76	39.6	6.87	5	4.76	45.95	8.62	5
varIDE	closed	55.56	1	1.96	6	55.56	1.34	2.61	7
MWETreeC	closed	0	0	0	n/a	0.42	44.52	0.84	8
CRF-Seq-nocategs	closed	0	0	0	n/a	35.16	2.85	5.27	6
CRF-DepTree-categs	closed	0	0	0	n/a	35.16	2.85	5.27	6
SHOMA	open	35.74	16.8	22.86	1	56.76	18.7	28.13	1

↑ Back to top

PL

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	77.02	59.22	66.96	1	81.85	59.03	68.59	1
TRAPACC	closed	71.62	52.43	60.54	2	78.01	53.79	63.68	3
GBD-NER-standard	closed	59.36	60.97	60.15	3	63.93	61.73	62.81	5
TRAPACC_S	closed	78.55	48.35	59.86	4	83.1	48.83	61.51	6
Veyn	closed	62.69	55.15	58.68	5	73.52	58.12	64.92	2
CRF-Seq-nocategs	closed	70.88	46.8	56.37	6	83.28	50.81	63.12	4
GBD-NER-resplit	closed	42.53	67.96	52.32	7	45.77	69.4	55.16	7
mumpitz	closed	62.07	38.45	47.48	8	80.92	41.34	54.72	8
CRF-DepTree-categs	closed	26.66	44.47	33.33	9	37.99	57.94	45.89	9
varIDE	closed	86.11	6.02	11.25	10	88.89	5.78	10.85	11
MWETreeC	closed	0	0	0	n/a	43.78	30.51	35.96	10
SHOMA	open	73.05	56.31	63.6	1	79.32	57.13	66.42	2
Deep-BGT	open	70.87	56.7	63	2	80.23	57.85	67.23	1

↑ Back to top

PT

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	76.8	52.08	62.07	1	85.14	54.69	66.6	1
varIDE	closed	72.8	52.26	60.84	2	76.1	52.85	62.38	2
Veyn	closed	62.55	52.26	56.95	3	71.26	52.69	60.58	3
TRAPACC	closed	55.01	50.63	52.73	4	61.55	53.01	56.96	5
TRAPACC_S	closed	65.75	43.4	52.29	5	72.63	43.62	54.51	7
CRF-DepTree-categs	closed	55.1	41.05	47.05	6	63.23	45.23	52.73	8
mumpitz	closed	44.77	47.2	45.95	7	63.96	52.37	57.58	4
CRF-Seq-nocategs	closed	48.51	35.26	40.84	8	78.01	44.67	56.81	6
Polirem-rich	closed	65.83	23.69	34.84	9	79.23	34.56	48.13	9
GBD-NER-standard	closed	21.08	49.37	29.55	10	23.79	51.24	32.49	10
GBD-NER-resplit	closed	14.83	63.29	24.03	11	16.4	66.96	26.34	11
Polirem-basic	closed	43.48	1.81	3.47	12	50	2.57	4.88	12
MWETreeC	closed	0	0	0	n/a	0.94	44.35	1.85	13
SHOMA	open	71.12	65.46	68.17	1	78.81	68.89	73.51	1
Deep-BGT	open	72.44	46.11	56.35	2	79.4	44.83	57.3	2

↑ Back to top

RO

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAPACC	closed	84.5	86.08	85.28	1	85.19	86.2	85.69	1
TRAPACC_S	closed	89.17	78.27	83.36	2	90.05	77.76	83.45	5
TRAVERSAL	closed	86.06	79.63	82.72	3	88.84	82.26	85.42	2
Veyn	closed	80.03	83.7	81.83	4	82.79	84.23	83.5	4
CRF-Seq-nocategs	closed	83.45	79.63	81.49	5	87.78	80.99	84.25	3
varIDE	closed	58.91	89.81	71.15	6	60.24	89.98	72.17	6
CRF-DepTree-categs	closed	74.69	62.14	67.84	7	81.11	63.64	71.32	7
GBD-NER-standard	closed	39.28	85.57	53.85	8	41.22	86.99	55.93	8
GBD-NER-resplit	closed	28.12	87.95	42.62	9	28.85	89.04	43.57	9
MWETreeC	closed	0	0	0	n/a	57.6	31.39	40.63	10
SHOMA	open	87.78	86.59	87.18	1	90.21	87.22	88.69	1
Deep-BGT	open	79.8	69.1	74.07	2	92.11	73.66	81.86	2

↑ Back to top

SL

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
TRAVERSAL	closed	79.41	54	64.29	1	83.61	54.54	66.01	1
varIDE	closed	30.31	70.2	42.34	2	32.64	71.61	44.84	6
GBD-NER-standard	closed	34.56	48.8	40.46	3	40.34	51.39	45.2	5
Veyn	closed	57.84	31	40.36	4	72.87	33.06	45.49	4
CRF-Seq-nocategs	closed	55.67	31.4	40.15	5	82.14	38.01	51.97	2
CRF-DepTree-categs	closed	54.51	30.2	38.87	6	84.21	37.38	51.77	3
TRAPACC_S	closed	33.33	29.6	31.36	7	47.24	32.26	38.33	7
GBD-NER-resplit	closed	21.16	59.2	31.17	8	24.49	64.24	35.46	8
TRAPACC	closed	20.21	26.8	23.04	9	36.99	32.7	34.72	9
MWETreeC	closed	0	0.2	0	n/a	1.23	44.83	2.4	10
SHOMA	open	58.56	47.2	52.27	1	74.81	52.29	61.55	1
Deep-BGT	open	58.9	38.4	46.49	2	72.19	40.34	51.76	2

↑ Back to top

TR

System	Track	MWE-based				Token-based
		P	R	F1	Rank	P	R	F1	Rank
GBD-NER-standard	closed	44.47	46.05	45.24	1	47.9	48.04	47.97	4
Veyn	closed	58.07	36.96	45.17	2	70.77	42.39	53.02	1
CRF-Seq-nocategs	closed	71.05	32.02	44.14	3	89.04	36.56	51.83	2
CRF-DepTree-categs	closed	67.71	29.84	41.43	4	84.62	33.68	48.19	3
TRAVERSAL	closed	81.48	26.09	39.52	5	88.38	27.66	42.13	5
GBD-NER-resplit	closed	44.11	25.89	32.63	6	47.4	27.08	34.47	6
varIDE	closed	4.19	65.22	7.87	7	5.23	78.95	9.81	7
TRAPACC	closed	1.64	1.58	1.61	8	5.26	4.21	4.68	9
TRAPACC_S	closed	1.12	0.59	0.78	9	6	2.01	3.01	10
MWETreeC	closed	0	0	0	n/a	3.35	8.52	4.81	8
SHOMA	open	81.4	45.85	58.66	1	87.22	47.66	61.63	1

↑ Back to top

Results per VMWE category (not ranked)

BG

System	Track	IAV						IRV						LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	0	0	0	0	0	0	85.54	81.5	83.47	85.54	81.5	83.47	40	3.85	7.02	57.14	3.51	6.61	70.91	14.23	23.71	77.27	14.94	25.04	38.1	9.76	15.53	41.82	11.22	17.69
TRAVERSAL	closed	0	0	0	0	0	0	81.3	78.74	80	82.32	82.48	82.4	83.33	9.62	17.24	86.67	11.4	20.16	72.88	31.39	43.88	85.14	33.22	47.79	46.94	28.05	35.11	59.62	30.24	40.13
Veyn	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	0	0	0	0	0	0	59.07	84.65	69.58	59.75	85.63	70.39	72.73	15.38	25.4	77.27	14.91	25	75.61	56.57	64.72	76.34	55.01	63.94	42.65	35.37	38.67	47.06	31.22	37.54
CRF-DepTree-categs	closed	0	0	0	0	0	0	86.21	68.9	76.59	86.63	68.9	76.75	100	5.77	10.91	100	5.26	10	66.36	26.64	38.02	69.41	26.71	38.58	32.73	21.95	26.28	38.4	23.41	29.09
CRF-Seq-nocategs	closed	0	0	0	0.94	40	1.84	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	0	0	0	0	0	0	89.3	85.43	87.32	89.51	85.63	87.53	43.48	19.23	26.67	45.65	18.42	26.25	86.73	35.77	50.65	91.07	35.85	51.45	41.18	17.07	24.14	65.15	20.98	31.73
GBD-NER-resplit	closed	0	0	0	0	0	0	65.49	94.88	77.49	65.4	94.88	77.43	26.32	19.23	22.22	30.77	21.05	25	27.89	57.3	37.51	29.04	58.7	38.86	14.42	37.8	20.88	17.93	41.46	25.04
GBD-NER-standard	closed	0	0	0	0	0	0	65.31	94.88	77.37	65.58	95.28	77.69	0	0	0	0	0	0	44.38	56.2	49.6	44.82	55.54	49.61	20	39.02	26.45	23.89	41.95	30.44
MWETreeC	closed	0	0	0	25	5	8.33	0	0	0	1.94	1.97	1.96	0	0	0	7.14	0.88	1.56	0	0	0	5.12	1.93	2.81	0	0	0	2.17	0.98	1.35
SHOMA	open	0	0	0	0	0	0	84.27	88.58	86.37	85.85	89.57	87.67	71.43	19.23	30.3	80.77	18.42	30	77.1	36.86	49.88	79.25	36.91	50.36	35.44	34.15	34.78	44.64	36.59	40.21
TRAPACC	closed	0	0	0	0	0	0	81.68	84.25	82.95	81.68	84.25	82.95	30.77	7.69	12.31	32	7.02	11.51	64.43	35.04	45.39	64.05	34.45	44.8	28.57	26.83	27.67	30.69	28.29	29.44

↑ Back to top

DE

System	Track	IRV						LVC.cause						LVC.full						VID						VPC.full						VPC.semi
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC	closed	16.67	12.5	14.29	45.31	30.21	36.25	0	0	0	0	0	0	13.64	7.14	9.37	13.64	6.19	8.51	34.85	25.14	29.21	43.11	26.41	32.75	72.14	48.1	57.71	71.2	46.74	56.43	18.18	17.39	17.78	14.81	12.9	13.79
TRAPACC_S	closed	28.57	20	23.53	53.97	35.42	42.77	0	0	0	0	0	0	16.67	7.14	10	20.51	8.25	11.76	31.37	26.23	28.57	40.8	30.74	35.06	73.89	55.24	63.22	74.88	54.3	62.95	60	13.04	21.43	66.67	12.9	21.62
TRAVERSAL	closed	40	30	34.29	59.72	44.79	51.19	0	0	0	0	0	0	40	9.52	15.38	61.9	13.4	22.03	41.57	20.22	27.21	64.85	28.35	39.46	81.97	47.62	60.24	84.95	54.3	66.25	57.14	17.39	26.67	66.67	19.35	30
Veyn	closed	33.33	10	15.38	75	15.62	25.86	0	0	0	0	0	0	0	0	0	0	0	0	19.71	14.75	16.87	36.3	22.08	27.46	37.43	33.33	35.26	46.67	36.08	40.7	0	0	0	0	0	0
mumpitz-preinit	open	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	0	0	0	0	0	0	100	50	66.67	100	40	57.14	100	9.52	17.39	100	8.25	15.24	57.14	2.19	4.21	58.33	1.52	2.95	80.56	13.81	23.58	81.4	12.03	20.96	66.67	8.7	15.38	66.67	6.45	11.76
CRF-DepTree-categs	closed	34.38	27.5	30.56	54.69	36.46	43.75	0	0	0	0	0	0	10.53	4.76	6.56	15.91	7.22	9.93	34.31	19.13	24.56	44.73	22.94	30.33	68.6	39.52	50.15	74.46	47.08	57.68	75	13.04	22.22	83.33	16.13	27.03
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	35.14	32.5	33.77	57.14	41.67	48.19	0	0	0	0	0	0	16.67	2.38	4.17	55.56	5.15	9.43	37.5	18.03	24.35	69.28	24.89	36.62	72.12	56.67	63.47	77.33	59.79	67.44	0	0	0	100	3.23	6.25
GBD-NER-resplit	closed	22.62	47.5	30.65	29.78	55.21	38.69	0	0	0	0	0	0	4.29	7.14	5.36	5.71	8.25	6.75	17.31	24.59	20.32	22.73	29.22	25.57	47.33	29.52	36.36	48.09	43.3	45.57	0	0	0	0	0	0
GBD-NER-standard	closed	25	35	29.17	36.67	45.83	40.74	0	0	0	0	0	0	0	0	0	0	0	0	13.74	23.5	17.34	21.74	31.39	25.69	40.13	30	34.33	40.13	43.3	41.65	0	0	0	0	0	0
MWETreeC	closed	0	0	0	9.8	5.21	6.8	0	0	0	0	0	0	0	0	0	4.76	1.03	1.69	0	0	0	2.7	1.08	1.55	2.59	2.86	2.71	10.34	8.25	9.18	0	0	0	0	0	0
Milos	open	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	54.95	71.43	62.11	56.09	68.04	61.49	0	0	0	0	0	0
SHOMA	open	40	35	37.33	59.15	43.75	50.3	0	0	0	0	0	0	0	0	0	25	6.19	9.92	43.9	29.51	35.29	62.35	34.42	44.35	63.31	50.95	56.46	70.14	50.86	58.96	50	17.39	25.81	60	19.35	29.27

↑ Back to top

EL

System	Track	LVC.cause						LVC.full						MVC						VID						VPC.full
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	0	0	0	0	0	0	53.24	48.05	50.51	55.89	49.61	52.56	0	0	0	0	0	0	49.15	17.16	25.44	60.39	18.6	28.44	100	9.09	16.67	100	8.7	16
TRAVERSAL	closed	0	0	0	0	0	0	63.04	37.66	47.15	75.27	43.68	55.28	0	0	0	0	0	0	68.97	35.5	46.87	82.4	38.4	52.39	0	0	0	28.57	8.7	13.33
Veyn	closed	0	0	0	0	0	0	28.52	25.65	27.01	47.58	35.26	40.5	0	0	0	0	0	0	20	0.59	1.15	25	0.6	1.17	0	0	0	0	0	0
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	33.33	9.09	14.29	28.57	8	12.5	92	22.4	36.03	92.5	23.09	36.95	0	0	0	0	0	0	76.92	17.75	28.85	76.09	14	23.65	44.44	36.36	40	61.11	47.83	53.66
CRF-DepTree-categs	closed	0	0	0	0	0	0	51.16	42.86	46.64	58.88	46.02	51.66	0	0	0	0	0	0	35.77	26.04	30.14	50.48	31.6	38.87	0	0	0	100	4.35	8.33
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	0	0	0	0	0	0	21.69	75.97	33.74	22.99	81.44	35.85	0	0	0	0	0	0	27.27	8.88	13.39	37.89	14.4	20.87	0	0	0	0	0	0
GBD-NER-standard	closed	0	0	0	0	0	0	33.01	44.16	37.78	35.8	46.02	40.27	0	0	0	0	0	0	19.05	7.1	10.34	38	15.2	21.71	0	0	0	0	0	0
MWETreeC	closed	0	0	0	16.67	12	13.95	0	0	0	38.19	29.02	32.98	0	0	0	33.33	16.67	22.22	0	0	0	35.18	14	20.03	0	0	0	33.33	4.35	7.69
Milos	open	0	0	0	0	0	0	33.78	40.58	36.87	46.23	49.77	47.93	0	0	0	0	0	0	14.89	4.14	6.48	45.45	11	17.71	0	0	0	0	0	0
SHOMA	open	0	0	0	0	0	0	59.37	60.71	60.03	71.33	65.99	68.56	100	50	66.67	100	50	66.67	66.02	40.24	50	80	43.2	56.1	14.29	9.09	11.11	40	17.39	24.24
TRAPACC	closed	7.14	9.09	8	6.45	8	7.14	58.13	38.31	46.18	60.14	39.31	47.55	100	50	66.67	100	50	66.67	40.62	15.38	22.32	54.9	16.8	25.73	100	9.09	16.67	100	8.7	16

↑ Back to top

EN

System	Track	IAV						LVC.cause						LVC.full						MVC						VID						VPC.full						VPC.semi
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	75	20.45	32.14	75	18.37	29.51	0	0	0	0	0	0	51.28	12.05	19.51	52.56	12.06	19.62	0	0	0	0	0	0	13.33	2.53	4.26	12.9	1.78	3.12	45.71	43.84	44.76	45.71	43.84	44.76	50	3.85	7.14	50	3.85	7.14
TRAVERSAL	closed	53.33	18.18	27.12	53.33	16.33	25	0	0	0	0	0	0	48.65	10.84	17.73	48	10.59	17.35	0	0	0	0	0	0	40	2.53	4.76	41.67	2.22	4.22	50.77	45.21	47.83	53.12	46.58	49.64	66.67	7.69	13.79	66.67	7.69	13.79
Veyn	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	24.07	8.9	13	32.97	10.27	15.67	0	0	0	0	0	0
varIDE	closed	0	0	0	0	0	0	0	0	0	0	0	0	54.84	10.24	17.26	56.45	10.29	17.41	0	0	0	0	0	0	53.85	8.86	15.22	55.56	6.67	11.9	53.85	28.77	37.5	53.85	28.77	37.5	25	3.85	6.67	25	3.85	6.67
CRF-DepTree-categs	closed	0	0	0	30	3.06	5.56	0	0	0	0	0	0	28	12.65	17.43	31.88	12.94	18.41	0	0	0	0	0	0	12.5	2.53	4.21	11.76	1.78	3.09	46.09	40.41	43.07	48.56	40.41	44.11	0	0	0	0	0	0
CRF-Seq-nocategs	closed	2.93	13.64	4.82	3.54	14.29	5.68	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	0	0	0	0	0	0	0	0	0	0	0	0	4.59	19.88	7.46	4.93	20.88	7.98	0	0	0	0	0	0	1.12	3.8	1.73	1.79	4.44	2.55	9.43	78.77	16.84	9.43	78.77	16.84	0	0	0	0	0	0
GBD-NER-standard	closed	0	0	0	0	0	0	0	0	0	0	0	0	4.17	1.2	1.87	4.17	1.18	1.83	0	0	0	0	0	0	0	0	0	0	0	0	9.83	73.97	17.35	9.96	75	17.58	0	0	0	0	0	0
MWETreeC	closed	0	0	0	46.15	6.12	10.81	0	0	0	0	0	0	0	0	0	19.59	5.59	8.7	0	0	0	0	0	0	13.04	3.8	5.88	43.48	4.44	8.06	0	0	0	17.33	13.36	15.09	0	0	0	27.5	21.15	23.91
Milos	open	0	0	0	0	0	0	0	0	0	0	0	0	32.31	12.65	18.18	38.66	13.53	20.04	0	0	0	0	0	0	0	0	0	0	0	0	26.38	75.34	39.08	27.68	76.03	40.59	0	0	0	0	0	0
SHOMA	open	33.33	2.27	4.26	33.33	2.04	3.85	0	0	0	0	0	0	28.57	1.2	2.31	50	1.47	2.86	0	0	0	0	0	0	16.67	2.53	4.4	53.33	3.56	6.67	45.36	30.14	36.21	49.46	31.51	38.49	12.5	3.85	5.88	23.08	5.77	9.23
TRAPACC	closed	50	18.18	26.67	50	16.33	24.62	0	0	0	0	0	0	32.43	14.46	20	34	15	20.82	0	0	0	0	0	0	6.45	2.53	3.64	10.77	3.11	4.83	34.48	54.79	42.33	34.48	54.79	42.33	0	0	0	0	0	0

↑ Back to top

ES

System	Track	IAV						IRV						LVC.cause						LVC.full						MVC						VID						VPC.full
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	26.72	54.69	35.9	29.02	56.08	38.25	33.06	66.12	44.08	33.26	66.53	44.35	0	0	0	5.56	3.17	4.04	30.77	14.12	19.35	32.1	14.77	20.23	22.28	40.57	28.76	25.69	44.58	32.6	35.9	14.74	20.9	41.18	15.56	22.58	0	0	0	0	0	0
TRAVERSAL	closed	17.69	40.62	24.64	29.52	54.05	38.19	35.19	67.77	46.33	35.47	68.6	46.76	100	3.57	6.9	100	3.17	6.15	48.39	17.65	25.86	57.81	21.02	30.83	22.6	31.13	26.19	37.95	46.18	41.67	24.14	7.37	11.29	48.53	12.22	19.53	0	0	0	0	0	0
Veyn	closed	16.81	31.25	21.86	24.4	34.46	28.57	24.79	24.79	24.79	28.12	26.03	27.04	0	0	0	0	0	0	3.74	4.71	4.17	7.65	7.39	7.51	15.83	20.75	17.96	26.34	27.71	27.01	0	0	0	0	0	0	0	0	0	0	0	0
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	11.61	84.38	20.42	11.7	81.08	20.44	15.24	73.55	25.25	15.84	76.45	26.24	50	21.43	30	53.33	25.4	34.41	27.34	41.18	32.86	31.8	47.16	37.99	14.59	25.47	18.56	17.27	28.92	21.62	25.35	18.95	21.69	30.91	18.89	23.45	0	0	0	0	0	0
CRF-DepTree-categs	closed	0.66	3.12	1.09	12.66	32.43	18.22	22.15	56.2	31.78	25.49	58.68	35.54	0	0	0	0	0	0	20	14.12	16.55	29.41	19.89	23.73	21.11	35.85	26.57	27.35	40.96	32.8	14.47	11.58	12.87	23.03	12.96	16.59	0	0	0	0	0	0
CRF-Seq-nocategs	closed	6	54.69	10.82	7.42	64.19	13.3	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	25.77	39.06	31.06	27.51	35.14	30.86	26.63	74.38	39.22	27.04	75.21	39.78	0	0	0	0	0	0	29.73	12.94	18.03	40.98	14.2	21.1	18.75	31.13	23.4	28.95	39.76	33.5	10.2	5.26	6.94	25.68	7.04	11.05	0	0	0	0	0	0
GBD-NER-resplit	closed	2.17	18.75	3.89	3.77	28.38	6.66	21.76	61.16	32.1	22.06	61.98	32.54	0	0	0	0	0	0	2.37	23.53	4.31	2.78	26.7	5.03	16.48	70.75	26.74	18.58	73.49	29.66	6.83	11.58	8.59	7.98	14.44	10.28	0	0	0	0	0	0
GBD-NER-standard	closed	4.56	18.75	7.34	7.05	25.68	11.06	15.56	79.34	26.02	15.96	81.4	26.69	0	0	0	0	0	0	3.46	16.47	5.71	3.95	18.18	6.49	15.04	72.64	24.92	16.74	75.5	27.41	6.82	6.32	6.56	11.5	9.63	10.48	0	0	0	0	0	0
MWETreeC	closed	0	0	0	29.41	20.27	24	0	0	0	34.85	19.01	24.6	0	0	0	0	0	0	0	0	0	20.48	9.66	13.13	0	0	0	19.35	24.1	21.47	0	0	0	41.18	5.19	9.21	0	0	0	0	0	0
SHOMA	open	28.33	53.12	36.96	31.52	54.73	40	34.62	66.94	45.63	35.73	67.77	46.79	16.67	3.57	5.88	20	4.76	7.69	19.3	25.88	22.11	28.02	32.95	30.29	24.6	58.49	34.64	30.32	67.47	41.84	45.83	23.16	30.77	53.33	23.7	32.82	0	0	0	0	0	0
TRAPACC	closed	21.48	50	30.05	23.69	52.03	32.56	32.66	66.94	43.9	32.86	67.36	44.17	0	0	0	2.47	3.17	2.78	33.33	12.94	18.64	32.86	13.07	18.7	20.76	46.23	28.65	23.6	50.6	32.18	15.15	10.53	12.42	25.43	16.3	19.86	0	0	0	0	0	0

↑ Back to top

EU

System	Track	LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAVERSAL	closed	100	5.88	11.11	100	5.88	11.11	78.79	63.41	70.27	83.96	70.77	76.8	71.43	41.1	52.17	75.56	45.03	56.43
Veyn	closed	0	0	0	16.67	2.94	5	62.86	68.54	65.58	67.93	71.99	69.9	29.63	32.88	31.17	36.36	37.09	36.72
varIDE	closed	45.45	58.82	51.28	45.45	58.82	51.28	44.52	78.29	56.76	45.63	80.15	58.15	22.38	64.38	33.22	23.1	64.24	33.98
CRF-DepTree-categs	closed	100	23.53	38.1	100	23.53	38.1	72.14	63.17	67.36	78.86	65.9	71.8	35.48	30.14	32.59	41.59	31.13	35.61
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	0	0	0	0	0	0	25.5	74.39	37.98	26.05	75.88	38.78	0	0	0	0	0	0
GBD-NER-standard	closed	0	0	0	0	0	0	40.09	63.17	49.05	40.48	63.7	49.5	0	0	0	0	0	0
MWETreeC	closed	0	0	0	0	0	0	0	0	0	3.84	1.83	2.48	0	0	0	1.61	0.66	0.94
SHOMA	open	57.14	47.06	51.61	61.54	47.06	53.33	82.71	75.85	79.13	86.84	77.95	82.16	67.27	50.68	57.81	69.37	50.99	58.78
TRAPACC	closed	28.57	11.76	16.67	30.77	11.76	17.02	81.54	64.63	72.11	83.08	65.77	73.42	47.62	27.4	34.78	48.31	28.48	35.83
TRAPACC_S	closed	33.33	17.65	23.08	40	17.65	24.49	81.16	68.29	74.17	82.32	69.18	75.18	50.98	35.62	41.94	50.94	35.76	42.02

↑ Back to top

FA

System	Track	LVC.full
		MWE-based			Token-based
		P	R	F1	P	R	F1
TRAVERSAL	closed	73.8	58.48	65.26	90.19	65.23	75.7
Veyn	closed	80.44	50.9	62.35	92.47	54.09	68.25
varIDE	closed	91.98	29.74	44.95	96.6	28.12	43.56
CRF-DepTree-categs	closed	78.05	57.49	66.21	88.15	59.48	71.03
CRF-Seq-nocategs	closed	0	0	0	0	0	0
GBD-NER-resplit	closed	78.23	77.45	77.83	84.13	78.62	81.28
GBD-NER-standard	closed	78.11	74.05	76.02	84.54	75.65	79.85
MWETreeC	closed	0	0	0	55.73	35.85	43.63
SHOMA	open	86.33	71.86	78.43	93.86	74.21	82.89
TRAPACC	closed	87.17	66.47	75.42	92.01	67.3	77.74
TRAPACC_S	closed	87.1	64.67	74.23	93.12	65.68	77.03

↑ Back to top

FR

System	Track	IRV						LVC.cause						LVC.full						MVC						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
Polirem-rich	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	51.61	22.64	31.48	51.42	21.67	30.49
SHOMA	open	72.63	63.89	67.98	74.6	65.28	69.63	0	0	0	0	0	0	66.67	41.25	50.97	76.17	44.14	55.89	100	25	40	100	25	40	71.43	56.6	63.16	77.88	58.87	67.06
TRAPACC	closed	80.95	47.22	59.65	80.95	47.22	59.65	0	0	0	0	0	0	53.61	32.5	40.47	55.05	32.73	41.05	50	25	33.33	50	25	33.33	61.48	35.38	44.91	70.96	40.44	51.52
TRAPACC_S	closed	82.26	47.22	60	82.26	47.22	60	0	0	0	0	0	0	58.23	28.75	38.49	58.86	27.93	37.88	0	0	0	0	0	0	71.43	28.3	40.54	82.88	31.4	45.54
TRAVERSAL	closed	81.58	57.41	67.39	82.69	59.72	69.35	50	7.14	12.5	50	7.14	12.5	72.22	40.62	52	78.53	45.05	57.25	0	0	0	0	0	0	75.21	42.92	54.65	85.59	48.63	62.02
Veyn	closed	58.41	61.11	59.73	63.18	64.35	63.76	0	0	0	0	0	0	24.39	25	24.69	42.66	36.64	39.42	0	0	0	0	0	0	36.04	18.87	24.77	63.56	25.6	36.5
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	54.05	55.56	54.79	56.31	57.87	57.08	0	0	0	0	0	0	45.96	46.25	46.11	49.54	48.65	49.09	33.33	25	28.57	33.33	25	28.57	68.09	45.28	54.39	75.21	45.56	56.75
CRF-DepTree-categs	closed	75.38	45.37	56.65	80.8	46.76	59.24	33.33	7.14	11.76	50	7.14	12.5	42.06	33.12	37.06	46.07	36.94	41	0	0	0	0	0	0	60.16	36.32	45.29	69.51	38.91	49.89
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	78.7	78.7	78.7	79.53	79.17	79.35	0	0	0	0	0	0	68.46	55.63	61.38	72.91	54.95	62.67	0	0	0	0	0	0	34.03	30.66	32.26	77.15	48.98	59.92
GBD-NER-resplit	closed	34.07	57.41	42.76	33.87	58.33	42.86	3.57	7.14	4.76	3.39	7.14	4.6	15.72	38.12	22.26	17.25	41.14	24.31	0	0	0	0	0	0	20.1	36.79	26	28.01	43.69	34.13
GBD-NER-standard	closed	43.42	61.11	50.77	43.75	61.57	51.15	33.33	7.14	11.76	33.33	7.14	11.76	21.29	20.62	20.95	25.08	23.42	24.22	0	0	0	0	0	0	35.95	25.94	30.14	45.68	27.99	34.71
MWETreeC	closed	0	0	0	45.59	28.7	35.23	0	0	0	16.67	3.57	5.88	0	0	0	33.18	22.22	26.62	0	0	0	0	0	0	0	0	0	26.27	23.89	25.02
Milos	open	42.65	80.56	55.77	43.14	81.48	56.41	0	0	0	0	0	0	27.06	43.13	33.25	31.81	45.95	37.59	0	0	0	0	0	0	14.92	12.74	13.74	39.04	24.91	30.42
Polirem-basic	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	58.54	11.32	18.97	57.73	9.56	16.4

↑ Back to top

HE

System	Track	LVC.cause						LVC.full						VID						VPC.full
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAVERSAL	closed	47.06	16.33	24.24	66.67	17.82	28.12	57.33	20.38	30.07	70.68	21.61	33.1	40.35	12.64	19.25	70.69	17.56	28.13	0	0	0	0	0	0
Veyn	closed	2.78	2.04	2.35	7.84	3.96	5.26	13.81	17.54	15.45	21.19	24.6	22.77	11.76	4.4	6.4	22.92	7.07	10.8	0	0	0	0	0	0
varIDE	closed	53.85	14.29	22.58	55.56	14.85	23.44	68.57	11.37	19.51	70	11.26	19.41	60	9.89	16.98	61.76	8.99	15.7	0	0	0	0	0	0
CRF-DepTree-categs	closed	26.32	10.2	14.71	51.85	13.86	21.88	39.02	15.17	21.84	55.13	19.77	29.1	10	4.95	6.62	20.18	9.64	13.04	0	0	0	0	0	0
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	26.09	36.73	30.51	28.57	39.6	33.2	6.81	51.66	12.03	7.28	54.71	12.85	4.94	31.32	8.53	6.48	35.76	10.97	0	0	0	0	0	0
GBD-NER-standard	closed	0	0	0	0	0	0	8.85	48.34	14.97	9.51	51.26	16.04	11	6.04	7.8	15.44	8.99	11.37	0	0	0	0	0	0
MWETreeC	closed	0	0	0	31.71	12.87	18.31	0	0	0	33.57	11.03	16.61	0	0	0	30.95	8.35	13.15	0	0	0	0	0	0
SHOMA	open	74.07	40.82	52.63	80.77	41.58	54.9	56.92	35.07	43.4	65.48	37.93	48.03	59.46	24.18	34.38	75.71	28.69	41.61	0	0	0	0	0	0
TRAPACC	closed	7.89	6.12	6.9	15.79	8.91	11.39	33.75	12.8	18.56	35.4	13.1	19.13	33.8	13.19	18.97	39.88	13.92	20.63	0	0	0	0	0	0
TRAPACC_S	closed	25	2.04	3.77	40	1.98	3.77	42.55	9.48	15.5	44.68	9.66	15.88	50	9.89	16.51	61.45	10.92	18.55	0	0	0	0	0	0

↑ Back to top

HI

System	Track	LVC.cause						LVC.full						MVC						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAVERSAL	closed	20	8.33	11.76	33.33	8.33	13.33	66.08	58.44	62.02	72.82	69.94	71.35	68.13	83.85	75.17	68.93	89.62	77.93	10	2.63	4.17	31.25	3.62	6.49
Veyn	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	0	0	0	0	0	0	86.86	37.19	52.08	86.8	38.56	53.4	82.08	66.92	73.73	83.02	67.69	74.58	60	7.89	13.95	60	4.35	8.11
CRF-DepTree-categs	closed	100	8.33	15.38	100	8.33	15.38	77.82	71.25	74.39	80.81	72.87	76.64	80	83.08	81.51	80.3	83.08	81.66	12.5	5.26	7.41	22.86	5.8	9.25
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	0	0	0	0	0	0	47.52	75	58.18	49.71	75.95	60.09	60.7	93.85	73.72	60.4	93.85	73.49	0	0	0	0	0	0
GBD-NER-standard	closed	0	0	0	0	0	0	47.52	75	58.18	49.71	75.95	60.09	60.7	93.85	73.72	60.4	93.85	73.49	0	0	0	0	0	0
MWETreeC	closed	0	0	0	25	4.17	7.14	0	0	0	38.44	23.17	28.91	0	0	0	34.1	40	36.81	0	0	0	35.71	3.62	6.58
SHOMA	open	66.67	16.67	26.67	80	16.67	27.59	74.25	69.37	71.73	80.66	72.14	76.16	85.19	88.46	86.79	87.59	89.62	88.59	25	7.89	12	47.06	5.8	10.32
TRAPACC	closed	50	8.33	14.29	66.67	8.33	14.81	70.1	68.13	69.1	73.72	69.94	71.78	72.46	76.92	74.63	72.92	77.69	75.23	33.33	7.89	12.77	31.58	4.35	7.64
TRAPACC_S	closed	0	0	0	0	0	0	80.17	58.13	67.39	82.99	58.65	68.73	86.21	76.92	81.3	86.21	76.92	81.3	100	7.89	14.63	100	4.35	8.33

↑ Back to top

HR

System	Track	IAV						IRV						LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAVERSAL	closed	70.55	54.79	61.68	83.33	63.1	71.82	70.73	73.73	72.2	75.1	77.64	76.35	36.36	12.9	19.05	33.33	12.5	18.18	51.67	23.66	32.46	59.84	24.91	35.18	30	9.09	13.95	50	10.47	17.31
Veyn	closed	12.62	6.91	8.93	36.17	12.14	18.18	41.18	59.32	48.61	59.93	68.78	64.05	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	100	2.13	4.17	100	2.86	5.56	0	0	0	0	0	0	83.33	16.13	27.03	83.33	15.62	26.32	74.19	17.56	28.4	79.69	17.41	28.57	100	6.06	11.43	100	4.65	8.89
CRF-DepTree-categs	closed	16.13	13.3	14.58	65.82	37.14	47.49	67.78	51.69	58.65	73.44	59.49	65.73	75	9.68	17.14	85.71	9.38	16.9	32.76	14.5	20.11	33.87	14.33	20.14	0	0	0	37.5	3.49	6.38
CRF-Seq-nocategs	closed	20.9	34.57	26.05	43.37	52.14	47.35	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	38.58	52.13	44.34	48.39	60.71	53.85	47.06	47.46	47.26	47.06	47.26	47.16	0	0	0	0	0	0	21.55	38.17	27.55	23.34	37.2	28.68	0	0	0	0	0	0
GBD-NER-standard	closed	38.67	46.28	42.13	49.78	54.76	52.15	50.7	61.02	55.38	51.41	61.6	56.05	0	0	0	0	0	0	18.67	32.06	23.6	19.25	29.69	23.36	0	0	0	0	0	0
MWETreeC	closed	0	0	0	53.28	17.38	26.21	0	0	0	59.32	14.77	23.65	0	0	0	30.77	6.25	10.39	0	0	0	35.9	9.56	15.09	0	0	0	15.15	5.81	8.4
SHOMA	open	55.4	40.96	47.09	79.5	52.62	63.32	70.16	73.73	71.9	79.06	78.06	78.56	44.44	12.9	20	60	14.06	22.78	39.66	17.56	24.34	61.7	19.8	29.97	37.5	9.09	14.63	52.94	10.47	17.48
TRAPACC	closed	62.83	37.77	47.18	72.47	42.62	53.67	52.59	60.17	56.13	55.56	63.29	59.17	11.76	6.45	8.33	16	6.25	8.99	42.25	22.9	29.7	42.96	20.82	28.05	14.29	12.12	13.11	13.85	10.47	11.92
TRAPACC_S	closed	72.82	39.89	51.55	80.63	42.62	55.76	59.05	52.54	55.61	60.56	54.43	57.33	16.67	6.45	9.3	25	6.25	10	37.5	18.32	24.62	37.98	16.72	23.22	0	0	0	0	0	0

↑ Back to top

HU

System	Track	LVC.cause						LVC.full						VID						VPC.full						VPC.semi
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	73.08	67.86	70.37	77.55	69.09	73.08	78.3	50	61.03	77.83	50.32	61.12	100	50	66.67	100	58.82	74.07	96.88	95.88	96.38	95.83	95.14	95.49	94.05	91.86	92.94	93.02	90.91	91.95
TRAVERSAL	closed	89.47	60.71	72.34	89.47	61.82	73.12	72.3	64.46	68.15	81.75	71.34	76.19	100	60	75	100	70.59	82.76	91.24	77.16	83.61	91.76	84.17	87.8	93.42	82.56	87.65	93.67	84.09	88.62
Veyn	closed	90	32.14	47.37	90	32.73	48	38.89	29.52	33.56	69.27	42.36	52.57	0	0	0	25	11.76	16	91.91	84.16	87.86	94.74	84.17	89.14	59.09	75.58	66.33	59.09	73.86	65.66
varIDE	closed	100	10.71	19.35	100	10.91	19.67	100	3.61	6.98	100	2.87	5.57	0	0	0	0	0	0	100	10.08	18.32	100	9.35	17.11	100	25.58	40.74	100	26.14	41.44
CRF-DepTree-categs	closed	66.67	71.43	68.97	84.31	78.18	81.13	64.89	51.2	57.24	75.86	56.05	64.47	62.5	50	55.56	92.31	70.59	80	95.31	79.42	86.64	95.26	79.5	86.67	94.81	84.88	89.57	94.87	84.09	89.16
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	58.06	64.29	61.02	64.91	67.27	66.07	70.16	52.41	60	76.15	57.96	65.82	83.33	50	62.5	100	64.71	78.57	75.32	72.84	74.06	76.82	75.72	76.27	94.87	86.05	90.24	94.87	84.09	89.16
GBD-NER-resplit	closed	0	0	0	0	0	0	58.56	39.16	46.93	60.81	42.99	50.37	45.45	50	47.62	50	64.71	56.41	54.46	12.55	20.4	54.46	21.94	31.28	0	0	0	0	0	0
GBD-NER-standard	closed	66.67	64.29	65.45	66.67	65.45	66.06	60.61	36.14	45.28	63.64	40.13	49.22	57.14	40	47.06	57.14	47.06	51.61	66.29	12.14	20.52	66.29	21.22	32.15	0	0	0	0	0	0
MWETreeC	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	25	0.21	0.41	25	0.18	0.36	0	0	0	0	0	0
SHOMA	open	91.3	75	82.35	93.18	74.55	82.83	71.79	50.6	59.36	88.35	57.96	70	88.89	80	84.21	93.75	88.24	90.91	94.74	88.89	91.72	95.4	89.57	92.39	80.39	95.35	87.23	80.77	95.45	87.5
TRAPACC	closed	52.5	75	61.76	57.53	76.36	65.62	74.4	56.02	63.92	72.8	55.41	62.93	100	70	82.35	100	76.47	86.67	95.72	96.71	96.21	95.04	96.58	95.81	96.15	87.21	91.46	95	86.36	90.48

↑ Back to top

IT

System	Track	IAV						IRV						LVC.cause						LVC.full						MVC						VID						VPC.full						LS.ICV
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
SHOMA	open	48.65	43.9	46.15	67.86	56.44	61.62	44.14	51.04	47.34	49.76	53.12	51.39	38.71	48	42.86	49.15	54.72	51.79	55.41	39.42	46.07	63.09	40.69	49.47	50	40	44.44	62.5	45.45	52.63	44.53	30.35	36.09	63.79	35.12	45.3	61.54	34.78	44.44	65.38	34	44.74	33.33	12.5	18.18	80	23.53	36.36
TRAPACC	closed	68.42	31.71	43.33	71.43	29.7	41.96	43.2	56.25	48.87	43.82	57.29	49.66	33.33	16	21.62	45	16.98	24.66	53.66	21.15	30.34	54.02	20.35	29.56	0	0	0	0	0	0	35.63	15.42	21.53	40.61	14.26	21.11	66.67	8.7	15.38	66.67	8	14.29	0	0	0	0	0	0
TRAPACC_S	closed	80	19.51	31.37	80.95	16.83	27.87	49.41	43.75	46.41	50	44.27	46.96	26.67	16	20	42.86	16.98	24.32	59.46	21.15	31.21	60	19.48	29.41	0	0	0	0	0	0	34.38	10.95	16.6	43.64	11.04	17.63	100	4.35	8.33	100	4	7.69	0	0	0	0	0	0
TRAVERSAL	closed	62.07	43.9	51.43	84.06	57.43	68.24	65.56	61.46	63.44	68.75	63.02	65.76	52.38	44	47.83	67.65	43.4	52.87	79.59	37.5	50.98	86.27	38.1	52.85	25	20	22.22	25	18.18	21.05	52	25.87	34.55	65.33	30.06	41.18	52.17	52.17	52.17	64.29	54	58.7	40	25	30.77	66.67	35.29	46.15
Veyn	closed	13.64	14.63	14.12	47.95	34.65	40.23	44.96	60.42	51.56	53.09	67.19	59.31	18.18	8	11.11	46.67	13.21	20.59	33.96	34.62	34.29	53.59	41.99	47.09	0	0	0	0	0	0	22.15	16.42	18.86	41.67	20.71	27.66	40	8.7	14.29	50	8	13.79	0	0	0	0	0	0
varIDE	closed	21.55	60.98	31.85	22.22	55.45	31.73	19.15	18.75	18.95	26.6	26.04	26.32	55	44	48.89	55	41.51	47.31	57.94	59.62	58.77	58.82	56.28	57.52	0	0	0	0	0	0	27.74	18.91	22.49	35.35	17.02	22.98	62.5	43.48	51.28	68.75	44	53.66	0	0	0	0.93	5.88	1.6
CRF-DepTree-categs	closed	38.46	24.39	29.85	55.36	30.69	39.49	46.05	36.46	40.7	56.62	40.1	46.95	54.55	24	33.33	59.09	24.53	34.67	50.82	29.81	37.58	60.83	31.6	41.6	100	20	33.33	100	18.18	30.77	28.57	14.93	19.61	42.81	20.09	27.35	54.55	26.09	35.29	66.67	32	43.24	50	12.5	20	40	11.76	18.18
CRF-Seq-nocategs	closed	4.12	24.39	7.04	7.9	42.57	13.33	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	50	19.51	28.07	75.86	21.78	33.85	56.25	46.88	51.14	58.6	47.92	52.72	28.57	16	20.51	45.45	18.87	26.67	43.33	25	31.71	59.43	27.27	37.39	100	20	33.33	100	18.18	30.77	15.38	6.97	9.59	53.09	13.19	21.13	73.33	47.83	57.89	79.31	46	58.23	0	0	0	0	0	0
GBD-NER-resplit	closed	0	0	0	1.78	3.96	2.45	23.51	61.46	34.01	23.66	61.98	34.24	0	0	0	0	0	0	4.67	25	7.87	5.75	27.71	9.52	0	0	0	0	0	0	4.99	10.45	6.75	9.87	17.18	12.53	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-standard	closed	4.88	9.76	6.5	7.27	11.88	9.02	20.87	75	32.65	21.01	75.52	32.88	0	0	0	0	0	0	6.1	20.19	9.38	6.83	20.35	10.23	0	0	0	0	0	0	10.17	8.96	9.52	13.63	9.51	11.2	0	0	0	0	0	0	0	0	0	0	0	0
MWETreeC	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	3.9	0.46	0.82	0	0	0	0	0	0	0	0	0	0	0	0
Polirem-basic	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	29.17	3.48	6.22	34.55	2.91	5.37	0	0	0	0	0	0	0	0	0	0	0	0
Polirem-rich	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	39.84	24.38	30.25	52.29	26.23	34.93	0	0	0	0	0	0	0	0	0	0	0	0

↑ Back to top

LT

System	Track	LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAVERSAL	closed	0	0	0	0	0	0	27.64	19.37	22.77	56.94	28.12	37.65	41.18	6.93	11.86	47.54	5.51	9.88
Veyn	closed	0	0	0	0	0	0	16.2	10.21	12.53	40.15	18.28	25.12	0	0	0	100	0.19	0.38
varIDE	closed	0	0	0	0	0	0	0	0	0	0	0	0	55.56	2.48	4.74	55.56	2.85	5.42
CRF-DepTree-categs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	100	21.43	35.29	100	21.43	35.29	3.34	44.72	6.22	3.67	49.03	6.83	1.3	9.41	2.29	2.31	14.26	3.98
GBD-NER-standard	closed	100	21.43	35.29	100	21.43	35.29	3.34	44.72	6.22	3.67	49.03	6.83	1.3	9.41	2.29	2.31	14.26	3.98
MWETreeC	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
SHOMA	open	90	64.29	75	94.74	64.29	76.6	39.05	23.24	29.14	62.07	28.47	39.04	16.07	4.46	6.98	31.11	5.32	9.09
TRAPACC	closed	4.71	28.57	8.08	8.89	28.57	13.56	39.53	35.92	37.64	41.28	37.43	39.26	15.28	5.45	8.03	18.87	5.7	8.76
TRAPACC_S	closed	7.5	21.43	11.11	13.95	21.43	16.9	52.38	30.99	38.94	53.57	31.63	39.78	22.22	6.93	10.57	23.97	6.65	10.42

↑ Back to top

PL

System	Track	IAV						IRV						LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	100	41.38	58.54	100	43.75	60.87	89.39	71.08	79.19	90.15	71.54	79.78	0	0	0	0	0	0	54.02	31.54	39.83	55.56	32.79	41.24	66.67	10.96	18.82	63.89	11.11	18.93
TRAVERSAL	closed	95	65.52	77.55	95.45	65.62	77.78	87.67	79.92	83.61	88.4	80.96	84.52	36.36	26.67	30.77	43.48	30.3	35.71	61.26	45.64	52.31	72.73	47.21	57.26	46.67	19.18	27.18	58.54	23.19	33.22
Veyn	closed	25	17.24	20.41	61.11	34.38	44	86.44	81.93	84.12	89.42	82.97	86.07	0	0	0	0	0	0	43.75	42.28	43	50.36	45.9	48.03	15.91	9.59	11.97	38.2	16.43	22.97
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	0	0	0	0	0	0	0	0	0	0	0	0	75	20	31.58	75	18.18	29.27	87.1	18.12	30	90.32	18.36	30.52	100	1.37	2.7	100	0.97	1.91
CRF-DepTree-categs	closed	2.33	10.34	3.8	16.08	35.94	22.22	42.37	70.28	52.87	52.84	91.38	66.96	0	0	0	1.75	3.03	2.22	24.07	26.17	25.08	29.78	35.74	32.49	7.03	12.33	8.96	12.54	16.91	14.4
CRF-Seq-nocategs	closed	4.12	48.28	7.59	6.21	65.62	11.35	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	100	44.83	61.9	100	40.62	57.78	82.07	82.73	82.4	83.33	83.17	83.25	18.18	13.33	15.38	22.22	12.12	15.69	69.89	43.62	53.72	75.84	44.26	55.9	4.55	2.74	3.42	59.49	22.71	32.87
GBD-NER-resplit	closed	31.25	34.48	32.79	43.75	43.75	43.75	66.27	89.16	76.03	67.01	89.98	76.82	5.33	26.67	8.89	5.92	27.27	9.73	29.85	53.69	38.37	30.11	53.11	38.43	15.04	23.29	18.28	22.27	27.54	24.62
GBD-NER-standard	closed	34.78	27.59	30.77	46.81	34.38	39.64	76.21	88.76	82	76.55	88.98	82.3	0	0	0	0	0	0	30.73	42.28	35.59	31.46	42.3	36.08	81.82	12.33	21.43	81.82	13.04	22.5
MWETreeC	closed	0	0	0	85	26.56	40.48	0	0	0	43.72	49.5	46.43	0	0	0	20	9.09	12.5	0	0	0	32.86	15.08	20.67	0	0	0	22.22	3.86	6.58
SHOMA	open	94.12	55.17	69.57	100	57.81	73.27	76.08	77.91	76.98	80	80.16	80.08	25	13.33	17.39	33.33	12.12	17.78	67.39	41.61	51.45	73.91	44.59	55.62	60	20.55	30.61	75.38	23.67	36.03
TRAPACC	closed	85.71	41.38	55.81	90.32	43.75	58.95	88.64	78.31	83.16	89.32	78.76	83.71	4.76	6.67	5.56	7.41	6.06	6.67	48.48	32.21	38.71	51.22	34.43	41.18	34.78	10.96	16.67	47.54	14.01	21.64

↑ Back to top

PT

System	Track	IRV						LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
SHOMA	open	79.49	68.13	73.37	86.09	70.65	77.61	60	42.86	50	70	46.67	56	71.34	67.95	69.6	79.06	73.23	76.03	60.95	54.24	57.4	69.2	56.02	61.92
TRAPACC	closed	80.6	59.34	68.35	81.34	59.24	68.55	0	0	0	0	0	0	51.92	52.23	52.07	55.17	54.85	55.01	44.87	29.66	35.71	49.12	31.37	38.29
TRAPACC_S	closed	83.33	60.44	70.06	83.33	59.78	69.62	0	0	0	0	0	0	59.53	45.4	51.52	63.44	47.47	54.3	64.52	16.95	26.85	70.79	17.65	28.25
TRAVERSAL	closed	81.33	67.03	73.49	82.99	66.3	73.72	28.57	28.57	28.57	46.15	40	42.86	77.09	51.93	62.06	85.28	57.02	68.34	69.12	39.83	50.54	83.15	42.86	56.56
Veyn	closed	75.95	65.93	70.59	81.33	66.3	73.05	0	0	0	0	0	0	69.82	56.97	62.75	75.09	58.9	66.02	27.18	23.73	25.34	45.29	28.29	34.83
mumpitz	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
varIDE	closed	73.17	32.97	45.45	74.39	33.15	45.86	75	42.86	54.55	77.78	46.67	58.33	71.06	57.57	63.61	73.33	58.9	65.33	78.48	52.54	62.94	82.27	50.7	62.74
CRF-DepTree-categs	closed	69.35	47.25	56.21	72.22	49.46	58.71	0	0	0	21.43	20	20.69	54.49	48.66	51.41	60.06	54.85	57.34	37.78	14.41	20.86	56.8	19.89	29.46
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	70.59	39.56	50.7	70.59	39.13	50.35	0	0	0	0	0	0	75.76	59.35	66.56	78.03	59.62	67.6	45.95	14.41	21.94	83.78	17.37	28.77
GBD-NER-resplit	closed	25.52	81.32	38.85	25.52	80.43	38.74	0	0	0	0	0	0	12.35	62.31	20.62	13.05	65.7	21.77	9.54	29.66	14.43	10.93	31.37	16.21
GBD-NER-standard	closed	35.61	80.22	49.32	35.61	79.35	49.16	0	0	0	0	0	0	17.04	49.55	25.36	18.14	51.52	26.83	12.73	11.86	12.28	17.89	15.69	16.72
MWETreeC	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Polirem-basic	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	39.13	7.63	12.77	31.25	5.6	9.5
Polirem-rich	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	31.66	53.39	39.75	36.76	56.02	44.4

↑ Back to top

RO

System	Track	IRV						LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	86.71	82.64	84.63	87.14	83.06	85.05	83.33	78.95	81.08	90	80.36	84.91	100	38.24	55.32	100	36.62	53.61	92.86	75.14	83.07	91.74	72.29	80.86
TRAVERSAL	closed	83.1	82.64	82.87	85.71	85.12	85.42	95	100	97.44	98.25	100	99.12	84.62	64.71	73.33	88.89	67.61	76.8	92.75	73.99	82.32	93.86	77.35	84.81
Veyn	closed	79.7	88.71	83.96	81.08	89.12	84.91	85.71	94.74	90	96.49	98.21	97.35	70	61.76	65.62	74.58	61.97	67.69	81.37	75.72	78.44	84.57	76.63	80.4
varIDE	closed	53.29	89.26	66.74	53.95	90.36	67.56	90.48	100	95	90.32	100	94.92	55.56	88.24	68.18	55.26	88.73	68.11	72.56	90.17	80.41	72.11	87.23	78.95
CRF-DepTree-categs	closed	70.17	68.04	69.09	76.79	71.07	73.82	81.25	68.42	74.29	85.71	75	80	64.29	26.47	37.5	64.29	25.35	36.36	85.71	55.49	67.37	87.94	54.46	67.26
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	89.67	81.27	85.26	90.12	81.68	85.69	4.17	5.26	4.65	84.44	67.86	75.25	83.33	58.82	68.97	83.33	56.34	67.23	65.41	50.29	56.86	94.68	60	73.45
GBD-NER-resplit	closed	40.63	92.56	56.47	40.93	93.25	56.89	29.03	94.74	44.44	31.07	98.21	47.21	12.92	67.65	21.7	13.17	66.2	21.96	16.77	75.14	27.43	18.02	74.94	29.05
GBD-NER-standard	closed	46.71	92.01	61.97	46.92	92.42	62.24	41.46	89.47	56.67	45.61	92.86	61.18	13.18	50	20.86	13.18	47.89	20.67	31.16	71.68	43.43	33.41	70.36	45.31
MWETreeC	closed	0	0	0	79.35	30.17	43.71	0	0	0	35.42	30.36	32.69	0	0	0	43.48	28.17	34.19	0	0	0	43.17	33.49	37.72
SHOMA	open	84.68	86.78	85.71	86.67	87.74	87.2	94.44	89.47	91.89	100	92.86	96.3	90.32	82.35	86.15	92.19	83.1	87.41	93.75	86.71	90.09	95.2	86.02	90.38
TRAPACC	closed	83.81	88.43	86.06	84.33	88.98	86.6	78.26	94.74	85.71	79.1	94.64	86.18	95.65	64.71	77.19	100	67.61	80.67	83.63	82.66	83.14	83.83	81.2	82.5

↑ Back to top

SL

System	Track	IAV						IRV						LVC.cause						LVC.full						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAPACC_S	closed	54.55	29.7	38.46	60.91	30.32	40.48	56.83	42.45	48.6	61.48	45.92	52.57	0	0	0	0	0	0	25	11.43	15.69	31.25	13.33	18.69	20	4.72	7.63	27.78	5.07	8.57
TRAVERSAL	closed	75.61	61.39	67.76	78.89	64.25	70.82	90.16	67.35	77.1	90.16	67.35	77.1	27.27	23.08	25	65	41.94	50.98	58.82	28.57	38.46	66.67	26.67	38.1	63.83	28.3	39.22	68.46	30.07	41.78
Veyn	closed	23.81	14.85	18.29	48.35	19.91	28.21	84.42	53.06	65.16	87.79	54.29	67.09	0	0	0	0	0	0	0	0	0	0	0	0	9.8	4.72	6.37	22.52	8.45	12.29
varIDE	closed	18.58	83.17	30.38	19.74	83.71	31.95	46.89	80	59.13	49.16	83.88	61.99	45.83	84.62	59.46	46.55	87.1	60.67	34.55	54.29	42.22	36.04	53.33	43.01	19.14	37.74	25.4	21.8	36.82	27.39
CRF-DepTree-categs	closed	13.36	36.63	19.58	19.43	43.44	26.85	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
CRF-Seq-nocategs	closed	13.83	38.61	20.37	19.42	45.25	27.17	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Deep-BGT	open	81.58	30.69	44.6	85.53	29.41	43.77	73.6	59.18	65.61	75.77	60	66.97	0	0	0	80	12.9	22.22	28.57	11.43	16.33	50	13.33	21.05	12.5	8.49	10.11	41.35	18.58	25.64
GBD-NER-resplit	closed	16.28	48.51	24.38	17.38	47.96	25.51	42.94	85.71	57.22	43.25	86.33	57.63	0	0	0	0	0	0	8.41	25.71	12.68	9.81	28	14.53	3.98	18.87	6.58	6.53	24.66	10.33
GBD-NER-standard	closed	20.18	21.78	20.95	20.09	19.91	20	39.22	86.12	53.9	39.68	87.14	54.53	0	0	0	0	0	0	4.55	5.71	5.06	5.68	6.67	6.13	20	2.83	4.96	31.43	3.72	6.65
MWETreeC	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
SHOMA	open	58.02	46.53	51.65	78.43	54.3	64.17	66.25	64.9	65.57	73.98	70.2	72.04	0	0	0	66.67	6.45	11.76	34.78	22.86	27.59	57.14	26.67	36.36	37.5	19.81	25.93	62.3	25.68	36.36
TRAPACC	closed	38.24	25.74	30.77	45.26	28.05	34.64	55.43	39.59	46.19	60.86	43.47	50.71	0.56	15.38	1.08	1.1	12.9	2.03	13.33	5.71	8	23.33	9.33	13.33	16	3.77	6.11	26.92	4.73	8.05

↑ Back to top

TR

System	Track	LVC.full						MVC						VID
		MWE-based			Token-based			MWE-based			Token-based			MWE-based			Token-based
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
TRAVERSAL	closed	90.91	29.41	44.44	93.79	30.29	45.79	0	0	0	0	0	0	62.16	19.74	29.97	73.33	22.22	34.11
Veyn	closed	67.48	40.44	50.57	79.69	46.53	58.76	0	0	0	0	0	0	40.88	27.9	33.16	50.65	31.31	38.7
varIDE	closed	4.24	56.62	7.9	5.64	74.64	10.48	0	0	0	0	0	0	3.03	55.36	5.75	4.14	71.31	7.83
CRF-DepTree-categs	closed	69.78	35.66	47.2	82.31	39.05	52.97	0	0	0	0	0	0	48.81	17.6	25.87	69.87	22.02	33.49
CRF-Seq-nocategs	closed	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
GBD-NER-resplit	closed	44.05	13.6	20.79	46.43	14.23	21.79	0	0	0	0	0	0	26.29	24.03	25.11	28.9	25.05	26.84
GBD-NER-standard	closed	23.93	10.29	14.4	24.79	10.58	14.83	0	0	0	0	0	0	18.18	31.76	23.12	21.01	34.55	26.13
MWETreeC	closed	0	0	0	5.81	1.82	2.78	0	0	0	0	0	0	0	0	0	2.63	0.81	1.24
SHOMA	open	88.31	50	63.85	92.83	52.01	66.67	0	0	0	0	0	0	64.12	36.05	46.15	70.45	37.58	49.01
TRAPACC	closed	23.08	1.1	2.11	42.31	2.01	3.83	0	0	0	0	0	0	42.86	1.29	2.5	57.14	1.62	3.14
TRAPACC_S	closed	0	0	0	16.67	0.18	0.36	0	0	0	0	0	0	66.67	0.86	1.69	66.67	0.81	1.6

↑ Back to top