Parser does not get best metrics #82
Description
Hello,
I wonder this is a bug or not. When I trained a model with 100 epochs, the last epoch had the highest metric but it did not save, instead of saving another lower metric model. See my log for more information.
Moreover, I remember that in old version, the parser had early stopping, but I did not see it in this version. It is really a cool features though.
Thank you for your help.
P/S: I use the latest code and not change anything.
2021-09-15 13:47:59 INFO Epoch 90 / 100:
2021-09-15 13:51:17 INFO lr: 5.6657e-06 - loss: 0.0040 - UCM: 96.24% LCM: 92.89% UAS: 99.81% LAS: 99.64%
2021-09-15 13:51:18 INFO dev: loss: 3.4414 - UCM: 26.50% LCM: 11.00% UAS: 85.42% LAS: 77.93%
2021-09-15 13:51:20 INFO test: loss: 3.6520 - UCM: 27.16% LCM: 12.94% UAS: 85.25% LAS: 78.42%
2021-09-15 13:51:20 INFO 0:03:20.924831s elapsed
2021-09-15 13:51:20 INFO Epoch 91 / 100:
2021-09-15 13:54:38 INFO lr: 5.1114e-06 - loss: 0.0024 - UCM: 96.17% LCM: 93.22% UAS: 99.81% LAS: 99.66%
2021-09-15 13:54:39 INFO dev: loss: 3.5128 - UCM: 26.50% LCM: 11.00% UAS: 85.61% LAS: 78.24%
2021-09-15 13:54:41 INFO test: loss: 3.7178 - UCM: 27.75% LCM: 12.94% UAS: 85.28% LAS: 78.40%
2021-09-15 13:54:41 INFO 0:03:20.681193s elapsed
2021-09-15 13:54:41 INFO Epoch 92 / 100:
2021-09-15 13:57:59 INFO lr: 4.5570e-06 - loss: 0.0021 - UCM: 96.27% LCM: 92.98% UAS: 99.81% LAS: 99.65%
2021-09-15 13:57:59 INFO dev: loss: 3.4911 - UCM: 27.00% LCM: 11.00% UAS: 85.42% LAS: 78.00%
2021-09-15 13:58:01 INFO test: loss: 3.6980 - UCM: 27.45% LCM: 13.43% UAS: 85.24% LAS: 78.37%
2021-09-15 13:58:01 INFO 0:03:20.802275s elapsed
2021-09-15 13:58:01 INFO Epoch 93 / 100:
2021-09-15 14:01:19 INFO lr: 4.0027e-06 - loss: 0.0045 - UCM: 96.47% LCM: 93.31% UAS: 99.82% LAS: 99.66%
2021-09-15 14:01:20 INFO dev: loss: 3.4927 - UCM: 26.00% LCM: 11.00% UAS: 85.38% LAS: 77.93%
2021-09-15 14:01:22 INFO test: loss: 3.7007 - UCM: 27.94% LCM: 13.43% UAS: 85.32% LAS: 78.43%
2021-09-15 14:01:22 INFO 0:03:20.380205s elapsed
2021-09-15 14:01:22 INFO Epoch 94 / 100:
2021-09-15 14:04:40 INFO lr: 3.4484e-06 - loss: 0.0009 - UCM: 96.39% LCM: 93.42% UAS: 99.81% LAS: 99.67%
2021-09-15 14:04:41 INFO dev: loss: 3.5013 - UCM: 26.50% LCM: 11.00% UAS: 85.49% LAS: 78.08%
2021-09-15 14:04:43 INFO test: loss: 3.6941 - UCM: 27.84% LCM: 13.53% UAS: 85.27% LAS: 78.38%
2021-09-15 14:04:43 INFO 0:03:21.236538s elapsed
2021-09-15 14:04:43 INFO Epoch 95 / 100:
2021-09-15 14:08:01 INFO lr: 2.8941e-06 - loss: 0.0002 - UCM: 96.30% LCM: 93.34% UAS: 99.81% LAS: 99.67%
2021-09-15 14:08:02 INFO dev: loss: 3.5176 - UCM: 26.50% LCM: 11.00% UAS: 85.45% LAS: 78.05%
2021-09-15 14:08:04 INFO test: loss: 3.7169 - UCM: 27.65% LCM: 13.53% UAS: 85.31% LAS: 78.47%
2021-09-15 14:08:04 INFO 0:03:20.700472s elapsed
2021-09-15 14:08:04 INFO Epoch 96 / 100:
2021-09-15 14:11:22 INFO lr: 2.3398e-06 - loss: 0.0027 - UCM: 96.48% LCM: 93.47% UAS: 99.82% LAS: 99.67%
2021-09-15 14:11:22 INFO dev: loss: 3.5252 - UCM: 26.50% LCM: 12.00% UAS: 85.54% LAS: 78.19%
2021-09-15 14:11:24 INFO test: loss: 3.7445 - UCM: 27.65% LCM: 13.14% UAS: 85.39% LAS: 78.54%
2021-09-15 14:11:24 INFO 0:03:20.688394s elapsed
2021-09-15 14:11:24 INFO Epoch 97 / 100:
2021-09-15 14:14:43 INFO lr: 1.7855e-06 - loss: 0.0029 - UCM: 96.69% LCM: 93.61% UAS: 99.83% LAS: 99.68%
2021-09-15 14:14:43 INFO dev: loss: 3.5313 - UCM: 26.50% LCM: 11.00% UAS: 85.54% LAS: 78.22%
2021-09-15 14:14:45 INFO test: loss: 3.7412 - UCM: 28.04% LCM: 13.63% UAS: 85.35% LAS: 78.53%
2021-09-15 14:14:45 INFO 0:03:21.046132s elapsed
2021-09-15 14:14:45 INFO Epoch 98 / 100:
2021-09-15 14:18:04 INFO lr: 1.2312e-06 - loss: 0.0042 - UCM: 96.91% LCM: 94.08% UAS: 99.84% LAS: 99.70%
2021-09-15 14:18:05 INFO dev: loss: 3.5500 - UCM: 26.50% LCM: 11.50% UAS: 85.59% LAS: 78.24%
2021-09-15 14:18:07 INFO test: loss: 3.7603 - UCM: 28.14% LCM: 13.73% UAS: 85.40% LAS: 78.55%
2021-09-15 14:18:07 INFO 0:03:21.250676s elapsed
2021-09-15 14:18:07 INFO Epoch 99 / 100:
2021-09-15 14:21:25 INFO lr: 6.7694e-07 - loss: 0.0031 - UCM: 96.66% LCM: 93.89% UAS: 99.83% LAS: 99.69%
2021-09-15 14:21:26 INFO dev: loss: 3.5548 - UCM: 26.50% LCM: 11.50% UAS: 85.56% LAS: 78.19%
2021-09-15 14:21:28 INFO test: loss: 3.7634 - UCM: 28.14% LCM: 13.53% UAS: 85.40% LAS: 78.53%
2021-09-15 14:21:28 INFO 0:03:21.061997s elapsed
2021-09-15 14:21:28 INFO Epoch 100 / 100:
2021-09-15 14:24:46 INFO lr: 1.2263e-07 - loss: 0.0029 - UCM: 96.87% LCM: 93.96% UAS: 99.84% LAS: 99.70%
2021-09-15 14:24:46 INFO dev: loss: 3.5592 - UCM: 26.50% LCM: 11.00% UAS: 85.59% LAS: 78.24%
2021-09-15 14:24:48 INFO test: loss: 3.7705 - UCM: 28.04% LCM: 13.43% UAS: 85.42% LAS: 78.59%
2021-09-15 14:24:48 INFO 0:03:20.730533s elapsed
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Some weights of the model checkpoint at vinai/phobert-base were not used when initializing RobertaModel: ['lm_head.layer_norm.weight', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.bias', 'lm_head.decoder.weight', 'lm_head.decoder.bias', 'lm_head.layer_norm.bias']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2021-09-15 14:25:03 INFO Epoch 21 saved
2021-09-15 14:25:03 INFO dev: UCM: 21.50% LCM: 13.00% UAS: 85.47% LAS: 78.31%
2021-09-15 14:25:03 INFO test: UCM: 25.88% LCM: 12.55% UAS: 84.71% LAS: 78.10%
2021-09-15 14:25:03 INFO 5:34:33.226983s elapsed, 0:03:20.732270s/epoch