Loading...
Publication
Avaliação de modelos transformers bert de pergunta/Resposta com ajuste fino num corpus em português europeu
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 5.48 MB | Adobe PDF |
Authors
Advisor(s)
Abstract(s)
Este trabalho visa a avaliação e comparação de modelos baseados na arquitetura de Transformers para a tarefa de Pergunta/Resposta que foram pré-treinados com dados em língua portuguesa. Foi feito o ajuste fino de três modelos de linguagem baseados no modelo BERTimbau (modelo BERT pré-treinado para português): o bert-base-cased-squad-v1.1-portuguese (SQUAD-PT-BASE); o faquad-bert-base-portuguese-cased (FAQUAD-PT-BASE) e o bert-large-cased-squad-v1.1-portuguese (SQUAD-PT-LARGE). Estes modelos fazem uso da arquitetura Transformer, possuindo respetivamente 108 334 082, 108 334 082 e 333 348 866 parâmetros. Para efetuar o ajuste fino, foi construído um dataset no formato SQuAD (Stanford Question Answering Dataset) com 265 perguntas em português europeu, acerca da Fundação da Nossa Senhora da Esperança. A validação do desempenho dos modelos foi efetuada através das métricas F1 e accuracy, tendo o melhor resultado sido alcançado com o modelo SQUAD-PT-BASE com um F1 de 0,91111 e uma accuracy de 0,96154. Os valores obtidos permitem concluir que o ajuste fino efetuado aos modelos pré-treinados, conduzem a resultados para tarefas de Pergunta/Resposta bastante satisfatórios, tendo atingido uma performance melhor do que os modelos originais e de outros trabalhos semelhantes
This work aims to evaluate and compare question/answering Transformer models that have been pre-trained with Portuguese data. Fine-tuning was performed on three language models based on BERTimbau model (BERT model pre-trained for Portuguese): bert-base-cased-squad-v1.1-portuguese (SQUAD-PT-BASE); faquad-bert-base-portuguese-cased (FAQUAD-PT-BASE) and bert-large-cased-squad-v1.1-portuguese (SQUAD-PT-LARGE). These models make use of the Transformer architecture and they have 108 334 082, 108 334 082 and 333 348 866 parameters respectively. For the fine-tuning, a dataset in SQuAD (Stanford Question Answering Dataset) format was constructed with 265 questions in European Portuguese about Fundação da Nossa Senhora da Esperança. The comparison was carried out using F1 and accuracy metrics, with the best result being achieved with the SQUAD-PT-BASE model with an F1 of 0,91111 and an accuracy of 0,96154. These results allow us to conclude that the fine-tuning performed on the pre-trained models leads to very good results for the Question/Answering task, having achieved better performance than the original models and other similar works.
This work aims to evaluate and compare question/answering Transformer models that have been pre-trained with Portuguese data. Fine-tuning was performed on three language models based on BERTimbau model (BERT model pre-trained for Portuguese): bert-base-cased-squad-v1.1-portuguese (SQUAD-PT-BASE); faquad-bert-base-portuguese-cased (FAQUAD-PT-BASE) and bert-large-cased-squad-v1.1-portuguese (SQUAD-PT-LARGE). These models make use of the Transformer architecture and they have 108 334 082, 108 334 082 and 333 348 866 parameters respectively. For the fine-tuning, a dataset in SQuAD (Stanford Question Answering Dataset) format was constructed with 265 questions in European Portuguese about Fundação da Nossa Senhora da Esperança. The comparison was carried out using F1 and accuracy metrics, with the best result being achieved with the SQUAD-PT-BASE model with an F1 of 0,91111 and an accuracy of 0,96154. These results allow us to conclude that the fine-tuning performed on the pre-trained models leads to very good results for the Question/Answering task, having achieved better performance than the original models and other similar works.
Description
Keywords
Inteligência Artificial Transformers Pergunta/Resposta Processamento de Linguagem Natural Artificial Intelligence Transformers Question/Answering Natural Language Processing