Brazil

This work focuses on the enrichment of existing Portuguese word embeddings with visual information. The combined text-image embeddings were tested against their text-only counterparts in common NLP tasks. The new embeddings were tested in two different domains - general news and a geosciences. The results show an increase in performance for several tasks, which indicates that visual information fusion for word embeddings can be useful for certain tasks.

PROPOR 2022

Enriching Portuguese Word Embeddings with Visual Information

geosciences

portuguese

multimodal

word embeddings

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). **ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. **STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS 2021

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

poster

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician. The meeting has been a very rich forum for the exchange of ideas and partnerships for the research and industry communities dedicated to the automated processing of Portuguese, promoting the development of methodologies, resources and projects that can be shared among researchers and practitioners in the field.

Find more information and pay the registration fee on [event organizer’s website](https://sites.universidadedefortaleza.com/propor2022/?page_id=401)

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician.

This work presents an analysis of Brazilian political discourse from speeches and social media posts, focusing on the ability to transfer learned models’ knowledge between different contexts. The analysis is conducted through PoliS, a new resource containing two datasets of political discussions labeled for party and ideological leaning from congressional speeches and social media posts by political influencers. The transfer learning experiments are performed using the transcripts of the congressional speeches to train a model used to predict the political leaning of social media influencers. To evaluate the robustness of the model, the analysis includes a time-lag study of the performance degradation of the transferred model. We find that relatively little social media data (about one hour) is needed to achieve reasonable performance in classification, and that performance does not degrade significantly over time.

A transfer learning analysis of political leaning classification in cross-domain content

Legal text processing is a challenging task for modeling approaches due to the peculiarities inherent to its features, such as long texts and their technical vocabulary. Topic modeling consists of discovering a semantic structure in the text. This paper investigates the application of topic modeling and the use of information about the legislation cited in identifying the subject of legal documents and evaluating its applicability in the classification of Brazilian lawsuits. The models were trained with a Golden Collection of 16 thousand initial petitions and indictments from the Court of Justice of the State of Ceará, in Brazil, whose lawsuits were classified in the five more representative National Council of Justice (CNJ) of Brazil classes - Common Civil Procedure, Execution of Extrajudicial Title, Criminal Action - Ordinary Procedure, Special Civil Court Procedure, and Tax Enforcement. The results obtained outperform the baseline, achieving 0.89 of F1 score (macro). Our interpretation is that the representation of the document through contextual embeddings generated by BERT, as well as the architecture of the model with bidirectional contexts, makes it possible to capture the specific context of the domain of legal documents. Thus, the use of the legislation mentioned in the representation of documents can improve the accuracy of the classification task.


Using Topic Modeling in Classification of Brazilian Lawsuits

Even though most of its energy generation comes from renewable sources, Brazil is one of the largest emitters of greenhouse gases in the world, due to intense farming and deforestation of biomes such as the Amazon Rainforest, whose preservation is essential for compliance with the Paris Agreement. Still, regardless of lobbies or prevailing political orientation, all government legal actions are published daily in the Brazilian Federal Official Gazette (BFOG, or ``Diário Oficial da União" in Portuguese). However, with hundreds of decrees issued every day by the authorities, it is absolutely burdensome to manually analyze all these processes and find out which ones can pose serious environmental hazards. In this paper, we present a strategy to compose automated techniques and domain expert knowledge to process all the data from the BFOG. We also provide the Government Actions Tracker, a highly curated dataset, in Portuguese, annotated by domain experts, on federal government acts about the Brazilian environmental policies. Finally, we build and compared four different NLP models on the classfication task in this dataset. Our best model achieved a F1-score of $0.714 \pm 0.031$. In the future, this system should serve to scale up the high-quality tracking of all oficial documents with a minimum of human supervision and contribute to increasing society's awareness of government actions.

Tracking Environmental Policy Changes in the Brazilian Federal Official Gazette

The anchorage to real data is one of the main parameters that guarantees the quality and the coverage of lexical resources, especially in the context of specialized domains. Thus, lexicon extraction from corpora is a consensual method for building lexical resources. However, given that data validation by experts in specialized contexts is a necessary step, the automatic screening of data becomes fundamental to maximize the informational value of the interaction with experts. In this presentation we present and discuss a hybrid methodology, combining linguistic and statistical approaches, focusing on the extraction of specialized lexical units and salient semantic information using CQL grammars. The proposed method involves several steps, from frequency information analyses, concordances and collocations extraction to manual revision and expert validation and encompasses the construction and application of knowledge-based patterns CQL grammars. We present two CQL grammars for lexical and semantic information extraction developed for Portuguese and Italian and evaluate results from its application to specialized corpora on Public Art domain, demonstrating the value of this method for lexicon and semantic information extraction from large data.

CQL grammars for lexical and semantic information extraction for Portuguese and Italian

We compiled a set of patterns that denote lexico-semantic relations in Portuguese, created templates where one argument is fixed and the other is replaced by a mask, and then used BERT for predicting the latter. For most relations and measures, when assessed in a test of lexico-semantic analogies, BERT predictions outperformed those of earlier methods computed from static word embeddings, either with a pattern alone or with a combination of patterns. There is still a large margin of progression, but this suggests that BERT can be used as a source of lexico-semantic knowledge in Portuguese.

Drilling Lexico-Semantic Knowledge in Portuguese from BERT

Named Entity Recognition (NER) is the task of identifying textual elements and categorizing them into predefined classes, such as names of people, locations, organizations, and others. Recently, NER systems based on neural networks have obtained state-of-the-art results. One of the main components of this kind of system is the word embeddings representation adopted that can be static (traditional) or contextual trained in language models. This work analyzes the strategy of combining traditional and contextual embeddings to obtain richer representations for Portuguese NER. We adopt a Bidirectional Long ShortTerm Memory (BiLSTM) as a classification model integrated with the Conditional Random Fields (CRF) algorithm. Experiments were carried out in different corpora, and the results obtained demonstrate that the combination strategy of word embeddings models is viable and led to results comparable with state-of-the-art NER systems in Portuguese.

Combining Word Embeddings for Portuguese Named Entity Recognition

Traditional text classification approaches often require a good amount of labeled data, which is difficult to obtain, especially in restricted domains or less widespread languages. This lack of labeled data has led to the rise of low-resource methods, that assume low data availability in natural language processing. Among them, zero-shot learning stands out, which consists of learning a classifier without any previously labeled data. The best results reported with this approach use language models such as Transformers, but fall into two problems: high execution time and inability to handle long texts as input. This paper proposes a new model, ZeroBERTo, which leverages an unsupervised clustering step to obtain a compressed data representation before the classification task. We show that ZeroBERTo has better performance for long inputs and shorter execution time, outperforming XLM-R by about 12% in the F1 score in the FolhaUOL dataset.

ZeroBERTo - Leveraging Zero-Shot Text Classification by Topic Modeling

Our contribution presents the first ever stance and sentiment annotated corpus in Portuguese. Using data from Twitter, we annotated a data set with over 15,000 tweets. In building the corpus, we made both random and supervised searches to maximize balance. Using four annotators, we have classified each tweet on four categories: support, against, neutral or inconclusive. Furthermore, we have annotated each tweet for sentiment: positive, negative, neutral, sarcasm or inconclusive. Our annotators yield strong inter-annotator agreement. In addition, we test a baseline model using a pre-trained BERT model for Portuguese. Results suggest that our data set is in line with its counterparts in other languages.

SS-PT: A Stance and Sentiment data set from Portuguese quoted tweets

This paper presents a targeted syntactic evaluation of Transformer models for Galician-Portuguese. We defined three experiments that allow to explore how these models, trained with a masked language modeling objective, encode syntactic knowledge. To do so, we created a new dataset including test instances of number (subject-verb), gender (subject-predicative adjective), and person (subject-inflected infinitive) agreement. This dataset was used to evaluate monolingual and multilingual BERT models, controlling for various aspects such as the presence of attractors or the distance between the dependent elements. The results show that Transformer models perform competently in many cases, but they are generally confounded by the presence of attractors in long-distance dependencies. Moreover, the different behavior of monolingual models trained with the same corpora reinforces the need for a deep exploration of the network architectures and their learning process.

A Targeted Assessment of the Syntactic Abilities of Transformer Models for Galician-Portuguese

This paper describes the development of the first test suite for the language direction Portuguese-English. Designed for fine-grained linguistic analysis, the test suite comprises 330 test sentences for 66 linguistic phenomena and 14 linguistic categories. Eight different MT systems were compared using quantitative and qualitative methods via the test suite: DeepL, Google Sheets, Google Translator, Microsoft Translator, Reverso, Systran, Yandex and an internally built NMT system
trained over 30 hours on 2,5M sentences. It was found that ambiguity, named entity & terminology and verb valency are the categories where MT systems struggle most. Negation, pronouns, subordination, verb tense/aspect/mood and false friends are the categories where MT systems perform best.

Premium content

Downloads

Next from PROPOR 2022

A transfer learning analysis of political leaning classification in cross-domain content

Similar lecture

Enriching Portuguese Word Embeddings with Visual Information

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES