Date of Defense

4-20-2021

Date of Graduation

6-2021

Department

Computer Science

First Advisor

Barcley Johnson

Second Advisor

Alvis Fong

Abstract

The goal of this project was to explore modern neural network technology in the application of discerning and generating statements that are ‘reasonable’, in what is known as commonsense reasoning. We built off of the work of Saeedi et al. In their work on the 2020 SemEval task, Commonsense Validation and Explanation (ComVE). SemEval is a workshop that creates a variety of semantic evaluation tasks to examine the state of the art in the practical application of natural language processing. This particular task involved three sections: task A, Validation, in which a program tries to select which of two statements is more sensical; task B, Explanation, in which the program is given an illogical statement and has to choose between three statements to find the one that works as the best explanation as to why the statement is illogical; and task C, generation, in which the program must generate a novel explanation as to why a given statement is illogical.

The existing project that we had taken on to improve had already had considerable success with tasks A and B using a relatively straightforward application of the huggingface transformer library, a python library that acts as a useful wrapper around a wide variety of pre-trained open source neural networks of the type called Transformers [5]. They also had, on paper, some success with task C, however this was entirely due to weaknesses in the assessment mechanisms given by SemEval, for reasons we will examine later. In this paper we will explain some background on transformer technology, we explain the work that was already done on both the first two and the latter tasks, and we will explain our attempts to get improved performance, particularly at task C. While we succeeded in some ways in improving the results of the generation portion of the task, we struggled to increase the actual score we received from the SemEval scoring system. This is likely due to both the difficulty of the task itself and the aforementioned problems with the scoring system.

Comments

Coauthored with: Anson Zeeping Lim, Nathan Brackenbury, and ThongSheng Ang

Access Setting

Honors Thesis-Open Access

Download

Share

COinS