Descriptive Evaluation

This repository contains all the code used while developing the descriptive answer evaluator at Ramaiah Institute of Technology. Done as a part of the project 'Lightweight Container-based Framework in Cloud Environment for Automated Online Assessment for Computer Science Courses' funded by VGST, Govt of Karnataka

Contributors

• Isha Gupta
• Mrigank Khandelwal
• Sreya Salil

About Each Project

BERT + Cosine Similarity

---

Uses BERT and Cosine Similairty to find the semantic similarity between two sentences

GloVe

---

Uses GloVe embeddings and Manhattan LSTM to determine semantic similarity between the answer key and student answers.

Files Required

• train_main.csv
• GloVe Embeddings

Hewlett Dataset

---

Uses GloVe, Poincaré and Word2Vec embeddings along with Manhattan LSTM to find semantic similairty between short answers from the Hewlett Dataset

Additional Files Required:

• GloVe Embeddings
• Word2Vec Embeddings
• Hewlett Dataset

Notes

• The Poincaré embeddings are automatically generated from the dataset in the notebook
• The dataset must be transformed to a certain format before running the codes. Use this notebook.

Extracting Keywords and Keyword Matching

---

Takes the OOP dataset and extracts keywords from both the answer key and student answers. Keyword matching techniques are used to check which matching technique accurately finds the similairty between the keywords. The keyword extraction techniques are also evaluated using mean average precision.

Datasets used

• train_main.csv
• Define OOPs.csv

Knowledge Based Measures

---

Uses Knowledge based measures like Lin, Jiang-Conrath Resnik and WuPalmer to determine semantic similarity between the answer key and student answers.

Dataset used

• train_main.csv

Poincaré

---

Uses Poincaré embeddings and Manhattan LSTM to find semantic similairty between the answer key and student answers.

Dataset used

• train_main.csv

Notes

• The Poincaré embeddings are automatically generated from the dataset in the notebook

Semantic Similarity Functions

---

Uses tf-idf vectors, Pairwise similarity, word2vec embeddings, Doc2vec model and BERT to find semantic similairty between ta list of documents.

Files Required

• Word2Vec Embeddings

Descriptive Answer Evaluator

---

Uses RAKE keyword Extractor, Keyphrase extracter using TF-IDF and BERT + Cosine similairty to evaluate student answers.

Files Required

• Define OOPs.csv
• Word2Vec Embeddings

Word2Vec

---

Uses Word2Vec embeddings and Manhattan LSTM to determine semantic similarity between the answer key and student answers.

Files Required

• train_main.csv
• Word2Vec Embeddings

fastText

---

Uses fastText embeddings and Manhattan LSTM to determine semantic similarity between the answer key and student answers.

Dataset used

• train_main.csv

Notes

• The fastText embeddings are downloaded from GitHub in the notebook

Node2Vec

---

Generates a graph from the corpus of answers. Then generates embeddings by performing a random walk on the graph. These embeddings with Manhattan LSTM are used to determine semantic similarity between the answer key and student answers.

Dataset used

• train_main.csv