Have you ever wondered how chess competitions keep track of their games? Traditionally, players record their moves on paper, then comes a process that involves collecting these papers and manually inputting the data into a computer system by tournament officials. This method, while straightforward, can be quite cumbersome and is error prone. This project - done under the supervision of the Centre for Artificial Intelligence (CAI) - aims to develop an optical character recognition model using a variety of machine learning tools for accurately translating the handwritten chess moves images into text. The provided data set consists of 2671 images of box cropped chess moves together with their correct labels. Together with our lab supervisor, Von Däniken Pius, we have formulated a strategy to address this challenge: First we would find an open source pre-trained model that does some similar optical character recognition tasks, and we would then fine tune it for our specific task by training it further on our data and by constraining its outputs to chess moves dictionary only. This choice was motivated by the fact that creating an OCR model from scratch is really cumbersome as it is really easy to get lost in the math and really hard to choose some hyperparameters.
The training.ipynb notebook has everything from training the models to evaluating them so be careful with which cell you run as some might take a very long time to evaluate. The Data and the game1 Files contains the data set on which the model is trained and evaluated. The second file is a subset of the big data set representing a single game of chess used to simulate a real life example of the code in action.
In order to be able to run the notebook correctly you will first need to download the two model and put them in the same directory of the code, pls do not change the name of the file or you'll have to change some of the code. You can find the model here: https://www.dropbox.com/scl/fi/8xocn0i4zlcof9e5emxq8/best_model_handwritten_lower.zip?rlkey=tk2iv4byxl2oxo7ka9fzqayy6&dl=0 and https://www.dropbox.com/scl/fi/0ailjenb1c311lusfuprz/best_model_stage1_lower.zip?rlkey=nkc5d8qns7ld1q17i0c2p0jx1&dl=0.
Then you can use the training notebook to evaluate the model on your own machine, just be aware of the warning so you don't accidentally run a cell that will train the model. Even if you do it's not bad you can just stop the cell and run the ones below the code will still work.
The requierment.txt is the file generated by anaconda showing all the library used in our environment.
Our report on the project can be found in the report.pdf File. It countains our journey through the project and our results.