lukas-blecher/LaTeX-OCR

The goal of this project is to create a learning based system that takes an image of a math formula and returns corresponding LaTeX code.

To run the model you need Python 3.7+

If you don't have PyTorch installed. Follow their instructions here.

Model checkpoints will be downloaded automatically.

There are three ways to get a prediction from an image.

The model works best with images of smaller resolution. That's why I added a preprocessing step where another neural network predicts the optimal resolution of the input image. This model will automatically resize the custom image to best resemble the training data and thus increase performance of images found in the wild. Still it's not perfect and might not be able to handle huge images optimally, so don't zoom in all the way before taking a picture.

Always double check the result carefully. You can try to redo the prediction with an other resolution if the answer was wrong.

Want to use the package?

I'm trying to compile a documentation right now.

Install a couple of dependencies pip install "pix2tex[train]".

To use your own tokenizer pass it via --tokenizer (See below).

You can find my generated training data on the Google Drive as well (formulae.zip - images, math.txt - labels). Repeat the step for the validation and test data. All use the same label text file.

If you want to use your own data you might be interested in creating your own tokenizer with