Installation

It’s recommended to install HintEval in a virtual environment using Python 3.11.9. If you’re not familiar with Python virtual environments, check out this user guide. Alternatively, you can create a new environment using Conda.

Set up the virtual environment

First, create and activate a virtual environment with Python 3.11.9:

conda create -n hinteval_env python=3.11.9 --no-default-packages
conda activate hinteval_env

Install PyTorch 2.4.0

You’ll need PyTorch 2.4.0 for HintEval. Refer to the PyTorch installation page for platform-specific installation commands. If you have access to GPUs, it’s recommended to install the CUDA version of PyTorch, as many of the evaluation metrics are optimized for GPU use.

Install HintEval

Once PyTorch 2.4.0 is installed, you can install HintEval via pip:

pip install hinteval

For the latest features, you can install the most recent version from the main branch:

pip install git+https://github.com/my-unknown-account/HintEval

Now that HintEval is installed, you can create a synthetic dataset using your own questions and answers. If you already have a dataset, learn how to evaluate it with HintEval.