LLM Essay Evaluator

A LLM tool for evaluating essays using local models.

The Essay Evaluator is a tool designed to automate the evaluation of essays, offering detailed feedback on aspects like grammar, structure, and content relevance. This project leverages the power of local models to streamline the essay grading process using sentiment analysis prompted with DSPY framework, making it both faster and more objective.

This was a self learning project. I focused on implementing the essay evaluation logic using sentiment anaylsis, as well as the user interface for seamless interaction. I wrote a guide with an introduction to DSPy and the project on medium.

Project Objectives

  • Automate Essay Evaluation: To reduce the time and effort required to grade essays by leveraging machine learning.
  • Provide Detailed Feedback: To offer comprehensive feedback on various aspects of the essay, such as grammar, structure, and content quality.
  • Enhance Objectivity: To minimize subjective biases in the grading process.

Components Used

  • Python: For developing the application and implementing machine learning models.
  • DSPy: Integrate prompt workflows into the application.
  • Ollama: Handle the management and deployment of large language models.
  • Streamlit: Create the interactive user interface.
  • WSL: Enabled the execution of Linux-based development environments on Windows.

Project Workflow

  1. Clone the repository: git clone https://github.com/Gayanukaa/Essay-Evaluator-LLM.git
  2. Navigate to the project directory: cd Essay-Evaluator-LLM
  3. Set up the environment with dependencies: conda env create -f environment.yml -n dspy-dev
  4. Activate the environment required for the application: conda activate dspy-dev
  5. Run the application: streamlit run app.py

Scenarios Demonstrated:

Multi-Criteria Evaluation
Grade and Evaluation Downloadable

Conclusion

The Essay Evaluator using DSPy represents a significant step forward in automating and enhancing the essay grading process. By leveraging state-of-the-art language models and sentiment analysis, it provides comprehensive, objective, and instant feedback, making it a feasible tool for educators and students alike.

References

For more details, please visit the Essay Evaluator GitHub Repository.