AutoAlign - Automated Alignment Toolkit for LLMs

Project Overview

AutoAlign is an open-source toolkit designed to automate the alignment of Large Language Models (LLMs) with human intentions and values while minimizing manual intervention.

My Contributions

I was responsible for adapting and optimizing SFT (Supervised Fine-Tuning) and DPO (Direct Preference Optimization) algorithms for the Megatron framework, enabling efficient large-scale distributed training.

Key Features

• Unified Framework: Integrates mainstream automated alignment algorithms through a consistent interface
• Accessible Workflow: Supports one-click execution for prompt synthesis, automatic alignment signal construction, and iterative model training
• Modular Components: Facilitates easy reproduction of existing results and development of novel approaches
• Efficient Implementation: Includes highly efficient inference and training, as well as low-resource training support

Publication

This work was published at ACL Demo 2025.