ms-swift

ms-swift is a framework for large language models and multi-mode models covering education, inference, evaluation and distribution processes. It supports PEFT and full-parameter fine-tuning methods, while Megatron is equipped with parallelism techniques and various reinforcement learning algorithms. The structure, which can work on a large number of hardware architectures, allows for large-scale model training with memory optimization and quantization technologies.