Ao Shen1, Zhiquan Lai1,*, Dongsheng Li1,*, Xiaoyu Hu2
CMC-Computers, Materials & Continua, Vol.82, No.1, pp. 307-325, 2025, DOI:10.32604/cmc.2024.057491
- 03 January 2025
Abstract Large-scale Language Models (LLMs) have achieved significant breakthroughs in Natural Language Processing (NLP), driven by the pre-training and fine-tuning paradigm. While this approach allows models to specialize in specific tasks with reduced training costs, the substantial memory requirements during fine-tuning present a barrier to broader deployment. Parameter-Efficient Fine-Tuning (PEFT) techniques, such as Low-Rank Adaptation (LoRA), and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency. Among these, QLoRA, which combines PEFT and quantization, has demonstrated notable success in reducing memory footprints during fine-tuning, prompting the development… More >
Login
Register
