Parameter Efficient Reinforcement Learning from Human Feedback

Hakim Sidahmed, Samrat Phatale, Alex Hutcheson et al.

2024 · arXiv (Cornell University) · 1 citations

While Reinforcement Learning from Human Feedback (RLHF) effectively aligns pretrained Large Language and Vision-Language Models (LLMs, and VLMs) with human preferences, its computational cost and complexity hamper its wider adoption. To alleviate some of the computational burden of fine-tuning, parameter efficient methods, like LoRA were introduced. In this work, we empirically evaluate the setup of Parameter Efficient Reinforcement Learning from Human Feedback (PE-RLHF) that leverages LoRA fine-tuning for Reward Modeling, and Reinforcement Learning. We benchmark the PE-RLHF setup on six dive…

Read the paper →

Explore this paper's citation graph on Constellation.