DPO vs RLHF: which should you use to align a model?
Start with DPO: same preference labels, no reward model or sampling loop. Switch to RLHF with PPO when you need online exploration. Full cost math.
rlhf
1 post with the tag “rlhf”
Start with DPO: same preference labels, no reward model or sampling loop. Switch to RLHF with PPO when you need online exploration. Full cost math.