CUDA OUT OF MEMORY

[Wiselnn570]

When running DPO, I found that the model crashes due to memory overflow if the supported image frames exceed 60, even though the training epoch is set to 1, and the training is conducted on 8 A100 GPUs.

[jzhang38]

Yes, it is expected. During DPO we only use deepspeed for parallelism and do not train with that many frames.