Inquiry About Training Settings

[pointcloud1999]

Hello,

During my training, I observed that in the ddp_mos_coarse_stage.yml file, the learning rate (lr) is set to 0.002, whereas in the mos_coarse_stage.yml file, it is set to 0.008. Could you please explain the reason for these different settings? Additionally, I noticed that the initial learning rate mentioned in the paper is 0.01. Which learning rate should I follow for my experiments? What are the differences between these two yml files?

Thank you.

[Terminal-K]

Hi @pointcloud1999,
lr is different in ddp_mos_coarse_stage.yml and mos_coarse_stage.yml because we keep the lr positively correlated with the batch size, meaning the learning rate for per batch size is fixed.
Besides, the inconsistency of lr in the files and the lr reported in the paper may be due to a small mistake, but it is unlikely to have a significant impact on performance in my opinion. I suggest conducting experiments based on the lr in the code first. However, if you have extra spare time, you can also perform comparative experiments to test its impact on performance [ I do not have a suitable experimental environment currently :( ]. Thanks for your keen observation!