flash-attention/training/configs/optimizer/adamw-apex-distributed.yaml

4 lines
123 B
YAML
Raw Normal View History

2022-11-29 09:31:19 +08:00
# @package train.optimizer
_target_: apex.contrib.optimizers.distributed_fused_adam.DistributedFusedAdam
adam_w_mode: True