瀏覽代碼

change bf16 default to false

Hamid Shojanazeri 1 年之前
父節點
當前提交
a977145a9b
共有 1 個文件被更改,包括 1 次插入1 次删除
  1. 1 1
      configs/fsdp.py

+ 1 - 1
configs/fsdp.py

@@ -13,7 +13,7 @@ class fsdp_config:
     sharding_strategy: ShardingStrategy = ShardingStrategy.FULL_SHARD
     checkpoint_type: StateDictType = StateDictType.SHARDED_STATE_DICT  # alternatively can use SHARDED_STATE_DICT save one file per rank, and can resize the world-size.
     fsdp_activation_checkpointing: bool=True
-    pure_bf16: bool = True
+    pure_bf16: bool = False
     optimizer: str= "AdamW"