Home Explore Help
Register Sign In
marx
/
llama-recipes
1
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Browse Source

adding barrier to end of trainer loop

Hamid Shojanazeri 1 year ago
parent
f74d57dc08
commit
442c1ccf7c
1 changed files with 1 additions and 1 deletions
Split View Show Diff Stats
  1. 1 1
      utils/train_utils.py

+ 1 - 1
utils/train_utils.py
View File 

@@ -177,7 +177,7 @@ def train(model, train_dataloader,eval_dataloader, tokenizer, optimizer, lr_sche
 
         results['avg_eval_prep'] = avg_eval_prep
 
         results['avg_eval_loss'] = avg_eval_loss
 
         
-
 
+    dist.barrier()
 
     return results
 
 
 def evaluation(model,train_config, eval_dataloader, local_rank, tokenizer):