 Matthias Reso
|
8620ab8ac2
Fix invalid labels for context in custom dataset/oasst1
|
vor 1 Jahr |
|
Matthias Reso
|
52c417b7d5
Merge branch 'fix/invalidate_label_for_chat' into feature/length_based_batch_sampling
|
vor 1 Jahr |
|
Matthias Reso
|
eafea7b366
Invalidate labels in dialog dataset to disable loss
|
vor 1 Jahr |
|
Matthias Reso
|
10f9367e56
fix missing labels in datasets
|
vor 1 Jahr |
|
Matthias Reso
|
ca41c1c697
Adjust tests to len based batch sampling
|
vor 1 Jahr |
|
Matthias Reso
|
ec00a2f722
Fix batching error
|
vor 1 Jahr |
|
Matthias Reso
|
dc507b4e55
Finish implementation oasst preprocessing for of custom dataset
|
vor 1 Jahr |
|
Matthias Reso
|
32cf7ad459
[WIP] Preprocess oasst dataset
|
vor 1 Jahr |
|
Matthias Reso
|
ba1c789c51
Rename custom dataset function to comply with documentation
|
vor 1 Jahr |
|
Matthias Reso
|
33c7738dcc
Added mechanism to supply a custom dataset without changing source files
|
vor 1 Jahr |
