v2.0.0: Sharding refactoring

This release marks a huge improvement in dealing with replicated parameters and automatic parameters synchronization. The Sharded class has been made private and it's functions have been hidden by the modules' interfaces, making it easier to deal with.

Backwards compatibility with older version has been maintained with warnings notifying about changes in interfaces and deprecations.

What's Changed

• ZeRO-3 refactoring (sharding) by @BlackSamorez in #106

Full Changelog: v1.3.2...v2.0.0

v1.3.2: Model input processing forwarding

What's Changed

• Forwarding _prepare_model_inputs by @BlackSamorez in #113

Full Changelog: v1.3.1...v1.3.2

v1.3.1: LLaMA-2 config fix

What's Changed

• Testing interfaces (soon to be refactored) by @BlackSamorez in #109
• Fix get_llama_config adding model attribute error. by @tonywang16 in #108
• find_predefined_tensor_parallel_config try-except fix by @BlackSamorez in #110

New Contributors

• @tonywang16 made their first contribution in #108

Full Changelog: v1.3.0...v1.3.1

v1.3.0: thread based communications for distributed

What's Changed

• explicitly choose whether or not to use torch.distribute by @tomoki0924 in #100
• Gpt2 fix by @BlackSamorez in #103

New Contributors

• @tomoki0924 made their first contribution in #100

Full Changelog: v1.2.9...v1.3.0

v1.2.9: LLaMA 2

What's Changed

• readme fixes by @BlackSamorez in #93
• LLaMA-2 by @BlackSamorez in #101
• Version bump by @BlackSamorez in #102

Full Changelog: v1.2.8...v1.2.9

v1.2.8: falcon hotfix

What's Changed

• Falcon lm_head split hotfix by @BlackSamorez in #92

Full Changelog: v1.2.7...v1.2.8

v1.2.7: falcon

What's Changed

• Falcon predefined config by @BlackSamorez in #91

Full Changelog: v1.2.6...v1.2.7

v1.2.6: tp.Sharded dispatch

What's Changed

• Fixed dispatch of tp.Sharded models by @BlackSamorez in #90

Full Changelog: v1.2.5...v1.2.6

v1.2.5: Removing PEFT from library dependencies

What's Changed

• tp.convert_state_dict readme example fix by @BlackSamorez in #78
• Actually using SplitInsideChunks for gpt2 by @BlackSamorez in #82
• Removing PEFT from dependencies. Replacing with runtime checks by @BlackSamorez in #87

Full Changelog: v1.2.4...v1.2.5

v1.2.4: Tied weights fixes

What's Changed

• State dict fixes for tied weights by @BlackSamorez in #73

Full Changelog: v1.2.3...v1.2.4