DeepSpeed: Accelerating large-scale model inference and training via system optimizations and compression
Last month, the DeepSpeed Team announced ZeRO-Infinity, a step forward in training models with tens of trillions of parameters. In addition to creating optimizations for scale, our team strives to introduce features