A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training von Singh, Siddharth, Olatunji Ruwase, Ammar Ahmad Awan, Rajbhandari, Samyam, He, Yuxiong, Bhatele, Abhinav

Volltext

Optimizing CNNs on Multicores for Scalability, Performance and Goodput von Rajbhandari, Samyam, He, Yuxiong, Ruwase, Olatunji, Carbin, Michael, Chilimbi, Trishul

Volltext

SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks von Yazdani Aminabadi, Reza, Ruwase, Olatunji, Zhang, Minjia, He, Yuxiong, Arnau, Jose-Maria, Gonazalez, Antonio

Volltext

Efficient Deep Neural Network Serving: Fast and Furious von Feng Yan, Yuxiong He, Ruwase, Olatunji, Smirni, Evgenia

Volltext

Flexible Hardware Acceleration for Instruction-Grain Program Monitoring von Chen, Shimin, Kozuch, Michael, Strigkos, Theodoros, Falsafi, Babak, Gibbons, Phillip B., Mowry, Todd C., Ramachandran, Vijaya, Ruwase, Olatunji, Ryan, Michael, Vlachos, Evangelos

Volltext

Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping von Wang, Guanhua, Zhang, Chengming, Shen, Zheyu, Li, Ang, Ruwase, Olatunji

Volltext bestellen

FastPersist: Accelerating Model Checkpointing in Deep Learning von Wang, Guanhua, Ruwase, Olatunji, Xie, Bing, He, Yuxiong

Volltext bestellen

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer von Yao, Jinghan, Jacobs, Sam Ade, Tanaka, Masahiro, Ruwase, Olatunji, Shafi, Aamir, Subramoni, Hari, Panda, Dhabaleswar K

Volltext bestellen

Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training von Lian, Xinyu, Jacobs, Sam Ade, Kurilenko, Lev, Tanaka, Masahiro, Bekman, Stas, Ruwase, Olatunji, Zhang, Minjia

Volltext bestellen

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding von Zhang, Zhenyu, Chen, Runjin, Liu, Shiwei, Yao, Zhewei, Ruwase, Olatunji, Chen, Beidi, Wu, Xiaoxia, Wang, Zhangyang

Volltext bestellen

ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning von Rajbhandari, Samyam, Ruwase, Olatunji, Rasley, Jeff, Smith, Shaden, He, Yuxiong

Volltext bestellen

Flexible Hardware Acceleration for Instruction-Grain Lifeguards von Chen, S., Kozuch, M., Gibbons, P.B., Ryan, M., Strigkos, T., Mowry, T.C., Ruwase, O., Vlachos, E., Falsafi, B., Ramachandran, V.

Volltext

DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention von Yao, Zhewei, Wu, Xiaoxia, Li, Conglong, Zhang, Minjia, Qin, Heyang, Ruwase, Olatunji, Awan, Ammar Ahmad, Rajbhandari, Samyam, He, Yuxiong

Volltext bestellen

LOOP CODE PROCESSOR OPTIMIZATIONS von RUWASE, Olatunji, SESHADRI, Vivek, CHILIMBI, Trishul, A

Volltext bestellen

ZeRO++: Extremely Efficient Collective Communication for Giant Model Training von Wang, Guanhua, Qin, Heyang, Jacobs, Sam Ade, Holmes, Connor, Rajbhandari, Samyam, Ruwase, Olatunji, Yan, Feng, Yang, Lei, He, Yuxiong

Volltext bestellen

Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping von Wang, Guanhua, Zhang, Chengming, Shen, Zheyu, Ang, Li, Olatunji Ruwase

Volltext

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design von Xia, Haojun, Zheng, Zhen, Wu, Xiaoxia, Chen, Shiyang, Yao, Zhewei, Youn, Stephen, Bakhtiari, Arash, Wyatt, Michael, Zhuang, Donglin, Zhou, Zhongzhu, Ruwase, Olatunji, He, Yuxiong, Song, Shuaiwen Leon

Volltext bestellen

ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks von Wu, Xiaoxia, Xia, Haojun, Youn, Stephen, Zheng, Zhen, Chen, Shiyang, Bakhtiari, Arash, Wyatt, Michael, Aminabadi, Reza Yazdani, He, Yuxiong, Ruwase, Olatunji, Song, Leon, Yao, Zhewei

Volltext bestellen

FastPersist: Accelerating Model Checkpointing in Deep Learning von Wang, Guanhua, Olatunji Ruwase, Xie, Bing, He, Yuxiong

Volltext

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models von Rajbhandari, Samyam, Rasley, Jeff, Ruwase, Olatunji, He, Yuxiong

Volltext bestellen