LLM systems mini-lectures
Date:
Original Bilibili lectures (2023) on LLM systems. A later compilation of the public course is 【大模型公开课】第一期合集; course code: mindspore-courses/step_into_chatgpt.
- Data parallelism (7 min)
- Model parallelism (14 min)
- Pipeline parallelism (10 min)
- Activation checkpointing / rematerialization (6 min)
- Optimizer parallelism / ZeRO (10 min)
- LLMs in NLP systems (7 min)
- Mixed precision (13 min)
- CodeGeeX (63 min)
- Zidong Taichu multimodal model (26 min)
- Instruction tuning (44 min)
- Chain-of-thought / emergent abilities (14 min)
Series on Bilibili: 大模型技术.
中文概要
2023 年在 B 站发布的大模型系统系列短讲，覆盖数据/模型/流水线并行、重计算、ZeRO、混合精度、指令微调、思维链等。完整公开课合集见 【大模型公开课】第一期合集，代码在 step_into_chatgpt。