Sharing Knowledge about Foundation Models
Homepage: https://xuefuzhao.github.io/
Twitter: https://twitter.com/XueFz
Email: xuefuzhao at outlook.com, f.xue at u.nus.edu
Mar 2024 | Take a Closer Look at the MoE LLM Routing
Nov 2024 | What I Learned in My PhD
Sep 2023 | Encoder-Decoder is actually not that different from Decoder-only
Aug 2023 | OpenMoE v0.2 Release
May 2023 | What is the relationship between transformer scaling and training objective？
Mar 27, 2024
MoE is so widely discussed and used now, but what are the experts specializing at? In this blog, let’s take a closer look at the MoE specializing and routing to understand MoE LLMs better.
Sep 27, 2023