Sharing Knowledge about Foundation Models
Homepage: https://xuefuzhao.github.io/
Twitter: https://twitter.com/XueFz
Email: xuefuzhao at outlook.com, f.xue at u.nus.edu
Dec 2024 | From PhD to Google DeepMind: Lessons and Gratitude on My Journey
Mar 2024 | Take a Closer Look at the MoE LLM Routing
Sep 2023 | Encoder-Decoder is actually not that different from Decoder-only
Aug 2023 | OpenMoE v0.2 Release
May 2023 | What is the relationship between transformer scaling and training objective？
Mar 27, 2024
MoE is so widely discussed and used now, but what are the experts specializing at? In this blog, let’s take a closer look at the MoE specializing and routing to understand MoE LLMs better.
Sep 27, 2023