Despite significant advancements in Large Language Models (LLMs), the challenge of balancing performance and cost persists in training and inference. Many high-performing LLMs remain inaccessible to academics and open-source developers due to their prohibitive costs.
To tackle this issue, a research titled 'OLMoE: Open Mixture-of-Experts Language Models' was published, presenting a fully open Mixture-of-Experts language model designed to offer state-of-the-art performance among similarly-sized models.
Join the author of the paper, Niklas Muennighoff, as they present OLMoE and share key insights from their research, including:
Niklas Muennighoff
Niklas Muennighoff is a PhD student at Stanford. His research focuses on improving large language models across pretraining, instruction finetuning, and retrieval via work such as OLMo, BLOOM, and StarCoder. He did his Bachelor's at Peking University.
©2024 SSI Club. All Rights Reserved