Constructing DBRX-class Customized LLMs with Mosaic AI Coaching
We lately launched DBRX: an open, state-of-the-art, general-purpose LLM. DBRX was skilled, fine-tuned, and evaluated utilizing Mosaic AI Coaching, scaling coaching to 3072 NVIDIA H100s and processing greater than 12 trillion tokens within the course of. Coaching LLMs, and particularly MoE fashions reminiscent of DBRX, is tough. It requires overcoming many infrastructure, efficiency, and scientific…
