Mosaic Research by Databricks: Revolutionizing the AI Landscape
Mosaic Research, a part of Databricks, is at the forefront of artificial intelligence innovation. This research division is dedicated to developing cutting-edge AI technologies that have a profound impact on various industries.
One of the notable achievements of Mosaic Research is the development of Shutterstock ImageAI. This text-to-image diffusion model is trained exclusively on Shutterstock's proprietary image repository, generating photorealistic images based on trusted data. It showcases the power of combining advanced AI capabilities with a rich source of visual information.
Another significant contribution is DBRX, an open source, commercially usable LLM. With its sparse mixture-of-expert architecture, DBRX offers high quality and efficiency, fitting extraordinary capabilities into just 36B active parameters.
In addition, Mosaic Research is involved in various other projects. For instance, Mosaic Diffusion is a generative model that turns text descriptions into images with high efficiency. Composer is an open source deep-learning training library optimized for scalability and usability.
The research team at Mosaic Research is not only focused on developing individual technologies but also on exploring the potential of continued pre-training (CPT) to customize large language models to specific domains. This approach requires attention to key hyperparameters and simple weight averaging to mitigate forgetting.
Overall, Mosaic Research by Databricks is driving the advancement of AI, providing solutions that have the potential to transform businesses and industries.
