[ad_1]
We’re excited to introduce a number of highly effective new capabilities to Mosaic AI Gateway, designed to assist our clients speed up their AI initiatives with even larger simplicity, safety, and governance.
As enterprises race to implement AI options, managing safety, compliance, and prices has turn out to be more and more difficult. That’s why we launched Mosaic AI Gateway final 12 months, now utilized by many organizations, to handle AI site visitors for a variety of fashions, together with OpenAI GPT, Anthropic Claude, and Meta Llama fashions.
At this time’s replace introduces superior options for utilization monitoring, payload logging, and guardrails, enabling enterprises to use safety and governance to any AI mannequin inside the Databricks Knowledge Intelligence Platform. With this launch, the Mosaic AI Gateway now gives production-grade safety and governance, even for essentially the most delicate knowledge and site visitors.
“Mosaic AI Gateway is offering us with a safe strategy to eat AI fashions and join them to our proprietary knowledge. This allows us to construct safe, compliant, and context-aware AI programs, bettering our productiveness and serving to us fulfill our mission to ship superior healthcare providers to all.”
— Kapil Ashar, Vice President, Enterprise Knowledge and Medical Platform, Accolade
What’s Mosaic AI Gateway?
Many enterprises combine and match a number of AI fashions from totally different suppliers to construct compound AI programs (e.g., RAG, multi-agent architectures) that obtain the standard wanted to deploy GenAI functions into manufacturing. Nonetheless, as enterprises combine a various array of open and proprietary fashions, they encounter challenges with operational inefficiencies, price overruns, and potential safety dangers.
Mosaic AI Gateway addresses these challenges by offering a unified service to entry, handle, and safe AI site visitors. It permits enterprise directors to implement guardrails and monitor AI utilization whereas providing builders a easy interface to shortly experiment, mix, and safely deploy functions into manufacturing. Corporations like OMV and Edmunds, which have adopted Mosaic AI Gateway, have been capable of speed up their AI initiatives extra successfully than their friends, all whereas sustaining compliance, safety, and operational effectivity.
What are you able to do with Mosaic AI Gateway?
Securely entry any AI fashions
Mosaic AI Gateway simplifies entry to any LLM via a single interface (API, SDK, SQL), considerably decreasing each improvement time and integration prices. You’ll be able to simply change between proprietary and open fashions with out adjustments to the consumer app. What units Mosaic AI Gateway aside is its help for all AI property—conventional fashions, GenAI fashions, chains, and brokers—eliminating the necessity for a number of programs. It additionally permits routing and site visitors splitting between fashions for A/B testing or distributing workloads throughout suppliers to deal with excessive demand.
“With Mosaic AI Gateway, we have been capable of confidently experiment with quite a lot of open and proprietary AI fashions, accelerating innovation whereas guaranteeing regulatory compliance. This allowed us to combine a number of GenAI apps, decreasing data search time and bettering data-driven decision-making.”
— Harisyam Manda, Senior Knowledge Scientist at OMV
Monitor and Debug Manufacturing AI Site visitors
Mosaic AI Gateway now captures utilization and payload knowledge for all site visitors flowing out and in of endpoints to Unity Catalog Delta Tables. We’re introducing two key tables:
- Endpoint Utilization Desk: This technique desk logs each request throughout all serving endpoints within the account, together with requester particulars, utilization statistics, and customized metadata. This knowledge helps optimize spending and maximize ROI—for instance, by setting fee limits on experimental endpoints or working with suppliers to extend quotas for manufacturing endpoints.
- Inference Desk: This desk repeatedly captures uncooked inputs, outputs, HTTP standing codes, and latency for every serving endpoint. You should use this knowledge to watch AI app high quality, debug points, and even as a coaching corpus to fine-tune AI fashions.
The very best half? All knowledge is captured in Unity Catalog, making it straightforward to securely share, search, visualize, and analyze utilizing acquainted knowledge instruments. For instance, you’ll be able to mix these tables with different tables, corresponding to label knowledge or enterprise metrics, to carry out customized evaluation or construct dashboards tailor-made to your enterprise wants.
“Mosaic AI Gateway permits us to securely eat any LLMs, be it OpenAI or different fashions hosted on Databricks, whereas guaranteeing LLM site visitors is correctly ruled and tracked. This has democratized GenAI, permitting us to deploy new use instances like a customer support bot that has improved buyer satisfaction and gross sales.”
— Eduardo Fernández Carrión, Lead Knowledge Scientist & AI Engineer at Santalucia Seguros
Repeatedly Safeguard Customers and Functions
Mosaic AI Gateway contains complete guardrails to safe site visitors to any mannequin API, implementing security insurance policies and defending delicate data in real-time. These guardrails embrace:
- Security Filtering: Filters dangerous content material corresponding to hate speech, insults, sexual content material, violence, misconduct, and different classes.
- PII Filters: Detects and blocks requests containing delicate content material corresponding to personally identifiable data (PII) in consumer inputs.
- Key phrase Filters: Blocks undesirable subjects in your functions to make sure protected, related interactions aligned with your enterprise and insurance policies.
- Matter Filters: Retains your software targeted on its skilled scope by avoiding responses to unrelated or dangerous subjects, minimizing legal responsibility.
These guardrails will be set on the endpoint or request ranges to suit particular use instances and insurance policies. All knowledge is logged in Inference Tables, which might then be analyzed with Lakehouse Monitoring to trace mannequin security over time.
“Guardrails helps us stop unsafe content material from reaching our finish consumer. With payload logging, we will additionally hint guardrails to trace efficiency of the applying.”
— Ryan Jockers, Assistant Director at North Dakota College System
Join AI with Your Knowledge Effortlessly
Mosaic AI Gateway is constructed on Databricks Knowledge Intelligence Platform, enabling enterprises to simply join LLMs to their knowledge utilizing strategies like RAG, agent workflows, and fine-tuning, serving to rework common intelligence into actionable knowledge intelligence. That is already in motion—many shoppers are utilizing Mosaic AI’s Vector Search with exterior fashions to create embeddings or leveraging agent frameworks and analysis instruments to construct brokers with exterior fashions.
“Databricks Mannequin Serving is accelerating our AI-driven initiatives by making it straightforward to securely entry and handle a number of SaaS and open fashions, together with these hosted on or exterior Databricks. Its centralized strategy simplifies safety and price administration, permitting our knowledge groups to focus extra on innovation and fewer on administrative overhead.”
— Greg Rokita, AVP, Know-how at Edmunds.com
Getting Began with Mosaic AI Gateway
Mosaic AI Gateway’s new monitoring and guardrails at the moment are out there for all mannequin serving workspaces. Allow Mosaic AI Gateway on new and current exterior fashions with just some clicks. Assist for added endpoints is coming quickly.
[ad_2]