deepset Cloud, a large language model (LLM) platform, is providing insights into the precision and fidelity of responses from LLM generative AI through the Groundedness Observability Dashboard.
With the 01/2024 release, the Groundedness Observability Dashboard displays trend data for how well generative AI responses are grounded in the source documents. This feature provides a quantifiable score to measure the factuality of an LLM's output. The results serve as a guide for developers in modifying their RAG setup, fine tuning models, and altering prompts to improve accuracy and reliability of generated responses. Simplified insights into what works enables users to track how well the model can use the provided data to answer queries in a reliable manner. When tracked over time, this allows for comparisons with other widely-available LLM platforms.
deepset Cloud’s Source Reference Prediction generative response annotation is also now generally available. Response Annotation adds academic-style citations to the LLM-generated answer. Those citations reference the respective document on which a statement is based. Users can then review the source material in order to fact-check generated answers or gain a better understanding of the source data in its original context.
The combination of deepset Cloud’s Groundedness Dashboard and Source Reference Prediction gives organizations greater confidence in the quality of the responses in their LLM applications, and provides visibility when an application’s accuracy does not meet requirements.
Groundedness isn't just a useful metric for measuring the faithfulness of your LLM-generated answers to a knowledge base. It can also be used as a proxy to identify the ideal hyperparameters for your retrieval step. Optimizing the number of documents embedded in the query can reduce your LLM costs by a significant factor.
These new features emphasize deepset’s commitment to building a robust trust layer within generative AI applications. The new features effectively detect hallucinations and provide benchmarking tools, allowing users to make informed decisions about the reliability of their AI models.
These features of deepset Cloud are in General Availability.
The Latest
In a fast-paced industry where customer service is a priority, the opportunity to use AI to personalize products and services, revolutionize delivery channels, and effectively manage peaks in demand such as Black Friday and Cyber Monday are vast. By leveraging AI to streamline demand forecasting, optimize inventory, personalize customer interactions, and adjust pricing, retailers can have a better handle on these stress points, and deliver a seamless digital experience ...
Broad proliferation of cloud infrastructure combined with continued support for remote workers is driving increased complexity and visibility challenges for network operations teams, according to new research conducted by Dimensional Research and sponsored by Broadcom ...
New research from ServiceNow and ThoughtLab reveals that less than 30% of banks feel their transformation efforts are meeting evolving customer digital needs. Additionally, 52% say they must revamp their strategy to counter competition from outside the sector. Adapting to these challenges isn't just about staying competitive — it's about staying in business ...
Leaders in the financial services sector are bullish on AI, with 95% of business and IT decision makers saying that AI is a top C-Suite priority, and 96% of respondents believing it provides their business a competitive advantage, according to Riverbed's Global AI and Digital Experience Survey ...
SLOs have long been a staple for DevOps teams to monitor the health of their applications and infrastructure ... Now, as digital trends have shifted, more and more teams are looking to adapt this model for the mobile environment. This, however, is not without its challenges ...
Modernizing IT infrastructure has become essential for organizations striving to remain competitive. This modernization extends beyond merely upgrading hardware or software; it involves strategically leveraging new technologies like AI and cloud computing to enhance operational efficiency, increase data accessibility, and improve the end-user experience ...
AI sure grew fast in popularity, but are AI apps any good? ... If companies are going to keep integrating AI applications into their tech stack at the rate they are, then they need to be aware of AI's limitations. More importantly, they need to evolve their testing regiment ...
If you were lucky, you found out about the massive CrowdStrike/Microsoft outage last July by reading about it over coffee. Those less fortunate were awoken hours earlier by frantic calls from work ... Whether you were directly affected or not, there's an important lesson: all organizations should be conducting in-depth reviews of testing and change management ...
In MEAN TIME TO INSIGHT Episode 11, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses Secure Access Service Edge (SASE) ...
On average, only 48% of digital initiatives enterprise-wide meet or exceed their business outcome targets according to Gartner's annual global survey of CIOs and technology executives ...
