2024 Sessions

Explore
Dreamforce 2024
sessions.

View 1,500+ sessions by role, topic, or industry to see an example of what's to come at Dreamforce 2025.

You are currently viewing sessions from Dreamforce 2024.
Back to Session CatalogBack to Session Catalog
Unified Evaluation, Testing, and Monitoring for Better AI
Learn how to streamline evaluation and monitoring of LLMs, prompts, RAG, agent plans and actions using CRM-specific benchmarks and a unified evaluation and testing framework.
Important: This session is first come, first seated. You do not have a reserved seat.

, Director, Product Management, Salesforce

, Senior Director, Product Management, Salesforce

, Principal Product Designer, Salesforce

Role: Administrator, Product Management
Industry: Technology
Product: Einstein, Agentforce
Topic: Artificial Intelligence
Session Level: Intermediate
Session Type: Theater
  • Thursday, Sep 1910:30 AM - 10:50 AM PDT
    (5:30 PM - 5:50 PM GMT)
    Moscone West, L2, Einstein Theater
Session Summary by Einstein: The recent Salesforce conference session delved into the intricacies of AI testing and monitoring, emphasizing the challenges of ensuring quality and trust in AI systems, particularly in the realm of Agentforce and other agent-based solutions. The session featured the introduction of a unified testing approach, including a comprehensive testing toolkit integrated with common libraries like Salesforce DX, and a native Salesforce UI for non-technical users, aiming to simplify AI asset testing across the lifecycle. Demonstrations showcased how the new Testing Center could be utilized for prompt and agent testing, offering a peek into the 'black box' of AI and enabling assertions on various output characteristics. The roadmap outlined the pilot phase for agent testing, expansion to prompt and RAG testing, synthetic data generation, and a general availability target set for early next year, underscoring Salesforce's commitment to enhancing AI reliability for developers and businesses alike.
Key Takeaways by Einstein:

• Salesforce is introducing a unified testing toolkit and UI to enable both technical and non-technical users to easily define, execute, and view AI asset tests in bulk, ensuring the quality and trustworthiness of AI applications.

• Live demonstrations highlighted the Testing Center's capabilities, such as generating responses, validating expected outcomes, and offering insights into AI processes, thereby demystifying the AI 'black box'.

• The session outlined a phased roadmap for the testing framework, starting with an agent testing pilot, followed by prompt testing, and culminating in general availability, reflecting Salesforce's proactive approach to AI quality assurance.

Einstein Feedback : Report a Concern

With sessions and hands-on activities led by product experts, visionaries, and fellow Trailblazers, Dreamforce offers a deep dive into the latest in CRM + AI + Data + Trust.

Merlinda Lu

Salesforce Developer, IBM