Overview
A healthy, well-maintained Knowledge Base is the cornerstone of Element451's BoltAI features, Bolt Assistants for Students, Bolt Assistants for Staff, and Bolt Discovery. These tools leverage your Knowledge Base to provide accurate, contextual, and conversational answers to students, staff, and users. Maintaining your Knowledge Base ensures your BoltAI solutions deliver consistent and accurate results, empowering students with the information they need while freeing up your team to focus on other tasks.
In this article, we’ll share best practices for curating and maintaining your Knowledge Base so you can get the most out of BoltAI. Before diving in, we recommend reviewing our Knowledge Base Overview article, which covers the basics, including the difference between private and public repositories, how to add/manage sources, source types, and more.
Why Curating and Maintaining Your Knowledge Base Is Essential
Your Knowledge Base serves as the foundation for BoltAI. It’s more than a repository of information—it powers the semantic understanding behind these tools. Semantic search enables BoltAI to analyze and retrieve the most relevant knowledge by understanding the meaning behind users’ questions (not just the keywords). This advanced capability makes the accuracy, clarity, and organization of your Knowledge Base critical for the successful deployment of Bolt Assistants and Bolt Discovery.
Without proper curation and regular maintenance:
Your assistants may deliver incomplete, outdated, or incorrect answers.
Gaps in knowledge can lead to uncertain responses or unanswered queries.
Students may lose trust in where they look for information.
By following these best practices, you can ensure your Knowledge Base is a powerful, reliable resource for both your students and your team.
Best Practices for Curating Your Knowledge Base
Utilize Categories When Adding Sources
Utilize Categories When Adding Sources
Categories are one of the most powerful tools for organizing your Knowledge Base.
Viewing + Filtering: Categories make it easier for your team to filter and review sources in the Source List or perform bulk actions.
Control Bolt Assistant Knowledge Access: Bolt Assistants are designed to specialize in certain functional areas, mirroring how we work in real life. Because of this, you can restrict their access to knowledge, and one of the easiest ways to do this is by category.
Example: For a Financial Aid Assistant, categorize all financial aid-related sources. By assigning that assistant access only to the “Financial Aid” category, you ensure it only answers questions related to that topic.
Organize Sources Using Folders
Organize Sources Using Folders
Folders are another helpful way to manage your Knowledge Base:
Create folders based on institutional needs, such as “Admissions,” “Academics,” or “Student Life.”
Use the bulk action feature to move multiple sources into a folder at once for easier organization.
Follow the 3 C’s of Knowledge Curation
Follow the 3 C’s of Knowledge Curation
When adding knowledge, always prioritize these principles:
Curated: Add sources that are relevant, accurate, and up-to-date.
Concise: Use clear, concise language. Avoid unnecessary details—your assistants and Discovery will handle making it conversational.
Clear: Ensure knowledge is well-written, avoids ambiguity, and does not include conflicting information.
By following the 3 C’s, you’ll set your AI tools up for success, ensuring they provide precise and reliable answers to users.
Use a Variety of Source Types
Use a Variety of Source Types
Each source type in your Knowledge Base offers unique advantages:
Website/URL Sources: These can be set to sync daily, so updates to your website automatically update your Knowledge Base. The links can also be displayed/referenced in answers from Bolt Assistants or Bolt Discovery for deeper exploration and additional context.
Custom Answers: Because these are prioritized over other source types, custom answers are great for strategic, high-priority Q&A pairs. Add crucial or frequently asked questions here to ensure they’re top of mind for your AI tools.
Text/Document Sources: Use these for longer or more detailed knowledge, but remember to break up large blocks of text for easier human review.
Pro Tip: Don’t limit yourself to just one type—using a mix of source types can help your Knowledge Base cover a broader range of content and use cases.
Directly Include Specific Information
Directly Include Specific Information
When curating your Knowledge Base, explicitly stating key facts, data, or answers ensures BoltAI tools provide the most accurate and relevant responses. Bolt Assistants and Discovery prioritize direct, specific information, so including these details is critical.
For example:
Instead of writing: “We offer graduate programs,” use: “We offer 10 graduate programs across business, education, and healthcare fields.”
Instead of: “Our deadline is soon,” use: “Our application deadline for Fall 2024 is January 15, 2024.”
Why this matters: Specific information like the above ensures that Bolt Assistants don’t rely on general statements or infer details from related sources. This is especially important for numeric or critical facts (e.g., deadlines, program counts, or event dates).
Break Up Large Chunks of Text
Break Up Large Chunks of Text
While BoltAI can process large blocks of text, breaking them into smaller sections makes it easier for your team to review and audit content. It also allows BoltAI to retrieve specific, actionable information more effectively.
Best Practices for Maintaining Your Knowledge Base
Assign a Point Person/Team to Conduct Audits
Assign a Point Person/Team to Conduct Audits
Designate one or more team members to be responsible for Knowledge Base maintenance. Define clear roles and responsibilities, and create a plan for how and when audits will take place (e.g., monthly, quarterly). Regular audits are vital to ensuring your Knowledge Base stays accurate and up-to-date.
Set a schedule for when audits should occur.
Divide responsibilities if multiple team members are involved.
Establish processes for reviewing, updating, and retiring outdated sources.
If multiple team members are responsible for maintaining the Knowledge Base:
Assign clear roles for reviewing specific categories, source types, or tools (Bolt Assistants vs Bolt Discovery).
Use shared logs or project management tools to track what’s been audited or updated.
For ideas on strategies to facilitate your audit, continue reading below on Leveraging the Insights Dashboards and Spotchecking Conversations.
Test BoltAI Tools
Test BoltAI Tools
Start by testing your Bolt Assistants and Discovery tools to ensure they’re performing as expected:
Use the Preview feature in Student Assistants to test questions directly and analyze the responses.
Create a test page with Messenger enabled, interact with your assistants, and verify they provide the correct answers.
Test Bolt Discovery with queries related to content in your Knowledge Base to ensure users can easily find the information they need.
Leverage the Insights Dashboards
Leverage the Insights Dashboards
The Insights Dashboards in Element451 provide comprehensive metrics for understanding how your Knowledge Base is performing. Use these tools to identify gaps and prioritize updates:
Conversations Dashboard (Bolt Assistants)
Go to Data + Automation > Insights > Conversations and select the Bolt Assistants tab at the top to review performance metrics.
Key data points include:
Knowledge Base Gap Card: Displays the number of times a Bolt Assistant encountered an uncertain response due to missing knowledge.
Knowledge Base Gap Log: A detailed list of those interactions, each with a link to the specific conversation. Use this to review gaps and update your Knowledge Base accordingly.
Knowledge Base Article References: Shows which articles are being used the most, helping you prioritize them for review and updates.
Gap Responses by Funnel Stage: Look at a visualization to understand where students are most likely to encounter gaps in knowledge.
Bolt Discovery Dashboard
Go to Data + Automation > Insights > Discovery and select the Overview tab at the top to review performance metrics.
Key metrics include:
Knowledge Base Gap Card: Similar to the Conversations/Bolt Assistants dashboard, this shows gaps in Discovery results.
Knowledge Base Gap Keywords: Review common search terms that led to unanswered queries.
Top Knowledge Base Articles Referenced: Identify your most used articles to ensure their content is accurate and current.
Knowledge Base Metrics by Topic or Date: Review trends to see which areas of knowledge are being referenced most frequently or need improvement.
Use the Query Log tab to access a complete list of all queries made and the responses provided. This allows you to evaluate the accuracy of answers and pinpoint areas for improvement. Each log entry also includes a link to the corresponding thread.
To quickly identify areas needing attention, use the Knowledge Base Gap control filter and set it to “Yes.” This will generate a focused list of queries where gaps occurred, helping you prioritize updates to your Knowledge Base efficiently.
Pro Tip: Use the logs and keyword data to identify common trends. For example, if multiple gaps are related to financial aid queries, prioritize adding more detailed financial aid content.
Spot-Check Conversations and Threads
Spot-Check Conversations and Threads
To ensure both Bolt Assistants and Bolt Discovery are performing optimally, it’s important to review their interactions with users regularly. Spot-checking conversations and threads help you verify that responses are accurate, aligned with your institution’s tone, and delivering the desired results.
Why it matters: Insights Dashboards will highlight Knowledge Base gaps and metrics, but they don’t account for cases where an assistant or Discovery gave an answer that might not be accurate or ideal.
Spot-Checking Bolt Assistants
Navigate to Engagement > Conversations > All Conversations and filter your inbox by adding the “Bolt Assistant - Participated” filter to view only those conversations.
Use the Advanced Filter Sidesheet at the bottom to save this filter as a Custom View for easy access in the future.
Once filtered, review a sampling of these conversations to ensure the assistant provided correct and appropriate answers.
Spot-Checking Bolt Discovery
To efficiently review and evaluate Bolt Discovery’s performance, use the Query Log within the Insights dashboard:
Navigate to Data + Automations > Insights > Bolt Discovery and click on the Query Log tab at the top. This provides a detailed list of all user queries and the responses given by Bolt Discovery.
The Query Log includes powerful filters to help you narrow down the data:
Date: Review queries within a specific time range.
Source Type, Title, or Category: Drill down into specific sources to analyze their performance.
Knowledge Base Gap: Filter by “Yes” to instantly generate a list of queries where Discovery encountered a knowledge gap. This is a great way to identify areas where additional content is needed or where existing content could be improved.
Pro Tip: Set a recurring schedule to review the Query Log weekly or monthly. Focus on queries with knowledge gaps first, as addressing these will have the greatest impact on improving Bolt Discovery’s performance.
What to Do If You Spot Issues
If you identify incorrect responses, review the Knowledge Base sources associated with the query.
Update, rewrite, or add sources to fill any identified gaps.
For questions that aren’t answered well, even with proper knowledge, consider creating a Custom Answer for a more precise response.
Final Thoughts
A well-maintained Knowledge Base is essential for empowering your BoltAI tools to deliver accurate, helpful, and engaging experiences for users. By following these best practices, you can ensure your Knowledge Base remains a valuable resource, and your AI-powered tools are performing at their best.
If you’re ready to start curating or auditing your Knowledge Base, check out our Knowledge Base Overview for detailed instructions on adding and managing sources.