Skip to main content

Monitoring and Improving

Once you've set up your initial training data, Teela gets smarter over time through user interactions and feedback. This page covers the tools for monitoring how Teela is performing and making targeted improvements.

For building training data from scratch, see Training the AI. For a step-by-step playbook, see Training Best Practices.

Training Validation Chat​

The Training Validation chat is your testing ground. It works just like the regular Teela chat, but it's designed for admins to verify accuracy before rolling out changes to users.

When you ask a question in the validation chat, Teela shows you:

  • The generated SQL query
  • The results
  • A failure analysis if the query doesn't look right, including which tables and columns were used and where things may have gone wrong
  • An impact assessment showing how a fix would affect other queries

Use the validation chat after making any significant training changes, such as adding documentation, creating aliases, or modifying SQL examples.

Learning Review Queue​

Teela learns from every interaction. When users ask questions, rate answers, or provide feedback, Teela generates learning candidates: potential improvements to its training data. The Learning Review Queue is where you review and manage these candidates.

The Three Tabs​

The Learning Review Queue is organized into three tabs:

Pending Review​

These are learning candidates that Teela has generated but hasn't acted on yet. They're waiting for you to decide whether they should become part of Teela's training data.

For each pending candidate, you'll see:

  • The user's question (and any alternate phrasings other users have used for the same intent). Teela groups similar questions together so you can see the full picture.
  • Multiple SQL variants: Teela may have generated several possible SQL translations for the same question. You can review each variant and pick the best one.
  • Context about which tables, columns, and aliases were involved.

Actions you can take:

  • Approve: Indexes the candidate as official training data. Teela will use it to improve future answers.
  • Dismiss: Removes the candidate from the queue without adding it to training. Use this for junk, irrelevant questions, or candidates that aren't worth keeping.

Auto-Approved​

These are candidates that met the auto-approve threshold (see below) and were automatically added to training data. You can review them here to make sure nothing slipped through that shouldn't have.

Actions you can take:

  • Unapprove: Removes the candidate from training data and moves it back to pending review. Use this if you spot something that was auto-approved but shouldn't have been.

Feedback-Flagged​

These are candidates generated from negative user feedback (thumbs-down ratings). They represent queries where something went wrong, and they need your attention.

For each feedback-flagged item, you'll see:

  • The original question and Teela's response
  • AI diagnosis: Teela's analysis of what went wrong. This might be something like "Used the wrong table for revenue calculation" or "Missing filter for deleted records."
  • Suggested improvements: Specific recommendations for how to fix the issue, such as adding documentation, creating an alias, or updating a SQL example.
  • A "Create Training From This" button: One click to turn the diagnosis into a new piece of training data (documentation, alias, or SQL example) with the relevant fields pre-filled.

Actions you can take:

  • Create Training From This: Opens a pre-filled form to add the fix as training data.
  • Approve: If the flagged response was actually correct (the user may have been mistaken), approve it.
  • Dismiss: If the feedback isn't actionable, dismiss it.

The Auto-Approve System​

Teela uses a simple signal-based system for auto-approval:

  • When a learning candidate accumulates 2 or more positive signals (thumbs-up ratings, DataClip saves) and 0 negative signals, it's automatically approved and incorporated into training data.
  • Candidates with any negative signals always require manual review.

This means Teela continuously improves with minimal admin effort, while still giving you control over anything questionable.

tip

Make it a habit to check the Feedback-Flagged tab at least once a week. These are your best leads for high-impact training improvements because they represent real questions from real users where Teela fell short.

Knowledge Gap Dashboard​

The Knowledge Gap Dashboard shows you terms and concepts that users are asking about but Teela can't map to your database. These are opportunities to improve.

What You'll See​

For each unmapped term, the dashboard displays:

  • The term or phrase users are using in their questions
  • Occurrence count: How many times this term has appeared in user queries
  • Last seen date: When the term was most recently used
  • A quick link to create an alias or add documentation

Time Range Filter​

You can filter the dashboard by time range to focus on what matters most:

  • 7 days: See what's trending right now. Great for catching new terminology or recent schema changes that broke existing mappings.
  • 30 days: The default view. Gives you a solid picture of ongoing gaps.
  • 90 days: The big picture. Useful for quarterly reviews or identifying long-standing gaps that keep coming back.

Quick Actions​

From the dashboard, you can take action on any gap with a single click:

  • Create Alias: Jumps to the Alias Manager with the term pre-filled, ready for you to map it to a table or column.
  • Add Documentation: Opens the documentation editor so you can explain what the term means in the context of your data.

Reviewing the Knowledge Gap Dashboard regularly is one of the best ways to keep Teela accurate and useful. See Aliases for details on turning gaps into mappings.

The Feedback Loop​

Training isn't a one-time task. It's an ongoing process that gets easier over time. Here's how the feedback loop works:

  1. Users ask questions and rate the answers with thumbs-up or thumbs-down.
  2. Teela generates learning candidates based on user interactions and feedback.
  3. High-confidence candidates are auto-approved, while others land in the Learning Review Queue for your review.
  4. You review flagged candidates, approve or reject them, and add documentation or aliases as needed.
  5. Teela gets smarter with each cycle.

The more your team uses Teela, the better it gets, especially when they take a moment to rate their answers.

  • Training the AI: Schema extraction, documentation, SQL examples, vocabulary wizard, and import/export
  • Training Best Practices: A step-by-step playbook for getting the most out of training
  • Aliases: Map business terms to database columns
  • Data Dictionary: Review and enhance table and column descriptions