jdZddlmZddlZddlZddlmZddlmZddl m Z m Z m Z ddl mZejeZdZd d Z d d Zy) um author: Yagnik Poshiya organization: Webbrains Technologies Private Limited Dedicated SQL verification / quality-gate LLM — system-controlled model that inspects the generated SQL, the query result, and the visitor's original question, then decides whether the answer is acceptable or whether a retry is needed (with actionable feedback for the SQL generator). ) annotationsN)Any)Settings)GenerationConfig LLMClientLLMUsage)build_llm_clientuD You are an SQL result quality-gate for a RAG agent that answers website visitor questions from a relational database. You receive: - The visitor's original question (plain text). - The database schema (tables and columns). - The SQL query that was generated. - The query result (JSON with columns + rows, an error message, or "[Query returned 0 rows]"). YOUR TASK: Decide whether the query result adequately answers the visitor's question. OUTPUT — a single JSON object with exactly these fields: { "verdict": "pass" | "retry", "reason": "one-line explanation", "feedback": "actionable hint for the SQL generator (only when verdict=retry)" } RULES: 1. verdict = "pass" when the result rows contain data that can reasonably answer the question, even if not perfect. 2. verdict = "retry" when: a. The query returned an SQL error. b. The query returned 0 rows but the schema suggests data should exist. c. The query columns are obviously wrong (e.g. selected IDs instead of names, missed a required JOIN, wrong aggregation). d. The SQL is syntactically valid but semantically misaligned with the question (wrong table, wrong filter, etc.). e. The result appears to expose raw identifier columns instead of human-readable fields when the visitor is asking for descriptive values (e.g., "name", "title", "label", "display", "status", "category", "location"). f. The SQL references table names or column names that are not present in the provided schema snapshot context. 3. When verdict = "retry", the "feedback" field MUST contain a concrete, actionable correction hint — e.g. "JOIN the `orders` table on `customer_id` to get order counts" — NOT generic advice like "try again". 4. Do NOT output anything except the JSON object. 5. Be lenient: partial results are "pass" as long as they're relevant. 6. Do NOT enforce strict exact-keyword matching on visitor terms. If the SQL result is semantically close and useful (about 80-90% relevant), prefer verdict = "pass" instead of "retry". 7. For place names or text fields, minor spelling differences or partial matches are acceptable when the returned data is still clearly relevant. HEURISTIC FOR e: - If the selected column names include exactly `id` or end with `_id`, and the visitor question is asking for a descriptive field (name/title/ label/display/etc.), set verdict = "retry". - In verdict="retry", set feedback to instruct replacing those raw IDs with JOINs to the relevant lookup/entity tables and selecting their descriptive columns (usually named `name` or `title`). cBt|j|jS)N)r sql_verifier_providersql_verifier_model)settingss J/var/www/html/agent-engine/apps/agents/rag-agent/src/agent/sql_verifier.py_build_verifier_clientrRs H::Htjdt| j| ddd dd |j fSt!| j#dxsd j%} | dvrd } | t!| j#dxsdt!| j#dxsdd|j fS#tj$r-tjd | ddd dd |j fcYSwxYw)zjReturn (verdict_dict, usage). verdict_dict has keys: verdict ("pass"|"retry"), reason, feedback. zVisitor's question: z Database schema: z Generated SQL: z Query result: z g?T) temperature response_json)configpassz verifier returned empty response)verdictreasonz#sql_verifier: non-JSON response: %sNzverifier returned non-JSONz.sql_verifier: expected JSON object, got %s: %sz!verifier returned non-object JSONr)rretryrfeedback)rrr)r_SYSTEM_INSTRUCTIONgeneratertextstripusagejsonloadsJSONDecodeErrorloggerwarning isinstancedicttype__name__strgetlower) r user_message schema_detailsql query_resultclient user_blockpromptresprdatars rverify_sql_resultr6Vs$H -F ~.*O,% & )$$D 5F ??CtD  D 99?? D !-OPRVR\R\\\Wzz$ dD !GdI\I\^bcgdg^hi!-PQSWS]S]]]$((9%/0668G''dhhx(.B/,23  zz     WrAs# $::(   8 $8vY222 2 2  2%2r