Factuality Tool

The Factuality Tool detector prevents potential ungrounded additions (Hallucinations) by checking truthfulness of claims with tools.

Vulnerability

LLMs can “hallucinate” information not present in the original dataset — commonly found in retrieval-based and generative models. This problem can be exacerbated when chat history is present because the model might mistakenly use that context to generate an answer that seems relevant but is factually incorrect.

Usage

Utilizing our Factuality Tool detector allows AI teams to detect and monitor LLMs confidently for untruthful content using claim extractions and search tools.

Note: Leverages Serper to retrieve Google search results. Set SERPER_API_KEY to .env to enable Google search results

Configuration

Initialize the Factuality Tool Detector with the desired options:

from guardrail.firewall.output_detectors import FactualityDetector

firewall = Firewall(no_defaults=True)
output_detectors = [FactualityDetector(data_link="../your_file_location.json")]

sanitized_prompt = "Introduce Percy Liang"
response_text = "Percy Liang is a professor at Harvard"

sanitized_response, valid_results, risk_score = firewall.scan_output(sanitized_prompt, response_text, output_detectors)

Example output breaking down evidences, claim_level_factuality, response_level_factuality, sources used and more.

{
    "average_claim_level_factuality": 0.0,
    "average_response_level_factuality": 0.0,
    "detailed_information": [
        {
            "prompt": "Introduce Percy Liang",
            "response": "Percy Liang is a professor at Harvard",
            "category": "kbqa",
            "search_type": "online",
            "claims": [
                {
                    "claim": "Percy Liang is a professor at Harvard"
                }
            ],
            "queries": [
                [
                    "Is Percy Liang a professor at Harvard?",
                    "Percy Liang Harvard"
                ]
            ],
            "evidences": [
                {
                    "evidence": [
                        "Percy Liang is an Associate Professor of Computer Science at Stanford University (B.S. from MIT, 2004; Ph. D. from UC Berkeley, 2011) and the director of the Center for Research on Foundation Models.",
                        "I publish mostly in machine learning (ICML, NeurIPS) and natural language processing (ACL, NAACL, EMNLP) conferences. I am drawn to simple things, want to ...",
                        "Percy Liang is an Associate Professor of Computer Science at Stanford University (B.S. from MIT, 2004; Ph.D. from UC Berkeley, 2011). His two research goals are ...",
                        "Missing: Harvard | Show results with:Harvard",
                        "Our aim is to produce methods that are provably robust to such deviations. In this talk, I will provide an overview of the work my group has done on this topic ...",
                        "AI to Accelerate the Life Sciences/Drug Discovery \u00b7 Alternative Models to Trials Using Real-World Data \u00b7 Driving Gene-by-Environment Discovery in Biobanked Data."
                    ],
                    "source": [
                        "None",
                        "https://cs.stanford.edu/~pliang/",
                        "https://profiles.stanford.edu/percy-liang",
                        "https://profiles.stanford.edu/percy-liang",
                        "https://mlfoundations.org/talk/percy/",
                        "https://dbmi.hms.harvard.edu/research-areas/artificial-intelligence"
                    ]
                }
            ],
            "claim_level_factuality": [
                {
                    "reasoning": "The given text states that Percy Liang is a professor at Harvard. However, the provided evidences clearly state that Percy Liang is an Associate Professor of Computer Science at Stanford University. The evidences consistently mention Stanford University as his affiliation, while there is no mention of Harvard University. Therefore, it can be concluded that the given text is non-factual.",
                    "error": "The given text is non-factual.",
                    "correction": "Percy Liang is an Associate Professor of Computer Science at Stanford University.",
                    "factuality": false,
                    "claim": "Percy Liang is a professor at Harvard"
                }
            ],
            "response_level_factuality": false
        }
    ]
}

The Factuality Detector can be further customized to use internal tools, knowledge bases, and external sites as well that domain-specific.