Coding Language Detector
Our Coding Language Detector is designed to analyze LLM outputs and identify potential code snippets. This capability proves invaluable for platforms seeking to monitor or control programming-related queries, ensuring the appropriate handling of such LLM outputs.
Vulnerability
Instances where users create crafty prompts for LLMs to generate code that can pose risks. Such actions might exploit vulnerabilities, test scripts, or engage in activities beyond the platform's intended scope. Controlling and monitoring the nature of inserted code becomes pivotal in maintaining system integrity and safety.
Usage
We leverage huggingface/CodeBERTa-language-id model, our Code Detector adeptly recognizes code snippets within LLM outputs across various programming languages. Developers can fine-tune the detector to either whitelist or blacklist specific languages, granting full control over the types of code permitted in user queries.
Configuration
Note
The detector excels at extracting and detecting code snippets from Markdown in the following languages:
- Go
- Java
- JavaScript
- PHP
- Python
- Ruby
from guardrail.firewall.output_detectors import CodingLanguageOutput
# List of programming languages allowed in outputs
allowed_languages = ["Python", "Go"]
# List of programming languages denied in outputs
denied_languages = ["JavaScript", "PHP", "Ruby", "Java"]
# Instantiate CodingLanguageOutput detector with allowed and denied programming languages
output_detectors = [CodingLanguageOutput(allowed=allowed_languages, denied=denied_languages)]
sanitized_response, valid_results, risk_score = firewall.scan_output(sanitized_prompt, response_text, output_detectors)