Bias

Step 1

Import the required libraries and initialize environment variables.

from testsavant.guard import InputGuard, OutputGuard
from testsavant.guard.output_scanners import Bias
import os
from dotenv import load_dotenv
load_dotenv()

Step 2

Prepare a long article to simulate output context, initialize the guard, and register the Bias scanner.

def dummy_llm(input):
    return input
long_article = open("../docs/article.txt", "r").read()
print(len(long_article))

ts_api = OutputGuard(
    API_KEY=os.environ.get("TEST_SAVANT_API_KEY"),
    PROJECT_ID=os.environ.get("TEST_SAVANT_PROJECT_ID"),
    remote_addr=os.environ.get("TEST_SAVANT_REMOTE_ADDR")
)
scanner = Bias(tag="base", threshold=0.5)
ts_api.add_scanner(scanner)

Step 3

Test output scanning with biased content both with and without added context.

prompt = "The government's policy clearly favors the wealthy elite."
prompts = [prompt, prompt + " " + long_article, long_article + " " + prompt]

for prompt in prompts:
    result = ts_api.scan(prompt, prompt)
    if result.is_valid:
        print("Prompt is valid")
    else:
        print("Prompt is invalid")

Parameters

tag: str (default: "base") — label to group rule configuration.
threshold: float (default: 0.5) — decision threshold; higher is stricter.