Advanced Bedrock Patterns: Structuring Security Analysis with Pydantic

In the world of automated security engineering, reliability is paramount. When we built the core logic for Code Guardian AI, we faced a common challenge with Generative AI: unpredictable outputs.

Early implementations of LLM-based analysis often rely on prompt engineering to "beg" the model to return JSON. "Please return only JSON," we say. "Do not include markdown prologue." And 95% of the time, it works. But that 5% failure rate breaks automation pipelines, causes Lambda timeouts, and floods logs with parsing errors.

We solved this by combining AWS Bedrock's Converse API with Pydantic. Here is how we did it.

The Problem: Unstructured Intelligence

We use LLMs to triage leaked credentials found in BitBucket. We need the AI to decide:

1. Is this a real secret or a false positive? (Boolean/Enum)
2. What is the severity? (Critical/High/Medium/Low)
3. What is the remediation? (String)

If the model replies with "Sure, here is the analysis: {json}", our pipeline crashes. We need a guarantee.

The Solution: Tool Use as a Schema Enforcer

The converse API in Bedrock allows us to define "tools"—essentially function signatures. By instructing the model that it must call a specific tool to report its findings, we force it to conform to a strict schema.

Step 1: Define the Schema with Pydantic

We define our desired output structure using Python's Pydantic library. This gives us rigorous type validation out of the box.

from pydantic import BaseModel, Field
from enum import Enum

class SeverityLevel(str, Enum):
    HIGHEST = "Highest"
    HIGH = "High"
    MEDIUM = "Medium"
    LOW = "Low"

class FindingLabel(str, Enum):
    SECRET_FINDING = "Secrets"
    FALSE_POSITIVE = "False-Positive"

class SecretFindingReport(BaseModel):
    """A structured report for analyzing a potential leaked secret."""
    
    risk_assessment: str = Field(..., description="Detailed risk assessment.")
    finding_analysis: str = Field(..., description="Analysis of the secret type.")
    contextual_clues: str = Field(..., description="Evidence from code context.")
    severity: SeverityLevel = Field(..., description="Assessed severity.")
    label: FindingLabel = Field(..., description="Final classification.")

Step 2: Generate the Tool Specification

Instead of writing complex JSON schemas manually, we generate them directly from the Pydantic model.

tool_name = "save_secret_finding_report"
tool_spec = {
    "name": tool_name,
    "description": "Analyzes and saves a report on a potential secret.",
    "inputSchema": {
        "json": SecretFindingReport.model_json_schema()
    },
}

Step 3: Forcing the Conversation

When we call Bedrock, we pass this tool configuration. The model "thinks" it is calling a function save_secret_finding_report to complete its task.

response = bedrock_runtime.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": prompt}]}],
    toolConfig={"tools": [{"toolSpec": tool_spec}]}
)

Step 4: Type-Safe Parsing

When the response comes back, we don't regex for JSON. We extract the tool input and feed it straight back into Pydantic. If the model hallucinated a field or used the wrong type, Pydantic throws a validation error immediately, catching issues before they hit our database.

output_message = response["output"]["message"]
tool_use = output_message["content"][0]["toolUse"]
tool_input = tool_use["input"]

# Validate via Pydantic
report = SecretFindingReport(**tool_input)

print(f"Severity: {report.severity}") # strictly typed!

Why This Matters for Enterprise

This pattern transforms "prompt engineering" into "software engineering".

1. Type Safety: We can trust the report.severity is a valid Enum member.
2. No Hallucinated Fields: Extra fields are stripped or flagged.
3. Localized Validation: Validation rules (like "must be at least 10 chars") are encoded in the model class.

At ZeroDotFive, we use this pattern to process hundreds of security findings daily with zero parsing failures. It’s how we turn stochastic AI into deterministic security workflows.