A comprehensive Python wrapper library for OpenTofu and Terraform that provides a Pythonic interface to infrastructure as code operations.
This project was built as an example for Chapter 11 of Terraform in Depth to demonstrate how to wrap the OpenTofu or Terraform binaries for control from Python. Despite being an example, this is an active and maintained open source project suitable for production use.
- Full Terraform/OpenTofu Lifecycle Management: Initialize, validate, plan, apply, and destroy infrastructure
- Structured Data Models: Rich Python data classes for all Terraform objects (plans, state, outputs, etc.)
- Real-time Event Handling: Stream and handle Terraform execution events as they occur
- Type Safety: Full type annotations and structured schemas for all Terraform JSON outputs
- Cross-Platform: Works with both OpenTofu and Terraform binaries
- Production Ready: Used in real-world scenarios with comprehensive error handling
- TofuPy
Either tofu
or terraform
must be installed and available in your PATH.
pip install tofupy
from tofupy import Tofu
# Initialize a workspace
workspace = Tofu(cwd="/path/to/terraform/config")
# Initialize Terraform
workspace.init()
# Validate configuration
validation = workspace.validate()
if not validation.valid:
print("Configuration is invalid!")
for diagnostic in validation.diagnostics:
print(f"Error: {diagnostic.summary}")
# Create and review a plan
plan_log, plan = workspace.plan()
if plan and not plan.errored:
print(f"Plan will create {len([c for c in plan.resource_changes.values() if 'create' in c.change.actions])} resources")
# Apply changes
apply_log = workspace.apply()
print(f"Applied: {apply_log.added} added, {apply_log.changed} changed, {apply_log.removed} removed")
# Get outputs
outputs = workspace.output()
for name, output in outputs.items():
print(f"{name}: {output.value} (type: {output.type})")
The Tofu
class is your main interface to Terraform/OpenTofu operations:
from tofupy import Tofu
# Basic initialization
workspace = Tofu() # Uses current directory
# Custom configuration
workspace = Tofu(
cwd="/path/to/terraform/config",
binary="terraform", # or "tofu"
log_level="DEBUG", # ERROR, WARN, INFO, DEBUG
env={"TF_VAR_environment": "production"}
)
# The workspace automatically detects binary version and validates compatibility
print(f"Using {workspace.binary_path} version {workspace.version} on {workspace.platform}")
TofuPy provides rich Python data classes that mirror Terraform's JSON structures:
Plan
- Terraform plan with resource changes, drift detection, and metadataState
- Current Terraform state with resources and outputsApplyLog
/PlanLog
- Execution logs with summaries and diagnosticsValidate
- Validation results with error/warning detailsOutput
- Terraform outputs with values, types, and sensitivity infoResource
- Individual resources with addresses, types, and valuesChange
- Resource changes with before/after states and actions
from tofupy import Tofu
workspace = Tofu(cwd="./terraform")
# Basic initialization
success = workspace.init()
# Initialize with backend configuration
success = workspace.init(backend_conf="backend.hcl")
# Initialize without backend (useful for validation)
success = workspace.init(disable_backends=True)
# Initialize with extra arguments
success = workspace.init(extra_args=["-upgrade"])
# Validate configuration
validation = workspace.validate()
print(f"Configuration valid: {validation.valid}")
print(f"Errors: {validation.error_count}, Warnings: {validation.warning_count}")
# Handle validation issues
if not validation.valid:
for diagnostic in validation.diagnostics:
if diagnostic.severity == "error":
print(f"❌ {diagnostic.summary}")
print(f" {diagnostic.detail}")
else:
print(f"⚠️ {diagnostic.summary}")
# Basic plan
plan_log, plan = workspace.plan()
# Plan with variables
plan_log, plan = workspace.plan(
variables={
"environment": "production",
"instance_count": "3"
}
)
# Plan with output to file
from pathlib import Path
plan_file = Path("./my-plan.tfplan")
plan_log, plan = workspace.plan(plan_file=plan_file)
# Plan with extra arguments
plan_log, plan = workspace.plan(
extra_args=["-target=aws_instance.example"]
)
# Analyze the plan
if plan and not plan.errored:
print(f"Terraform version: {plan.terraform_version}")
print(f"Plan is applyable: {plan.applyable}")
# Review resource changes
for address, change_container in plan.resource_changes.items():
change = change_container.change
actions = ", ".join(change.actions)
print(f"{address}: {actions}")
# Check for creates
if "create" in change.actions:
print(f" Will create {change_container.type} resource")
# Check for updates
if "update" in change.actions:
print(f" Will modify {change_container.type} resource")
# Check for destroys
if "delete" in change.actions:
print(f" Will destroy {change_container.type} resource")
# Apply from plan file
apply_log = workspace.apply(plan_file=plan_file)
# Direct apply with variables
apply_log = workspace.apply(
variables={"environment": "staging"}
)
# Apply with auto-approval (default)
apply_log = workspace.apply()
# Review apply results
print(f"Operation: {apply_log.operation}")
print(f"Resources: +{apply_log.added} ~{apply_log.changed} -{apply_log.removed}")
# Handle errors
if apply_log.errors:
print("Apply encountered errors:")
for error in apply_log.errors:
print(f" {error.summary}: {error.detail}")
# Get current state
state = workspace.state()
print(f"State serial: {state.serial}")
print(f"State lineage: {state.lineage}")
print(f"Terraform version: {state.terraform_version}")
# Access state outputs
for name, output in state.outputs.items():
if output.sensitive:
print(f"{name}: <sensitive>")
else:
print(f"{name}: {output.value} ({output.type})")
# Access state resources
root_module = state.root_module
if root_module:
for address, resource in root_module.resources.items():
print(f"Resource: {address}")
print(f" Type: {resource.type}")
print(f" Provider: {resource.provider_name}")
print(f" Values: {resource.values}")
# Get all outputs
outputs = workspace.output()
for name, output in outputs.items():
print(f"Output '{name}':")
print(f" Value: {output.value}")
print(f" Type: {output.type}")
print(f" Sensitive: {output.sensitive}")
# Work with specific outputs
if "database_url" in outputs:
db_url = outputs["database_url"].value
print(f"Connecting to database: {db_url}")
# Destroy all resources
destroy_log = workspace.destroy()
print(f"Destroyed {destroy_log.removed} resources")
# Handle destruction errors
if destroy_log.errors:
print("Destroy encountered errors:")
for error in destroy_log.errors:
print(f" {error.summary}")
TofuPy allows you to hook into Terraform's execution stream for real-time monitoring:
def progress_handler(event):
"""Handle progress events during plan/apply"""
if event.get("type") == "apply_progress":
resource = event.get("hook", {}).get("resource", {})
action = resource.get("action")
addr = resource.get("addr")
print(f"Progress: {action} {addr}")
return True
def error_handler(event):
"""Handle error events"""
if event.get("@level") == "error":
print(f"Error: {event.get('@message')}")
return True
def summary_handler(event):
"""Handle change summaries"""
if event.get("type") == "change_summary":
changes = event.get("changes", {})
add = changes.get("add", 0)
change = changes.get("change", 0)
remove = changes.get("remove", 0)
print(f"Summary: +{add} ~{change} -{remove}")
return True
# Use event handlers during operations
plan_log, plan = workspace.plan(
event_handlers={
"apply_progress": progress_handler,
"diagnostic": error_handler,
"change_summary": summary_handler,
"all": lambda e: print(f"Event: {e.get('type')}") # Catch all events
}
)
apply_log = workspace.apply(
event_handlers={
"apply_progress": progress_handler,
"change_summary": summary_handler
}
)
from tofupy import Tofu
def scan_security_groups(workspace_path):
"""Scan for overly permissive security groups"""
workspace = Tofu(cwd=workspace_path)
workspace.init()
plan_log, plan = workspace.plan()
if not plan or plan.errored:
print("❌ Planning failed")
return
issues = []
for address, change_container in plan.resource_changes.items():
if change_container.type == "aws_security_group_rule":
change = change_container.change
# Check for rules allowing all traffic from internet
if change.after and change.after.get("cidr_blocks") == ["0.0.0.0/0"]:
if change.after.get("from_port") == 0 and change.after.get("to_port") == 65535:
issues.append(f"⚠️ {address} allows all traffic from internet")
elif change.after.get("from_port") == 22:
issues.append(f"⚠️ {address} allows SSH from internet")
if issues:
print("Security issues found:")
for issue in issues:
print(f" {issue}")
else:
print("✅ No security issues detected")
# Usage
scan_security_groups("./infrastructure")
from tofupy import Tofu
from pathlib import Path
class InfrastructureManager:
def __init__(self, base_path):
self.base_path = Path(base_path)
self.environments = {}
def setup_environment(self, env_name, variables=None):
"""Initialize a Terraform workspace for an environment"""
env_path = self.base_path / env_name
workspace = Tofu(
cwd=str(env_path),
env={f"TF_VAR_{k}": str(v) for k, v in (variables or {}).items()}
)
# Initialize with environment-specific backend
backend_config = env_path / "backend.hcl"
if backend_config.exists():
workspace.init(backend_conf=backend_config)
else:
workspace.init()
self.environments[env_name] = workspace
return workspace
def deploy_all(self, environments=None):
"""Deploy to multiple environments"""
target_envs = environments or self.environments.keys()
for env_name in target_envs:
print(f"\n🚀 Deploying to {env_name}...")
workspace = self.environments[env_name]
# Validate first
validation = workspace.validate()
if not validation.valid:
print(f"❌ {env_name} configuration invalid")
continue
# Plan changes
plan_log, plan = workspace.plan()
if not plan or plan.errored:
print(f"❌ {env_name} planning failed")
continue
# Show summary
creates = len([c for c in plan.resource_changes.values()
if 'create' in c.change.actions])
updates = len([c for c in plan.resource_changes.values()
if 'update' in c.change.actions])
deletes = len([c for c in plan.resource_changes.values()
if 'delete' in c.change.actions])
print(f" Plan: +{creates} ~{updates} -{deletes}")
# Apply changes
apply_log = workspace.apply()
if apply_log.errors:
print(f"❌ {env_name} apply failed")
for error in apply_log.errors:
print(f" {error.summary}")
else:
print(f"✅ {env_name} deployed successfully")
# Usage
manager = InfrastructureManager("./environments")
# Setup environments
manager.setup_environment("dev", {"instance_type": "t3.micro"})
manager.setup_environment("staging", {"instance_type": "t3.small"})
manager.setup_environment("prod", {"instance_type": "t3.medium"})
# Deploy to all environments
manager.deploy_all()
def generate_infrastructure_report(workspace_path):
"""Generate a comprehensive infrastructure report"""
workspace = Tofu(cwd=workspace_path)
# Get current state
state = workspace.state()
print("=== Infrastructure Report ===\n")
print(f"Terraform Version: {state.terraform_version}")
print(f"State Serial: {state.serial}")
print(f"State Lineage: {state.lineage}")
# Analyze resources by provider
providers = {}
resource_types = {}
if state.root_module:
for address, resource in state.root_module.resources.items():
provider = resource.provider_name or "unknown"
providers[provider] = providers.get(provider, 0) + 1
resource_types[resource.type] = resource_types.get(resource.type, 0) + 1
print(f"\n=== Resources by Provider ===")
for provider, count in sorted(providers.items()):
print(f" {provider}: {count} resources")
print(f"\n=== Resources by Type ===")
for rtype, count in sorted(resource_types.items()):
print(f" {rtype}: {count}")
# List outputs
print(f"\n=== Outputs ===")
if state.outputs:
for name, output in state.outputs.items():
status = "sensitive" if output.sensitive else "public"
print(f" {name} ({output.type}): {status}")
else:
print(" No outputs defined")
# Usage
generate_infrastructure_report("./terraform")
def validate_infrastructure_policies(workspace_path):
"""Run custom policy validations on planned infrastructure"""
workspace = Tofu(cwd=workspace_path)
workspace.init()
plan_log, plan = workspace.plan()
if not plan or plan.errored:
print("❌ Cannot validate - planning failed")
return False
violations = []
# Policy: All EC2 instances must have Name tags
for address, change_container in plan.resource_changes.items():
if change_container.type == "aws_instance":
change = change_container.change
if change.after:
tags = change.after.get("tags", {})
if "Name" not in tags:
violations.append(f"EC2 instance {address} missing Name tag")
# Policy: No publicly accessible RDS instances
for address, change_container in plan.resource_changes.items():
if change_container.type == "aws_db_instance":
change = change_container.change
if change.after and change.after.get("publicly_accessible"):
violations.append(f"RDS instance {address} is publicly accessible")
# Policy: S3 buckets must have versioning enabled
for address, change_container in plan.resource_changes.items():
if change_container.type == "aws_s3_bucket_versioning":
change = change_container.change
if change.after:
config = change.after.get("versioning_configuration", [{}])[0]
if config.get("status") != "Enabled":
violations.append(f"S3 bucket versioning not enabled for {address}")
# Report results
if violations:
print("❌ Policy violations found:")
for violation in violations:
print(f" • {violation}")
return False
else:
print("✅ All policy checks passed")
return True
# Usage
if validate_infrastructure_policies("./terraform"):
print("Proceeding with deployment...")
else:
print("Fix policy violations before deploying")
TofuPy provides comprehensive error handling through structured diagnostics:
from tofupy import Tofu, Diagnostic
workspace = Tofu(cwd="./terraform")
try:
workspace.init()
# Validation errors
validation = workspace.validate()
if not validation.valid:
print("Configuration errors:")
for diagnostic in validation.diagnostics:
if diagnostic.severity == "error":
print(f" ❌ {diagnostic.summary}")
print(f" {diagnostic.detail}")
# Apply errors
apply_log = workspace.apply()
if apply_log.errors:
print("Apply errors:")
for error in apply_log.errors:
print(f" ❌ {error.summary}")
if error.detail:
print(f" {error.detail}")
except FileNotFoundError as e:
print(f"Terraform/OpenTofu binary not found: {e}")
except RuntimeError as e:
print(f"Terraform operation failed: {e}")
TofuPy respects standard Terraform environment variables and allows custom ones:
workspace = Tofu(
env={
"TF_VAR_region": "us-west-2",
"TF_VAR_environment": "production",
"AWS_PROFILE": "production",
"TF_LOG": "DEBUG" # Override log level
}
)
# Use OpenTofu
workspace = Tofu(binary="tofu")
# Use specific Terraform path
workspace = Tofu(binary="/usr/local/bin/terraform")
# Let TofuPy find the binary
workspace = Tofu() # Tries "tofu" first, then "terraform"
# Set Terraform log level
workspace = Tofu(log_level="DEBUG") # ERROR, WARN, INFO, DEBUG
# TofuPy automatically sets TF_IN_AUTOMATION=1 for consistent output
Constructor: Tofu(cwd=None, binary="tofu", log_level="ERROR", env={})
Methods:
init(disable_backends=False, backend_conf=None, extra_args=[])
→bool
validate()
→Validate
plan(variables={}, plan_file=None, event_handlers={}, extra_args=[])
→Tuple[PlanLog, Plan | None]
apply(plan_file=None, variables={}, destroy=False, event_handlers={}, extra_args=[])
→ApplyLog
destroy()
→ApplyLog
state()
→State
output()
→Dict[str, Output]
All data models provide structured access to Terraform's JSON outputs with proper typing and nested object support. Key models include:
- Plan: Resource changes, drift detection, variables, and metadata
- State: Current infrastructure state with resources and outputs
- ApplyLog/PlanLog: Execution logs with summaries and diagnostics
- Validate: Configuration validation results
- Output: Terraform output values with type information
- Resource: Individual infrastructure resources
- Change: Detailed before/after states for resource modifications
- Python 3.10+
- OpenTofu 1.x or Terraform 1.x installed and available in PATH
- No additional Python dependencies (uses only standard library)
This project is part of the Terraform in Depth book but welcomes contributions. See the repository for development setup and contribution guidelines.
See LICENSE file for details.