๐ Technical Design 026 โ Integration Test Quality Standards¶
| Field | Value |
|---|---|
| ID | TD-026 |
| Title | Integration Test Quality Standards |
| Status | Active |
| Owner | Contributors |
| Audience | Contributors, test developers |
| Created | 2025-10-18 |
| Last Updated | 2025-10-18 |
| Reviewers | TBD |
| Approvers | TBD |
| Tracking | Phase of DI Roadmap |
| Related | TD-024 (Meta), TD-025 (Unit) |
๐งญ 1. Summary¶
This technical design establishes prescriptive standards for integration tests in tests/integration/. It implements universal principles from TD-024 with patterns specific to integration testing, which validates component interactions and end-to-end workflows using real implementations.
Integration tests validate how components work together (<500ms execution, realistic scenarios, no mocking except external dependencies).
๐ 2. Context¶
See TD-024 (Meta) for full context. This TD focuses on integration-specific requirements:
- Scope:
tests/integration/- multi-component workflows - Characteristics: Real implementations, longer execution, cross-component
- Approach: Apply universal rules + integration-specific patterns
- Priority: Lower than unit tests (currently audit needed)
๐ฏ 3. Integration Test Standards¶
3.1 Rule: Use Real Implementations¶
Statement: Integration tests use REAL components, not mocks (except external services).
Principle: Integration tests validate actual component interactions, not mocked ones.
Bad Pattern:
# โ FORBIDDEN: Mocking internal components in integration test
def test_di_workflow() -> None:
"""Test DI workflow."""
# โ WRONG: Mocking Container defeats integration test purpose
mock_container = Mock(spec=Container)
mock_container.resolve.return_value = MockService()
bootstrap_runtime = BootstrapRuntime()
bootstrap_runtime.bootstrap(mock_container, plan)
# This isn't really testing integration!
Good Pattern:
# โ
GOOD: Use real Container
def test_di_bootstrap_workflow() -> None:
"""Test complete DI bootstrap workflow."""
# Real implementations
container = Container()
plan = BootstrapPlanBuilder.create().scan_packages(config).build()
bootstrap_runtime = BootstrapRuntime()
bootstrap_runtime.bootstrap(container, plan) # Real interaction
# Verify actual behavior
service = container.resolve(ServiceType)
assert isinstance(service, ServiceType)
Exception: External services (databases, APIs, file systems)
# โ
OK: Mocking external services
def test_application_with_external_api() -> None:
"""Test application behavior with external API."""
with patch("requests.get") as mock_api: # External service: OK to mock
mock_api.return_value = MockResponse({"status": "ok"})
app = Application()
result = app.fetch_data()
assert result["status"] == "ok"
3.2 Rule: No Shared State Between Tests¶
Statement: Each integration test runs in complete isolation. No test affects others.
Bad Pattern:
# โ FORBIDDEN: Shared state
global_container = None
def setup_module():
"""Set up shared container for all tests."""
global global_container
global_container = Container()
global_container.register(Service, Service)
def test_workflow_1():
"""Test first workflow."""
service = global_container.resolve(Service)
# โ Test depends on setup_module state
def test_workflow_2():
"""Test second workflow."""
# โ If test_workflow_1 modifies state, this test is affected
Good Pattern:
# โ
GOOD: Each test isolated
def test_workflow_1() -> None:
"""Test first workflow."""
# Fresh setup for this test only
container = Container()
container.register(Service, Service)
service = container.resolve(Service)
assert isinstance(service, Service)
def test_workflow_2() -> None:
"""Test second workflow."""
# Completely independent setup
container = Container()
container.register(Service, Service)
container.register(OtherService, OtherService)
service = container.resolve(Service)
other = container.resolve(OtherService)
assert isinstance(service, Service)
assert isinstance(other, OtherService)
Using Fixtures for Isolation:
@pytest.fixture
def bootstrap_container() -> Container:
"""Provide a fresh container for each integration test."""
container = Container()
container.register(Service, Service)
return container
def test_workflow_1(bootstrap_container: Container) -> None:
"""Test workflow 1 with isolated container."""
service = bootstrap_container.resolve(Service)
assert isinstance(service, Service)
def test_workflow_2(bootstrap_container: Container) -> None:
"""Test workflow 2 with its own isolated container."""
service = bootstrap_container.resolve(Service)
assert isinstance(service, Service)
3.3 Rule: End-to-End Workflow Testing¶
Statement: Integration tests validate complete workflows from entry to exit.
Example:
def test_application_bootstrap_and_execution() -> None:
"""Test complete application lifecycle: bootstrap, resolve, execute."""
# 1. Setup application configuration
@application(scan_packages=["test_services"])
class TestApp(ApplicationBase):
def run(self) -> None:
pass
# 2. Create module
import sys
sys.modules["test_app"] = type(sys)("test_app")
sys.modules["test_app"].TestApp = TestApp
try:
# 3. Execute through public API
ApplicationRuntime.launch("test_app:TestApp")
# 4. Verify outcome: no exceptions, workflow completed
# Additional assertions about state after execution
finally:
del sys.modules["test_app"]
3.4 Rule: Independent Test Resources¶
Statement: Each integration test suite has dedicated resources with no cross-test sharing.
Bad Pattern:
tests/resources/
โโโ shared_services/ # โ Shared across multiple test suites
โโโ test_di/ # Uses shared_services
โโโ test_reflection/ # Also uses shared_services (coupling!)
Good Pattern:
tests/resources/
โโโ di/
โ โโโ bootstrap/ # Resources for bootstrap tests
โ โโโ injection/ # Resources for injection tests
โ โโโ scanning/ # Resources for scanning tests
โโโ integration/
โโโ di_reflection/ # Resources for DI+Reflection integration
3.5 Rule: Document Why Integration Test Exists¶
Statement: Explain what cross-component behavior is being validated.
def test_di_with_reflection_integration() -> None:
"""Test DI container integrates correctly with reflection system.
Validates that the DI container uses reflection to:
1. Scan packages for annotated services
2. Inspect service constructors for dependencies
3. Build dependency graphs correctly
4. Resolve services with transitive dependencies
This is an integration test of DI + Reflection subsystems.
"""
# Implementation
3.6 Rule: Realistic Scenarios¶
Statement: Test realistic use cases, not artificial scenarios.
Artificial (Bad):
def test_container_with_empty_service() -> None:
"""Test container with minimal service."""
class EmptyService:
pass
container = Container()
container.register(EmptyService, EmptyService)
service = container.resolve(EmptyService)
# Too simple, doesn't reflect real usage
Realistic (Good):
def test_di_resolves_service_with_transitive_dependencies() -> None:
"""Test container resolves service with multiple levels of dependencies.
Realistic scenario: API service depends on database service and logger,
database depends on connection pool. Tests that all levels resolve correctly.
"""
class ConnectionPool:
pass
class DatabaseService:
def __init__(self, pool: ConnectionPool) -> None:
self.pool = pool
class LoggerService:
pass
class ApiService:
def __init__(self, db: DatabaseService, logger: LoggerService) -> None:
self.db = db
self.logger = logger
container = Container()
container.register_singleton(ConnectionPool, ConnectionPool)
container.register_singleton(DatabaseService, DatabaseService)
container.register_singleton(LoggerService, LoggerService)
container.register_transient(ApiService, ApiService)
# Resolve complete tree
api = container.resolve(ApiService)
assert isinstance(api, ApiService)
assert isinstance(api.db, DatabaseService)
assert isinstance(api.db.pool, ConnectionPool)
assert isinstance(api.logger, LoggerService)
๐ 4. Integration Test Patterns¶
4.1 Testing Component Interactions¶
Pattern: Validate how components communicate and work together.
def test_bootstrap_runtime_integrates_with_container() -> None:
"""Test bootstrap runtime correctly initializes container."""
# Real components
container = Container()
plan = BootstrapPlanBuilder.create().with_validation(True).build()
runtime = BootstrapRuntime()
# Execute integration
runtime.bootstrap(container, plan)
# Verify interaction: metrics recorded
assert runtime.metrics.total_registrations >= 0
assert runtime.metrics.preparation_time_ms >= 0
4.2 Testing Workflows¶
Pattern: Multi-step scenarios from start to end.
def test_application_launch_workflow() -> None:
"""Test complete application launch workflow.
Validates: module resolution โ config reading โ bootstrap โ execution
"""
@application(scan_packages=["test_packages"])
class TestApp(ApplicationBase):
executed = False
def run(self) -> None:
TestApp.executed = True
# Setup module
import sys
sys.modules["app_module"] = type(sys)("app_module")
sys.modules["app_module"].TestApp = TestApp
try:
# Execute workflow through public API
ApplicationRuntime.launch("app_module:TestApp")
# Verify all steps succeeded
assert TestApp.executed is True
finally:
del sys.modules["app_module"]
4.3 Testing Error Propagation¶
Pattern: Errors from one component should propagate correctly.
def test_bootstrap_failure_propagates_through_container() -> None:
"""Test that bootstrap errors are properly propagated.
When bootstrap fails (e.g., validation error), error should
propagate through container and be available to caller.
"""
container = Container()
# Create plan with validation enabled
plan = BootstrapPlanBuilder.create().with_validation(True).build()
# Create runtime that will detect issues
runtime = BootstrapRuntime()
# If plan has dependency issues, should raise
# (Exact behavior depends on plan content)
try:
runtime.bootstrap(container, plan)
# Verify bootstrap completed
except BootstrapError as e:
# Verify error is properly propagated
assert "dependency" in str(e).lower()
4.4 Testing Resource Management¶
Pattern: Verify resources are properly created, used, and cleaned up.
def test_singleton_lifecycle_integration() -> None:
"""Test singleton services are created once and reused.
Verifies the full singleton lifecycle: creation on first resolution,
reuse on subsequent resolutions, cleanup when container closes.
"""
class SingletonService:
instance_count = 0
def __init__(self) -> None:
SingletonService.instance_count += 1
container = Container()
container.register_singleton(SingletonService, SingletonService)
# First resolution creates instance
service1 = container.resolve(SingletonService)
assert SingletonService.instance_count == 1
# Second resolution reuses
service2 = container.resolve(SingletonService)
assert service1 is service2
assert SingletonService.instance_count == 1
๐ 5. Integration Test Organization¶
5.1 File Structure¶
tests/integration/
โโโ conftest.py # Shared integration fixtures
โโโ di/
โ โโโ test_bootstrap_di_workflow.py # Bootstrap workflows
โ โโโ test_container_integration.py # Container interactions
โ โโโ test_di_lifecycle.py # Service lifecycle
โโโ reflection/
โ โโโ test_reflection_workflow.py # Reflection workflows
โ โโโ test_scanning_integration.py # Package scanning
โโโ cross_system/
โโโ test_di_reflection_integration.py # DI + Reflection
โโโ test_annotation_integration.py # Annotation processing
5.2 Naming Convention¶
Pattern: test_component1_with_component2_integration() or test_workflow_name()
test_bootstrap_with_container_integration()
test_di_resolves_transitive_dependencies()
test_application_launch_workflow()
test_reflection_scanning_with_filtering()
๐งช 6. Success Criteria¶
- All integration tests use real implementations (no internal mocking)
- No shared state between integration tests
- Each test validates end-to-end workflow
- Tests use dedicated resources (no cross-test sharing)
- All integration tests have explanatory docstrings
- Integration tests run <500ms each
- Tests fail only when actual integration behavior breaks
๐ 7. Progress Tracking¶
Legend: [x] = complete, [ ] = pending
Initial Audit¶
- Audit existing integration tests for violations
- Identify tests using mocks inappropriately
- Identify shared state patterns
Refactoring¶
- Remove inappropriate mocking
- Isolate test state
- Add descriptive docstrings
- Reorganize resources
Verification¶
- Run full integration test suite
- Verify execution time <500ms per test
- Code review
๐ 8. Changelog¶
- 2025-10-18: Initial integration test quality standards TD created
๐ 9. Glossary¶
- Integration Test: Multi-component test validating interactions (<500ms)
- End-to-End Workflow: Complete scenario from entry to exit
- Resource: Test data (code samples, configuration, test modules)
- Fixture: Test helper that sets up test environment
- Real Implementation: Actual component code, not mocked
- External Dependency: Service outside the library (databases, APIs)