Skip to content

๐Ÿ“˜ Technical Design 026 โ€” Integration Test Quality Standards

Field Value
ID TD-026
Title Integration Test Quality Standards
Status Active
Owner Contributors
Audience Contributors, test developers
Created 2025-10-18
Last Updated 2025-10-18
Reviewers TBD
Approvers TBD
Tracking Phase of DI Roadmap
Related TD-024 (Meta), TD-025 (Unit)

๐Ÿงญ 1. Summary

This technical design establishes prescriptive standards for integration tests in tests/integration/. It implements universal principles from TD-024 with patterns specific to integration testing, which validates component interactions and end-to-end workflows using real implementations.

Integration tests validate how components work together (<500ms execution, realistic scenarios, no mocking except external dependencies).

๐Ÿ“š 2. Context

See TD-024 (Meta) for full context. This TD focuses on integration-specific requirements:

  • Scope: tests/integration/ - multi-component workflows
  • Characteristics: Real implementations, longer execution, cross-component
  • Approach: Apply universal rules + integration-specific patterns
  • Priority: Lower than unit tests (currently audit needed)

๐ŸŽฏ 3. Integration Test Standards

3.1 Rule: Use Real Implementations

Statement: Integration tests use REAL components, not mocks (except external services).

Principle: Integration tests validate actual component interactions, not mocked ones.

Bad Pattern:

# โŒ FORBIDDEN: Mocking internal components in integration test
def test_di_workflow() -> None:
    """Test DI workflow."""

    # โŒ WRONG: Mocking Container defeats integration test purpose
    mock_container = Mock(spec=Container)
    mock_container.resolve.return_value = MockService()

    bootstrap_runtime = BootstrapRuntime()
    bootstrap_runtime.bootstrap(mock_container, plan)

    # This isn't really testing integration!

Good Pattern:

# โœ… GOOD: Use real Container
def test_di_bootstrap_workflow() -> None:
    """Test complete DI bootstrap workflow."""

    # Real implementations
    container = Container()
    plan = BootstrapPlanBuilder.create().scan_packages(config).build()

    bootstrap_runtime = BootstrapRuntime()
    bootstrap_runtime.bootstrap(container, plan)  # Real interaction

    # Verify actual behavior
    service = container.resolve(ServiceType)
    assert isinstance(service, ServiceType)

Exception: External services (databases, APIs, file systems)

# โœ… OK: Mocking external services
def test_application_with_external_api() -> None:
    """Test application behavior with external API."""

    with patch("requests.get") as mock_api:  # External service: OK to mock
        mock_api.return_value = MockResponse({"status": "ok"})

        app = Application()
        result = app.fetch_data()

        assert result["status"] == "ok"

3.2 Rule: No Shared State Between Tests

Statement: Each integration test runs in complete isolation. No test affects others.

Bad Pattern:

# โŒ FORBIDDEN: Shared state
global_container = None

def setup_module():
    """Set up shared container for all tests."""
    global global_container
    global_container = Container()
    global_container.register(Service, Service)

def test_workflow_1():
    """Test first workflow."""
    service = global_container.resolve(Service)
    # โŒ Test depends on setup_module state

def test_workflow_2():
    """Test second workflow."""
    # โŒ If test_workflow_1 modifies state, this test is affected

Good Pattern:

# โœ… GOOD: Each test isolated
def test_workflow_1() -> None:
    """Test first workflow."""

    # Fresh setup for this test only
    container = Container()
    container.register(Service, Service)

    service = container.resolve(Service)
    assert isinstance(service, Service)

def test_workflow_2() -> None:
    """Test second workflow."""

    # Completely independent setup
    container = Container()
    container.register(Service, Service)
    container.register(OtherService, OtherService)

    service = container.resolve(Service)
    other = container.resolve(OtherService)
    assert isinstance(service, Service)
    assert isinstance(other, OtherService)

Using Fixtures for Isolation:

@pytest.fixture
def bootstrap_container() -> Container:
    """Provide a fresh container for each integration test."""
    container = Container()
    container.register(Service, Service)
    return container

def test_workflow_1(bootstrap_container: Container) -> None:
    """Test workflow 1 with isolated container."""
    service = bootstrap_container.resolve(Service)
    assert isinstance(service, Service)

def test_workflow_2(bootstrap_container: Container) -> None:
    """Test workflow 2 with its own isolated container."""
    service = bootstrap_container.resolve(Service)
    assert isinstance(service, Service)

3.3 Rule: End-to-End Workflow Testing

Statement: Integration tests validate complete workflows from entry to exit.

Example:

def test_application_bootstrap_and_execution() -> None:
    """Test complete application lifecycle: bootstrap, resolve, execute."""

    # 1. Setup application configuration
    @application(scan_packages=["test_services"])
    class TestApp(ApplicationBase):
        def run(self) -> None:
            pass

    # 2. Create module
    import sys
    sys.modules["test_app"] = type(sys)("test_app")
    sys.modules["test_app"].TestApp = TestApp

    try:
        # 3. Execute through public API
        ApplicationRuntime.launch("test_app:TestApp")

        # 4. Verify outcome: no exceptions, workflow completed
        # Additional assertions about state after execution
    finally:
        del sys.modules["test_app"]

3.4 Rule: Independent Test Resources

Statement: Each integration test suite has dedicated resources with no cross-test sharing.

Bad Pattern:

tests/resources/
โ”œโ”€โ”€ shared_services/  # โŒ Shared across multiple test suites
โ”œโ”€โ”€ test_di/          # Uses shared_services
โ””โ”€โ”€ test_reflection/  # Also uses shared_services (coupling!)

Good Pattern:

tests/resources/
โ”œโ”€โ”€ di/
โ”‚   โ”œโ”€โ”€ bootstrap/     # Resources for bootstrap tests
โ”‚   โ”œโ”€โ”€ injection/     # Resources for injection tests
โ”‚   โ””โ”€โ”€ scanning/      # Resources for scanning tests
โ””โ”€โ”€ integration/
    โ””โ”€โ”€ di_reflection/ # Resources for DI+Reflection integration

3.5 Rule: Document Why Integration Test Exists

Statement: Explain what cross-component behavior is being validated.

def test_di_with_reflection_integration() -> None:
    """Test DI container integrates correctly with reflection system.

    Validates that the DI container uses reflection to:
    1. Scan packages for annotated services
    2. Inspect service constructors for dependencies
    3. Build dependency graphs correctly
    4. Resolve services with transitive dependencies

    This is an integration test of DI + Reflection subsystems.
    """
    # Implementation

3.6 Rule: Realistic Scenarios

Statement: Test realistic use cases, not artificial scenarios.

Artificial (Bad):

def test_container_with_empty_service() -> None:
    """Test container with minimal service."""

    class EmptyService:
        pass

    container = Container()
    container.register(EmptyService, EmptyService)
    service = container.resolve(EmptyService)

    # Too simple, doesn't reflect real usage

Realistic (Good):

def test_di_resolves_service_with_transitive_dependencies() -> None:
    """Test container resolves service with multiple levels of dependencies.

    Realistic scenario: API service depends on database service and logger,
    database depends on connection pool. Tests that all levels resolve correctly.
    """

    class ConnectionPool:
        pass

    class DatabaseService:
        def __init__(self, pool: ConnectionPool) -> None:
            self.pool = pool

    class LoggerService:
        pass

    class ApiService:
        def __init__(self, db: DatabaseService, logger: LoggerService) -> None:
            self.db = db
            self.logger = logger

    container = Container()
    container.register_singleton(ConnectionPool, ConnectionPool)
    container.register_singleton(DatabaseService, DatabaseService)
    container.register_singleton(LoggerService, LoggerService)
    container.register_transient(ApiService, ApiService)

    # Resolve complete tree
    api = container.resolve(ApiService)

    assert isinstance(api, ApiService)
    assert isinstance(api.db, DatabaseService)
    assert isinstance(api.db.pool, ConnectionPool)
    assert isinstance(api.logger, LoggerService)

๐Ÿ“ 4. Integration Test Patterns

4.1 Testing Component Interactions

Pattern: Validate how components communicate and work together.

def test_bootstrap_runtime_integrates_with_container() -> None:
    """Test bootstrap runtime correctly initializes container."""

    # Real components
    container = Container()
    plan = BootstrapPlanBuilder.create().with_validation(True).build()
    runtime = BootstrapRuntime()

    # Execute integration
    runtime.bootstrap(container, plan)

    # Verify interaction: metrics recorded
    assert runtime.metrics.total_registrations >= 0
    assert runtime.metrics.preparation_time_ms >= 0

4.2 Testing Workflows

Pattern: Multi-step scenarios from start to end.

def test_application_launch_workflow() -> None:
    """Test complete application launch workflow.

    Validates: module resolution โ†’ config reading โ†’ bootstrap โ†’ execution
    """

    @application(scan_packages=["test_packages"])
    class TestApp(ApplicationBase):
        executed = False

        def run(self) -> None:
            TestApp.executed = True

    # Setup module
    import sys
    sys.modules["app_module"] = type(sys)("app_module")
    sys.modules["app_module"].TestApp = TestApp

    try:
        # Execute workflow through public API
        ApplicationRuntime.launch("app_module:TestApp")

        # Verify all steps succeeded
        assert TestApp.executed is True
    finally:
        del sys.modules["app_module"]

4.3 Testing Error Propagation

Pattern: Errors from one component should propagate correctly.

def test_bootstrap_failure_propagates_through_container() -> None:
    """Test that bootstrap errors are properly propagated.

    When bootstrap fails (e.g., validation error), error should
    propagate through container and be available to caller.
    """

    container = Container()

    # Create plan with validation enabled
    plan = BootstrapPlanBuilder.create().with_validation(True).build()

    # Create runtime that will detect issues
    runtime = BootstrapRuntime()

    # If plan has dependency issues, should raise
    # (Exact behavior depends on plan content)
    try:
        runtime.bootstrap(container, plan)
        # Verify bootstrap completed
    except BootstrapError as e:
        # Verify error is properly propagated
        assert "dependency" in str(e).lower()

4.4 Testing Resource Management

Pattern: Verify resources are properly created, used, and cleaned up.

def test_singleton_lifecycle_integration() -> None:
    """Test singleton services are created once and reused.

    Verifies the full singleton lifecycle: creation on first resolution,
    reuse on subsequent resolutions, cleanup when container closes.
    """

    class SingletonService:
        instance_count = 0

        def __init__(self) -> None:
            SingletonService.instance_count += 1

    container = Container()
    container.register_singleton(SingletonService, SingletonService)

    # First resolution creates instance
    service1 = container.resolve(SingletonService)
    assert SingletonService.instance_count == 1

    # Second resolution reuses
    service2 = container.resolve(SingletonService)
    assert service1 is service2
    assert SingletonService.instance_count == 1

๐Ÿ“‹ 5. Integration Test Organization

5.1 File Structure

tests/integration/
โ”œโ”€โ”€ conftest.py                           # Shared integration fixtures
โ”œโ”€โ”€ di/
โ”‚   โ”œโ”€โ”€ test_bootstrap_di_workflow.py     # Bootstrap workflows
โ”‚   โ”œโ”€โ”€ test_container_integration.py     # Container interactions
โ”‚   โ””โ”€โ”€ test_di_lifecycle.py              # Service lifecycle
โ”œโ”€โ”€ reflection/
โ”‚   โ”œโ”€โ”€ test_reflection_workflow.py       # Reflection workflows
โ”‚   โ””โ”€โ”€ test_scanning_integration.py      # Package scanning
โ””โ”€โ”€ cross_system/
    โ”œโ”€โ”€ test_di_reflection_integration.py # DI + Reflection
    โ””โ”€โ”€ test_annotation_integration.py    # Annotation processing

5.2 Naming Convention

Pattern: test_component1_with_component2_integration() or test_workflow_name()

test_bootstrap_with_container_integration()
test_di_resolves_transitive_dependencies()
test_application_launch_workflow()
test_reflection_scanning_with_filtering()

๐Ÿงช 6. Success Criteria

  • All integration tests use real implementations (no internal mocking)
  • No shared state between integration tests
  • Each test validates end-to-end workflow
  • Tests use dedicated resources (no cross-test sharing)
  • All integration tests have explanatory docstrings
  • Integration tests run <500ms each
  • Tests fail only when actual integration behavior breaks

๐Ÿ“ˆ 7. Progress Tracking

Legend: [x] = complete, [ ] = pending

Initial Audit

  • Audit existing integration tests for violations
  • Identify tests using mocks inappropriately
  • Identify shared state patterns

Refactoring

  • Remove inappropriate mocking
  • Isolate test state
  • Add descriptive docstrings
  • Reorganize resources

Verification

  • Run full integration test suite
  • Verify execution time <500ms per test
  • Code review

๐Ÿ“ 8. Changelog

  • 2025-10-18: Initial integration test quality standards TD created

๐Ÿ“– 9. Glossary

  • Integration Test: Multi-component test validating interactions (<500ms)
  • End-to-End Workflow: Complete scenario from entry to exit
  • Resource: Test data (code samples, configuration, test modules)
  • Fixture: Test helper that sets up test environment
  • Real Implementation: Actual component code, not mocked
  • External Dependency: Service outside the library (databases, APIs)