📘 Technical Design 025 — Unit Test Quality Standards¶

🧭 1. Summary¶

This technical design establishes prescriptive standards for unit tests in tests/unit/. It implements universal principles from TD-024 with concrete rules, patterns, and examples specific to unit testing. The design includes a module-by-module migration strategy prioritized by issue severity.

Unit tests validate individual components in isolation (<100ms execution, focused scope, prefer real implementations).

📚 2. Context¶

See TD-024 (Meta) for full context. This TD focuses on:

• Current Pain Points:
• ~33% of tests call private methods
• 22 inline fixtures scattered across suite

• Tests brittle to non-behavioral changes

• Scope: tests/unit/ (580 tests across reflection, DI, markers, annotations)

• Approach: Module-by-module migration with high-priority targets

🎯 3. Unit Test Standards¶

3.1 Rule: Test Public APIs Only¶

Statement: NEVER call _private() or _protected() methods.

Bad Example - tests/unit/reflection/test_high_fidelity_components.py:268:

def test_attempt_direct_resolution_success(self) -> None:
    """Test _attempt_direct_resolution with successful resolution."""

    # ❌ FORBIDDEN: Testing private method directly
    hints, priority = self.provider._attempt_direct_resolution(
        test_func, {}, f"{test_func.__module__}.{test_func.__qualname__}"
    )
    assert hints is not None

Problem: - Changes to _attempt_direct_resolution signature break test - Implementation details leak into test - No guarantee behavior is actually tested

Good Example - tests/unit/di/application/test_application_decorator.py:11:

def test_decorator_attaches_metadata_correctly() -> None:
    """Test that @application decorator attaches configuration metadata."""

    @application(scan_packages=["myapp.services"])
    class TestApp(ApplicationBase):
        def run(self) -> None:
            pass

    # ✅ GOOD: Test through public API (decorator)
    annotation = find_marker(TestApp, ApplicationMarker)
    assert annotation is not None
    assert annotation.config.scan_packages == ["myapp.services"]

Benefit: - Tests public API contract - Implementation changes don't break tests - Clear what's being validated

3.2 Rule: Behavior-Based Assertions¶

Statement: Assert observable outcomes, never internal state.

Bad Pattern:

# ❌ FORBIDDEN: Checking internal state
def test_caching():
    result1 = provider.for_callable(func)
    result2 = provider.for_callable(func)

    assert result1 is result2
    assert len(provider._cache) == 1  # ❌ Testing _cache internals
    assert isinstance(provider._validator, RegistrationValidator)  # ❌ Type check on private

def test_dependency_injection():
    container.register(Service, Service)

    assert "Service" in container._registrations  # ❌ Checking internal dict
    assert len(container._instances) == 0  # ❌ Checking internal state

Good Pattern:

# ✅ GOOD: Observable behavior
def test_caching_returns_same_instance():
    """Test that cached results are reused."""
    result1 = provider.for_callable(func)
    result2 = provider.for_callable(func)
    assert result1 is result2  # Observable: same instance returned

def test_dependency_injection():
    """Test that registered services can be resolved."""
    container.register(Service, Service)

    # Observable: service resolves correctly
    resolved = container.resolve(Service)
    assert isinstance(resolved, Service)

    # Observable: error when dependency missing
    with pytest.raises(DiResolutionError):
        container.resolve(MissingService)

3.3 Rule: Fixture Organization¶

Statement: - Centralize shared fixtures in tests/fixtures/ - Localize test-specific fixtures in test file (when needed) - Never duplicate fixtures across multiple files

Bad Pattern:

# ❌ test_file_1.py
@pytest.fixture
def simple_container():
    return Container()

# ❌ test_file_2.py
@pytest.fixture
def simple_container():  # Same fixture duplicated!
    return Container()

Good Pattern:

# ✅ tests/fixtures/di/container.py
@pytest.fixture
def simple_container() -> Container:
    """Provide a simple DI container for testing."""
    return Container()

# ✅ tests/unit/di/test_container.py
from tests.fixtures.di.container import simple_container

def test_container_registration(simple_container):
    """Test container registration."""
    simple_container.register(Service, Service)

Fixture Organization:

3.4 Rule: Clear, Descriptive Naming¶

Statement: Test names should answer "what is being tested?" and "when?"

Pattern: test_component_behavior_when_condition()

Good Examples:

test_container_resolves_registered_service()
test_container_raises_error_when_dependency_missing()
test_decorator_preserves_class_identity_when_applied()
test_provider_attempts_fallback_when_direct_resolution_fails()
test_cache_returns_same_instance_when_called_twice()

Bad Examples:

test_container()  # What about container?
test_error()  # What error?
test_success()  # Success of what?
test_case_1()  # No information about scenario
test_failure_handling()  # Too vague

3.5 Rule: Every Test Has Explanatory Docstring¶

Statement: Every test must have a docstring explaining "why this test exists."

def test_container_validates_circular_dependencies() -> None:
    """Test that container detects circular dependency chains.

    This test ensures the container's dependency validator catches cycles
    like A → B → C → A and raises BootstrapError before registration completes.
    Critical for preventing infinite recursion at runtime.
    """
    # Test implementation

Why: - Documents expected behavior - Explains edge cases or non-obvious scenarios - Helps future maintainers understand requirements

3.6 Rule: Use AAA Pattern¶

Statement: Structure all tests as Arrange-Act-Assert.

def test_example() -> None:
    """Test example behavior."""

    # ARRANGE: Set up test data and fixtures
    container = make_container()
    service = MockService()

    # ACT: Perform the action being tested
    container.register(MockService, service)
    resolved = container.resolve(MockService)

    # ASSERT: Verify outcomes
    assert resolved is service

📐 4. Unit Test Patterns¶

4.1 Testing Public APIs¶

Pattern: Test through defined public interface.

def test_reflection_context_inspects_callable() -> None:
    """Test that context inspects callable types."""

    def sample_func(x: int) -> str:
        return str(x)

    # Public API: inspect_callable
    context = ReflectionContext()
    info = context.inspect_callable(sample_func)

    # Observable behavior: returns CallableInfo with correct hints
    assert info.hints["x"] is int
    assert info.hints["return"] is str

4.2 Testing Error Conditions¶

Pattern: Use pytest.raises() for expected exceptions.

def test_container_raises_error_when_dependency_missing() -> None:
    """Test container raises DiResolutionError for missing dependencies."""

    container = Container()

    # Act & Assert: verify exception is raised
    with pytest.raises(DiResolutionError, match="Cannot resolve MissingService"):
        container.resolve(MissingService)

4.3 Testing Observable State Changes¶

Pattern: Assert results, not internal state.

def test_registration_makes_service_resolvable() -> None:
    """Test that registered services become resolvable."""

    container = Container()
    service_instance = Service()

    # Register service
    container.register(Service, service_instance, Lifecycle.SINGLETON)

    # Observable: service can be resolved
    resolved = container.resolve(Service)
    assert resolved is service_instance

    # Observable: resolved again gets same instance (singleton)
    resolved2 = container.resolve(Service)
    assert resolved is resolved2

4.4 Testing with Fixtures¶

Pattern: Use fixtures for common setup.

@pytest.fixture
def provider_with_context() -> tuple[TypeHintProvider, ReflectionContext]:
    """Provide a type hint provider with reflection context."""
    context = ReflectionContext()
    provider = SimpleTypeHintProvider(context)
    return provider, context

def test_provider_resolves_builtin_types(
    provider_with_context: tuple[TypeHintProvider, ReflectionContext]
) -> None:
    """Test provider correctly handles builtin types."""
    provider, context = provider_with_context

    hints = provider.for_callable(builtin_func)
    assert hints["param"] is int

🔄 5. Migration Strategy¶

5.1 Priority Modules¶

Audit Results:

5.2 Migration Process per Module¶

For each target module:

1. Audit: Count private API calls

   grep "provider\._\|context\._\|obj\._" tests/unit/reflection/test_*.py
   

2. Identify Patterns: What private methods are being tested?

3. Internal resolution steps?
4. Cache checking?

5. Type checking on private fields?

6. Refactor: Convert to test public APIs

7. Test the public method that uses private logic
8. Assert observable outcome

9. Remove direct private calls

10. Fill Gaps: Add tests for scenarios only testable via private API

11. If scenario can't be tested publicly: likely code smell

12. Consider exposing behavior through public API

13. Verify: Ensure all tests pass

14. make test for full suite
15. No new violations introduced

5.3 Example Refactoring¶

Before (testing private method):

def test_attempt_direct_resolution_success(self) -> None:
    """Test _attempt_direct_resolution with successful resolution."""

    def test_func(x: int) -> str:
        return str(x)

    # ❌ Testing private method
    hints, priority = self.provider._attempt_direct_resolution(
        test_func, {}, f"{test_func.__module__}.{test_func.__qualname__}"
    )

    assert hints is not None
    assert "x" in hints
    assert hints["x"] is int

After (testing public API):

def test_provider_resolves_simple_function_types() -> None:
    """Test provider correctly resolves type hints for simple functions."""

    def sample_func(x: int) -> str:
        return str(x)

    # ✅ Testing public API (for_callable)
    hints = self.provider.for_callable(sample_func)

    # Observable: correct hints returned
    assert hints is not None
    assert "x" in hints
    assert hints["x"] is int
    assert hints["return"] is str

📊 6. Success Criteria¶

• Zero private API calls in unit tests (grep -r "._" returns 0)
• All high-priority modules migrated
• All tests have explanatory docstrings
• All fixtures centralized or documented as test-specific
• Tests follow AAA pattern consistently
• No test breaks due to non-behavioral code changes

📈 7. Progress Tracking¶

Legend: [x] = complete, [~] = in progress, [ ] = pending

Phase 1: High Fidelity Type Hints ✅ Completed¶

Module: tests/unit/reflection/high_fidelity_type_hints/

Test files (21 function-based tests): - test_type_hint_cache_manager.py: 4 tests, TypeHintCacheManager public API - test_type_hint_metrics_emitter.py: 7 tests with mock-based metric verification - test_high_fidelity_provider.py: 10 tests covering type resolution workflows

Completion status: - [x] Audit test_high_fidelity_components.py (identified 10+ private API calls, 4 test classes) - [x] Refactor into 3 focused test files with 21 function-based tests - [x] Remove all private API calls - [x] Localize test-specific fixtures - [x] Implement mock-based metrics verification - [x] Add descriptive docstrings to all tests - [x] Organize with subdirectory structure - [x] All tests passing (705/705)

Phase 2: Type Resolution Module ✅ Completed¶

Module: tests/unit/reflection/type_resolution/ - Component Tests Only

Unit test files (8 function-based tests): - test_import_resolver.py: 2 tests, component API only - test_import_resolution.py: 5 tests, component API only - test_runtime_type_resolver.py: 1 test, RuntimeTypeResolver introspection and object mapping

Integration test files (3 tests moved): - Moved to tests/integration/reflection/test_dotted_path_resolution.py: 3 tests - Tests ReflectionContext.resolve.dotted() facade method - Reason: Tests high-level context API, not component in isolation

Completion status: - [x] Audit for private API calls - 0 violations found - [x] All unit tests function-based (no test classes) - [x] All tests have descriptive docstrings - [x] Unit tests passing (8/8) ✅ - [x] Unit/integration boundary correctly identified

Phase 3: Type Hints Components ✅ Completed¶

Module: tests/unit/reflection/type_hints/ - Component Tests Only

Unit test files (3 function-based tests): - test_type_hints_annotations.py: 1 test, component API only - test_composite_providers.py: 2 tests, component behavior only

Integration test files (2 tests moved): - Moved to tests/integration/reflection/test_field_type_resolution.py: 1 test - Tests ReflectionContext.hints.for_field() facade method - Reason: Tests high-level context API - Moved to tests/integration/reflection/test_context_builder.py: 1 test - Tests ReflectionContext constructor/factory helpers used to customize metrics/type-hint behaviour - Reason: Tests builder integration, not component in isolation

Completion status: - [x] Audit for private API calls - 0 violations found - [x] All unit tests function-based (no test classes) - [x] All tests have descriptive docstrings - [x] Fixture updates applied (member_builder, null_metrics) - [x] Unit tests passing (3/3) ✅ - [x] Unit/integration boundary correctly identified

Phase 4: Scanning Module ✅ Completed¶

Module: tests/unit/reflection/scanning/ - Component Tests Only

Unit test files (3 function-based tests): - test_scanner.py: 1 test, PackageScanner direct behavior - test_scanner_component.py: 2 tests, PackageScanner component functionality

Integration test files (5 tests moved): - Moved to tests/integration/reflection/test_context_scanning_policies.py: 3 tests - Tests ReflectionContext.scan.* facade methods - Tests: error policies, public-only filtering, member-kind filtering - Reason: Tests high-level context scanning API - Moved to tests/integration/reflection/test_context_error_policies.py: 2 tests - Tests ScanError exception structure and behavior - Reason: Tests error propagation through context

Completion status: - [x] Audit for private API calls - 0 violations found - [x] All unit tests function-based (no test classes) - [x] All tests have descriptive docstrings - [x] Fixture updates applied (scanner fixture for component tests) - [x] Unit tests passing (3/3) ✅ - [x] CRITICAL: Unit/integration boundary discovery - Revealed that ReflectionContext tests should NOT be unit tests - Facade pattern recognized: context delegates to components - All context tests moved to integration - Applies to Phases 2, 3, and future phases

Phase 5: Member Information Builder ✅ Completed¶

Module: tests/unit/reflection/ - Split by Component

Unit test files (5 function-based tests):

Component: MemberBuilder (tests/unit/reflection/member_builder/): - test_member_builder.py: 4 tests, module building, type structure, function params, caching

Component: SafeCallableInvoker & TypeInstanceFactory (tests/unit/reflection/invocation/): - test_invocation.py: 1 test, constructor invocation and instance creation

Completion status: - [x] Audit for private API calls - 0 violations found - [x] All tests already function-based (no test classes) - [x] All tests have descriptive docstrings with Sphinx format - [x] Fixture consolidation applied: - Phase-specific member_builder fixture in member_builder/conftest.py - Reused shared null_metrics fixture - RuntimeTypeResolver, SafeCallableInvoker inline (test-specific) - [x] All tests passing (5/5) ✅ - [x] Tests organized by component (MemberBuilder vs Invocation)

Phase 6: Metrics & Error Handling [ ]¶

Test files (7 existing tests with test classes): - test_metrics_components.py - Metrics component functionality - test_metrics_hook.py - Metrics hook integration - test_error_payloads_schema.py - Error payload schemas

Audit checklist: - [ ] Audit for private API calls - [ ] Convert test classes to function-based tests - [ ] Add docstrings to all tests - [ ] Verify all tests pass

Phase 7: Fixture Organization & Consolidation [ ]¶

Cross-module fixture work: - [ ] Review and localize test-specific fixtures - [ ] Consolidate shared fixtures in tests/fixtures/reflection/ - [ ] Update centralized fixture imports - [ ] Document fixture usage patterns

Phase 8: Full Verification & Closure [ ]¶

• Run full test suite with all modules migrated
• Code review of all migrated modules
• Update documentation with completion status
• Archive completed TD-025

📝 8. Changelog¶

• 2025-10-19: Phase 5 (Member Information Builder) - Component Organization
• Unit tests: 5 (MemberBuilder: 4 tests, Invocation: 1 test)
• Tests organized by component into separate directories:
  • tests/unit/reflection/member_builder/: 4 tests, phase-specific fixture
  • tests/unit/reflection/invocation/: 1 test, shared fixtures
• Applied fixture consolidation:
  • Created member_builder/conftest.py with local fixture
  • Reused shared null_metrics fixture
  • Test-specific components inline (RuntimeTypeResolver, SafeCallableInvoker)
• All tests compliant from start: function-based, documented, 0 private API calls

• All 5 tests passing ✅

• 2025-10-19: RuntimeTypeResolver test consolidation

• Renamed: test_type_resolver.py → test_runtime_type_resolver.py
• Clarified test function name for better intent
• Confirmed test_invocation.py uses RuntimeTypeResolver as helper (not subject of test)

• Confirmed test_type_hints_annotations.py uses RuntimeTypeResolver as helper (not subject of test)

• 2025-10-19: CRITICAL: Unit/Integration boundary reclassification

• Discovery: ReflectionContext tests incorrectly classified as unit tests
• Issue: Testing facades (coordinators/delegates) requires mocking internals = implementation testing
• Solution: Moved all ReflectionContext tests to integration layer
• Affected phases: 2, 3, 4 - total 10 tests reclassified
• Details:
  • Phase 2: 3 tests moved (test_resolve_dotted_success.py → integration)
  • Phase 3: 2 tests moved (test_type_hints_fields.py + builder test → integration)
  • Phase 4: 5 tests moved (context/*.py → integration)
• Lesson: Updated TD-024 with facade pattern recognition and unit/integration classification criteria

• Going Forward: Future phases must apply this boundary correctly from start

• 2025-10-19: Phase 4 (Scanning Module) - Reclassified

• Unit tests: 3 (PackageScanner component)
• Reclassified to integration: 5 (ReflectionContext tests)
• Created scanner fixture for component tests

• Updated documentation with unit/integration classification guidance

• 2025-10-19: Phase 3 (Type Hints Components) - Reclassified

• Unit tests: 3 (component behavior only)

• Reclassified to integration: 2 (ReflectionContext tests)

• 2025-10-19: Phase 2 (Type Resolution Module) - Reclassified

• Unit tests: 8 (component behavior only)

• Reclassified to integration: 3 (ReflectionContext tests)

• 2025-10-19: Phases 1-4 Summary (After Reclassification)

• Total unit tests: 35 (Phase 1: 21, Phase 2: 8, Phase 3: 3, Phase 4: 3)
• Total reclassified to integration: 10 (Phases 2, 3, 4)
• All comply with public API only rule
• All function-based, no test classes

• All have descriptive Sphinx docstrings

• 2025-10-19: TD-024 Updated with Patterns Documentation

• Added Section 5.6: Unit vs Integration classification with facade pattern recognition

• Added Section 5.7: Successful patterns from Phases 1-4

  • Real implementations over mocks
  • Fixture factory pattern
  • Progressive fixture scoping
  • Sphinx docstring requirement

• 2025-10-18: Initial unit test quality standards TD created

📖 9. Glossary¶

• Unit Test: Fast (<100ms) test of single component with own logic/behavior
• Integration Test: Cross-component workflow test (<500ms) validating how components work together
• Facade: Component whose primary responsibility is coordinating/delegating to other components
• Component: Class/module with distinct responsibility and observable behavior (not just delegation)
• Private API: Methods/properties starting with underscore (_internal)
• Public API: Methods/properties without underscore
• Observable Behavior: Outcomes visible through public interface
• Implementation Testing: Verifying how something works (internal details), not what it does (behavior)
• Fixture: Test helper (factory, mock, builder) that sets up test data
• AAA Pattern: Arrange-Act-Assert structure for tests