🚀 Dependency Injection System Roadmap¶

📢 Important Update (2025-12-22)¶

The loryth.common.reflection module has been removed from the codebase. The DI system now uses Python standard library introspection (inspect, typing.get_type_hints(), pkgutil) for all dependency analysis and package scanning. Phase 10 (Reflection Integration) is no longer applicable.

1. 📊 Status Dashboard¶

Overall Progress: 57% Implemented (8/14 phases implemented)

Recent Milestones: - ✅ 2025-10-18: TD-022 completed - Unified runtime marker system successfully implemented (702 tests passing) - ✅ 2025-10-12: TD-021 completed - Application entry point with zero-boilerplate startup - ✅ 2025-10-12: TD-019 completed - Configuration binding with multi-format support - ✅ 2025-10-11: TD-013 completed - Rule-based package scanning replacing processor pattern - ✅ 2025-10-10: TD-020 completed - External bootstrap runtime with idempotent registration - ✅ 2025-10-08: TD-012 completed - Spring Boot-style annotations with lifecycle control - ✅ 2025-10-06: TD-011 completed - Unified injection system with constructor/property/method support

Status Legend: - ✅ Implemented - Implementation finished and tested - 🟢 Accepted - Approved for implementation - 📝 Draft - Design under development/review - ⏳ Planned - Not yet started (no design document)

graph LR
    P1[Phase 1<br/>Container API] --> P3[Phase 3<br/>Injection System]
    P3 --> P6[Phase 6<br/>Configuration Binding]
    P1 --> P2[Phase 2<br/>Bootstrap Runtime]
    P1 --> P4[Phase 4<br/>Enhanced Annotations]
    P2 --> P5[Phase 5<br/>Module Scanning]
    P4 --> P5
    P5 --> P7[Phase 7<br/>Application Entry Point]

    P13[Phase 13<br/>Unit Test Quality]
    P8[Phase 8<br/>Test Resource Cleanup]
    P14[Phase 14<br/>Integration Test Quality]
    P12[Phase 12<br/>Test Quality Meta] 

    P14 --> P12
    P13 --> P12
    P8 --> P12
    P12 --> P9[Phase 9<br/>Type Hints Modularization]
    P9 --> P10[Phase 10<br/>Reflection Integration]
    P11[Phase 11<br/>Runtime Markers] --> P10


    classDef implemented fill:none,stroke:#4caf50,stroke-width:3px,color:#4caf50
    classDef draft fill:none,stroke:#ff9800,stroke-width:2px,color:#ff9800
    classDef planned fill:none,stroke:#9e9e9e,stroke-width:2px,color:#9e9e9e

    class P1,P2,P3,P4,P5,P6,P7,P11 implemented
    class P8,P12,P13,P14 draft
    class P9,P10 planned

2. 📋 Overview¶

This document outlines the comprehensive roadmap for enhancing the dependency injection (DI) system in loryth.common. The roadmap prioritizes DI development to validate and provide feedback on the reflection system, while building a Spring Boot-inspired but Python-native dependency injection framework.

Strategic Goals: 1. Validate Reflection System - Use DI as primary client to exercise and validate reflection capabilities 2. Spring Boot Patterns - Provide familiar annotation-driven development experience 3. Python-Native Design - Embrace Python idioms and constraints (no Java-isms) 4. Zero-Boilerplate - Automatic service discovery and wiring with minimal configuration

3. 🔍 Current State¶

DI System¶

• ✅ Unified Container: Single DiContainer with comprehensive registration API
• ✅ External Bootstrap: BootstrapRuntime orchestrates initialization with plan-based approach
• ✅ Idempotent Registration: O(1) duplicate detection with conflict validation
• ✅ Injection System: Constructor, property, and method injection with unified pipeline
• ✅ Enhanced Annotations: @service, @component, @configuration/@bean decorators
• ✅ Module Scanning: BootstrapPlanBuilder with rule-based service discovery

Reflection System¶

• Highly modular with swappable components
• Two TypeHintProviders: SimpleTypeHintProvider and HighFidelityTypeHintProvider
• Sophisticated forward reference resolution and caching
• Package scanner with predicate-based filtering

Test Organization¶

• Scattered test resources with unclear naming (m5_*_pkg/ patterns)
• Mixed organization between unit/integration/regression tests
• Needs: Feature-based reorganization and consolidation

4. 🛣️ Roadmap Phases¶

Phase 1 (Container API): Container Consolidation & Public API ✅ COMPLETE¶

Objective: Unified container with rich registration and resolution API

Key Capabilities: - Type-safe service registration (singleton, transient, factory, instance) - Generic type resolution with runtime checking: container.resolve[T](type[T]) -> T - Provider-based lazy loading architecture - Thread-safe operations with internal locking

Deliverables: - ✅ Single DiContainer replacing dual implementation approach - ✅ Comprehensive registration API with lifecycle control - ✅ Generic resolution with type safety

Status: Completed during initial DI system development

Phase 2 (Bootstrap): External Bootstrap Runtime ✅ COMPLETE (TD-020)¶

Original Goal: Automatic bootstrapping with manual override capability

What Changed: Bootstrap orchestration was externalized from DiContainer into dedicated BootstrapRuntime to improve separation of concerns, testability, and integration with scanning infrastructure.

Actual Implementation: - Container no longer owns bootstrap logic (removed DiContainer.bootstrap(), auto-bootstrap) - Plan-based bootstrap with type-safe RegistrationStep protocol - Idempotent registration semantics enable safe re-bootstrap scenarios - Comprehensive observability via ContainerMetrics and BootstrapMetrics

Key Deliverables: - ✅ BootstrapRuntime orchestrator with three-phase execution (registration, validation, preparation) - ✅ BootstrapPlan with immutable registration steps - ✅ Container idempotency with O(1) conflict detection via RegistrationKey - ✅ DiContainer.prepare() for explicit eager singleton resolution

Breaking Changes: - Removed: DiContainer.bootstrap(), auto-bootstrap on first resolve, resolve_all() - Changed: Registration APIs now idempotent (duplicate detection) - Added: container.prepare() for fail-fast singleton hydration

Why Changed: - TD-013 scanning requires inspectable plans before bootstrap - Testability improved with external orchestration - Better separation between container (registry) and bootstrap (lifecycle) concerns

Example:

# New pattern - explicit bootstrap workflow
runtime = BootstrapRuntime()
container = DiContainer()
plan = BootstrapPlan(registrations=(
    SingletonRegistrationStep(IService, Service),
))
runtime.bootstrap(container, plan)

📖 See: TD-020 External DI Bootstrap Runtime

Phase 3 (Injection): Injection System ✅ COMPLETE (TD-011)¶

Objective: Automatic dependency injection for constructor, property, and method scenarios

Key Deliverables: - ✅ Constructor injection with reflection-based parameter analysis (ConstructorInjector) - ✅ Property injection using Annotated[..., Autowired()] metadata (AutowiredInjector) - ✅ Method wrapping via InjectionProxyFactory for cross-cutting concerns - ✅ Unified injector pipeline with priority-based execution (InjectionManager) - ✅ Comprehensive error handling with InjectorError hierarchy - ✅ Integration with TD-020 bootstrap lifecycle

Remaining Work: - ⏳ Performance benchmarking (moved to TD-018) - ⏳ TD-012 annotation-driven auto-registration integration

Example:

# Constructor injection (automatic)
class UserService:
    def __init__(self, repo: UserRepository, logger: Logger):
        ...

# Property injection
class OrderService:
    user_service: Annotated[UserService, Autowired()]
    config: Annotated[AppConfig, Autowired()]

# Custom injectors for method wrapping
container = (DiContainerBuilder()
    .with_injector(TransactionalInjector())
    .with_injector(CachingInjector())
    .build())

📖 See: TD-011 DI Injection System (Archived - Complete)

Phase 4 (Annotations): Enhanced DI Annotations ✅ COMPLETE (TD-012)¶

Objective: Spring Boot-style declarative service registration with lifecycle control

Key Deliverables: - ✅ @service annotation with implicit singleton lifecycle - ✅ @component annotation with explicit lifecycle control (@singleton, @transient) - ✅ @configuration classes with @bean factory methods - ✅ Annotation metadata capture for TD-013 scanning compatibility - ✅ DiContainerBuilder.process_annotated_classes() API - ✅ Protocol registration for directly implemented interfaces

Remaining Work: - ⏳ Configuration file integration (YAML/JSON/TOML) - moved to TD-019

Note: TD-013 replaced the AnnotationProcessor pattern with RegistrationRule system. Annotations now feed into scanning infrastructure via built-in rules.

Example:

@service  # implicit singleton
class UserService:
    ...

@component
@transient
class RequestHandler:
    ...

@configuration
class AppConfig:
    @bean
    def database_connection(self) -> DatabaseConnection:
        ...

📖 See: TD-012 Enhanced DI Annotations

Phase 5 (Scanning): Module Scanning & Auto-Registration ✅ COMPLETE (TD-013)¶

Original Goal: Automatic service discovery through package scanning

What Changed: TD-013 replaced the TD-012 AnnotationProcessor pattern (direct container mutation) with composable RegistrationRule system that builds inspectable BootstrapPlan objects compatible with TD-020.

Actual Implementation: - BootstrapPlanBuilder with fluent API for scanning and plan building - RegistrationRule protocol for extensible annotation-to-step translation - Built-in rules for @service, @component, @configuration, @bean - Integration with reflection scanner using predicate-based filtering

Key Deliverables: - ✅ BootstrapPlanBuilder.create() with sensible defaults - ✅ ScanningConfiguration for package filtering - ✅ Union semantics - all matching rules apply to each class - ✅ Step deduplication delegated to TD-020 container idempotency

Breaking Changes: - Removed: AnnotationProcessor, InjectorBridge, AnnotationRegistry - Added: RegistrationRule protocol, BootstrapPlanBuilder fluent API

Why Changed: - TD-020 plan-based bootstrap incompatible with direct container mutation - Inspectable plans enable validation and debugging before bootstrap - Composable rules more extensible than monolithic processor

Example:

builder = BootstrapPlanBuilder.create()
config = ScanningConfiguration(
    packages=["myapp.services"],
    include_annotations=[ServiceAnnotation],
)
plan = builder.scan_packages(config).build()

runtime = BootstrapRuntime()
container = DiContainer()
runtime.bootstrap(container, plan)

Note: Application entry point (@application decorator) split into separate TD-021 for cleaner separation of concerns.

📖 See: TD-013 Bootstrap Plan Building via Package Scanning

Phase 6 (Config Binding): Configuration Binding & Sources ✅ COMPLETE (TD-019)¶

Objective: Declarative configuration binding via injectors with multi-format support

Key Deliverables: - ✅ Annotated[T, Config("path")] metadata for configuration dependencies - ✅ Integration with TD-011 injector pipeline for property/parameter injection - ✅ Multi-format configuration sources (YAML, JSON, TOML) with deterministic precedence - ✅ Configuration registry with dotted-key resolution (database.host) - ✅ Extensible loader system for custom configuration sources - ✅ Pydantic model support with validation error wrapping - ✅ Immutable deep copies for configuration safety

Example:

from typing import Annotated

from loryth.common.di.configuration import (
    BaseConfigResolver,
    Config,
    ConfigInjector,
    ConfigResolver,
    ConfigurationRegistry,
    PydanticConfigResolver,
)

@service
class DatabaseClient:
    host: Annotated[str, Config("database.host")]
    port: Annotated[int, Config("database.port")] = 5432

@configuration
class AppConfig:
    @bean
    def metrics_client(
        self,
        api_key: Annotated[str, Config("metrics.api_key")],
        endpoint: Annotated[str, Config("metrics.endpoint")],
    ) -> MetricsClient:
        return MetricsClient(api_key=api_key, endpoint=endpoint)

container = DiContainer()

registry = ConfigurationRegistry.create()
for source in (
    "config/application.yaml",
    "config/environment.yaml",
):
    registry.load(source)

resolver = PydanticConfigResolver(BaseConfigResolver(registry))
container.register_instance(ConfigResolver, resolver)
container.register_injector(ConfigInjector())

Dependencies: - Requires Phase 3 (injection system) for injector integration - Complements Phase 4 (annotations) - configuration was explicitly deferred from TD-012

Implementation Summary: - Total Tests: 82 (76 unit + 6 integration), all passing - Code Quality: 100% mypy compliance (strict mode), 100% ruff compliance - Full Test Suite: 668 tests passing (no regressions) - Implementation Date: 2025-10-12 - Files Created: 7 source files, 6 test files, 4 test resource files

📖 See: TD-019 DI Configuration Binding & Sources

Phase 7 (App Entry): Application Entry Point & Bootstrap Orchestration 🟢 ACCEPTED (TD-021)¶

Objective: Standardized application entry point with zero-boilerplate startup

Key Goals: - @application decorator for marking application entry point classes - ApplicationRuntime.launch("module:Class") coordinating scan → plan → bootstrap → run workflow - ApplicationBase contract with a single run() lifecycle method fed by constructor/autowired injection - Integration with TD-013 scanning and TD-020 bootstrap runtime metrics - Zero-boilerplate application startup via module path resolution

Example:

@application(scan_packages=["myapp.services", "myapp.repositories"])
class MyApplication(ApplicationBase):
    def __init__(self, service: MainService) -> None:
        self._service = service

    def run(self) -> None:
        # Application entry point
        self._service.start()

# Zero-boilerplate startup
if __name__ == "__main__":
    ApplicationRuntime.launch("myapp.entry:MyApplication")

Dependencies: - Requires Phase 5 (scanning) for automatic service discovery - Builds on TD-020 (bootstrap runtime) for plan execution

Note: Split from TD-013 scope for cleaner separation between scanning infrastructure and application lifecycle orchestration.

📖 See: TD-021 Application Entry Point and Bootstrap Orchestration

Phase 8 (Test Cleanup): Test Resource Organization ⏳ PLANNED (TD-014)¶

Objective: Organize test resources by feature for improved maintainability

Key Goals: - Feature-based organization (reflection/, di/, annotations/) - Consolidate scattered m5_*_pkg test resources - Independent test suites with no shared resources - Clear naming conventions

Target Structure:

tests/resources/
├── reflection/
│   ├── forward_refs/
│   ├── type_hints/
│   └── scanning/
├── di/
│   ├── injection/
│   ├── lifecycle/
│   └── scanning/
└── integration/
    └── di_reflection/

📖 See: TD-014 Test Resource Cleanup

Phase 9 (Modularization): Type Hints Provider Modularization ⏳ PLANNED¶

Objective: Extract type hints into submodule for improved readability and maintainability

Key Goals: - Extract to loryth.common.reflection.type_hints submodule - Reduce file sizes and improve navigability - Maintain backward compatibility with existing API

Target Structure:

src/loryth/common/reflection/type_hints/
├── providers/
│   ├── simple_provider.py
│   ├── high_fidelity_provider.py
│   └── composite_provider.py
└── resolution/
    ├── forward_refs.py
    └── annotation_extraction.py

Phase 10 (Integration): Reflection Components & DI Integration ⏳ PLANNED¶

Objective: Establish DI as primary client of reflection system with global context

Key Goals: - Global ReflectionContext with provider extensibility - Thread-safe global state management - DI uses reflection for scanning, annotation inspection, type resolution - Reflection remains independent (DI doesn't register providers)

Example:

from loryth.common.reflection import reflection

# Add custom providers globally
reflection.add_type_hint_provider(CustomProvider())

# DI leverages global reflection context
builder = BootstrapPlanBuilder.create()  # uses global reflection

graph LR
    subgraph Reflection
        RC[ReflectionContext]
        Scanner[Scanner]
        TypeHints[Type Hints]
    end

    subgraph DI
        Builder[PlanBuilder]
        Runtime[BootstrapRuntime]
        Container[DiContainer]
    end

    RC --> Builder
    Scanner --> Builder
    TypeHints --> Container
    Builder --> Runtime
    Runtime --> Container

    classDef reflection fill:none,stroke:#2196f3,stroke-width:2px,color:#2196f3
    classDef di fill:none,stroke:#4caf50,stroke-width:2px,color:#4caf50

    class RC,Scanner,TypeHints reflection
    class Builder,Runtime,Container di

Phase 11 (Markers): Unified Runtime Marker System 🟢 ACCEPTED (TD-022)¶

Objective: Refactor runtime metadata system from loryth.common.annotations to loryth.common.markers with improved terminology and unified discovery

Key Goals: - Eliminate terminology confusion between Python's compile-time annotations and our runtime metadata system - Unified API supporting both callable markers (classes, functions, methods) and attribute markers (via Annotated[]) - Single discovery interface for querying both marker types consistently - Complete DI system migration to new marker infrastructure - Zero compatibility shims - clean module replacement

Example:

from loryth.common.markers import Marker, attach, find_marker

class ServiceMarker(Marker):
    def __init__(self, lifecycle: str = "singleton") -> None:
        super().__init__(name="service")
        self.lifecycle = lifecycle

# Callable marker
@service
class UserRepository:
    pass

# Attribute marker
class OrderService:
    db: Annotated[DatabaseService, InjectedMarker()]

# Unified discovery
service_marker = find_marker(UserRepository, ServiceMarker)
db_marker = find_marker((OrderService, "db"), InjectedMarker)

Dependencies: - Requires Phase 1-7 completion for understanding current DI architecture - Builds on existing loryth.common.annotations patterns

Implementation Phases: 1. Foundation: Create loryth.common.markers module with core infrastructure 2. Handlers & Context: Implement concrete handlers and global context 3. DI Integration: Migrate all DI decorators and utilities to new module

📖 See: TD-022 Unified Runtime Marker System

5. 📅 Major Changes Changelog¶

This section documents significant course changes and rationale for divergences from the original plan.

TD-020: External Bootstrap Runtime (Phase 2)¶

Date: 2025-10-10 Impact: Breaking change - removed embedded bootstrap from container

What Changed: - Bootstrap orchestration moved from DiContainer to external BootstrapRuntime - Container no longer has bootstrap() method or auto-bootstrap behavior - Registration APIs made idempotent with O(1) duplicate detection - Introduced plan-based bootstrap with type-safe RegistrationStep protocol

Why Changed: 1. Separation of Concerns: Container focuses on registry/resolution, runtime handles lifecycle 2. Testability: Easier to test container without bootstrap side effects 3. TD-013 Integration: Scanning requires inspectable plans before bootstrap execution 4. Idempotency Needs: Safe re-bootstrap required for plugin/module discovery scenarios

Migration Impact: All code using container.bootstrap() must migrate to:

runtime = BootstrapRuntime()
runtime.bootstrap(container, plan)

TD-013: Rule-Based Scanning (Phase 5)¶

Date: 2025-10-11 Impact: Replaced TD-012 AnnotationProcessor pattern

What Changed: - AnnotationProcessor (direct container mutation) replaced with RegistrationRule protocol - Introduced BootstrapPlanBuilder with fluent API for plan composition - Rules produce registration steps instead of directly mutating container - Application entry point (@application) split into separate TD-021

Why Changed: 1. TD-020 Compatibility: Plan-based bootstrap requires inspectable plans, not direct mutation 2. Extensibility: Composable rules more flexible than monolithic processor 3. Debuggability: Plans can be inspected/validated before bootstrap 4. Separation of Concerns: Scanning infrastructure (TD-013) vs application lifecycle (TD-021)

Migration Impact: Code using AnnotationProcessor must migrate to BootstrapPlanBuilder:

# Before (TD-012 processor)
processor.process([UserService, EmailService])

# After (TD-013 rules)
builder.add_class(UserService).add_class(EmailService)
plan = builder.build()

TD-019: Configuration Binding (Phase 6)¶

Date: 2025-10-06 (Draft) Impact: New phase for configuration value injection

What Changed: - Configuration binding explicitly deferred from TD-012 (annotations) to separate TD-019 - Config[T, "path"] using typing.Annotated for declarative configuration dependencies - Configuration injector integrates with TD-011 property/method injection pipeline - Multi-format sources (YAML, JSON, TOML) with pluggable provider system

Why Changed: 1. Separation of Concerns: Annotation metadata (TD-012) separate from configuration binding 2. Injector Integration: Natural fit with TD-011 injector pipeline 3. Extensibility: Pluggable providers enable custom configuration sources

Status: TD-019 currently in Draft status

TD-021: Application Entry Point Split (Phase 7)¶

Date: 2025-10-12 (Accepted) Impact: New phase split from TD-013 scope

What Changed: - Application entry point (@application decorator) moved from TD-013 to separate TD-021 - ApplicationRuntime coordinator for standardized application startup - Lifecycle hooks (configure(), run()) for post-bootstrap customization

Why Changed: 1. Scope Management: TD-013 grew too large with both scanning and application lifecycle 2. Cleaner Design: Scanning infrastructure (TD-013) separate from application orchestration (TD-021) 3. Phased Implementation: Can complete scanning before tackling entry point patterns

Status: TD-021 now Accepted for implementation

6. 🔮 Future Work¶

Advanced Features¶

• Lifecycle Scopes: Request, session, application scopes beyond singleton/transient
• Configuration Integration: @ConfigValue annotation for property injection from config files
• AOP Interceptors: Cross-cutting concerns (logging, security, transactions)
• Named Registrations: Multiple implementations with qualifiers
• Conditional Registration: Environment-based or profile-based service activation

Performance & Observability¶

• Startup Performance: Optimize bootstrap time for large applications
• Memory Profiling: Efficient caching and instance management
• Metrics Dashboard: Real-time DI resolution metrics and performance monitoring
• Dependency Visualization: Graph visualization for debugging complex dependency trees

7. 🎯 Success Criteria¶

Overall Roadmap Success: 1. ✅ Phase 1-5 Complete: Container, bootstrap, injection, annotations, and scanning operational 2. ⏳ Phase 6 Planned: Configuration binding for declarative config injection 3. ⏳ Phase 7 Planned: Application entry point with zero-boilerplate startup 4. ⏳ Phase 8-10 Planned: Organization and integration improvements

Quality Metrics: - Sub-second bootstrap for typical applications (<100 services) - Comprehensive test coverage (unit, integration, regression) - Clear error messages with actionable guidance - Complete user documentation with examples

Reflection Validation: - DI successfully exercises all reflection capabilities (scanning, type hints, annotations) - Feedback loop identifies and resolves reflection system gaps - Reflection system validated as suitable foundation for framework development

8. 🔧 Technical Notes¶

Thread Safety¶

• Container operations thread-safe via internal RLock
• Bootstrap runtime thread-safe through container locking
• Global reflection context requires synchronized provider registration

Idempotency Semantics¶

• Identical registrations are no-ops (O(1) duplicate detection)
• Conflicting registrations raise DiRegistrationError
• Re-bootstrap with same plan produces no observable changes

Python Constraints¶

• Generic type erasure requires explicit type parameters: resolve[T](type[T])
• Property injection uses Autowired[T] type hints (not decorator syntax)
• Forward reference resolution handled by reflection system

Last Updated: 2025-10-11 Next Review: After TD-021 implementation kickoff

This roadmap prioritizes DI development to validate the reflection system while building a comprehensive dependency injection framework inspired by Spring Boot but designed for Python's unique characteristics and constraints.