📘 Technical Design 015 — DI System Modularization¶

Field	Value
ID	TD-015
Title	DI System Modularization
Status	Draft
Owner	Contributors
Audience	Contributors, maintainers, architecture reviewers
Created	2025-10-02
Last Updated	2025-10-02
Reviewers	TBD
Approvers	TBD
Tracking	Phase 7 of DI Roadmap
Related	001-dependency-injection-roadmap.md

🧭 1. Summary¶

This technical design implements Phase 7 of the DI roadmap by extracting DI components into organized submodules for improved maintainability and readability. The design reorganizes the monolithic DI implementation into focused, cohesive modules while maintaining backward compatibility.

At a glance:

Extract DI components into logical submodules under loryth.common.di
Reorganize injection system, annotation processing, and scanning into separate modules
Improve file size management and reduce complexity per module
Maintain backward compatibility through preserved public API
Create clear module boundaries with well-defined responsibilities

📚 2. Context and Problem Statement¶

The DI system has grown through Phases 1-6 into a comprehensive framework with potential maintainability challenges:

Current State Analysis¶

src/loryth/common/di/
├── __init__.py              # Module exports (growing large)
├── new_container.py         # Core container (500+ lines projected)
├── exceptions.py            # DI-specific exceptions
├── lifecycle.py             # Service lifecycle enum
├── container_item.py        # Container item dataclass
├── annotations.py           # DI annotations (Phase 4 - will be large)
├── injection.py             # Injection system (Phase 3 - will be large)
├── scanning.py              # Package scanning (Phase 5 - will be large)
└── (other growing components)

Projected Issues¶

Large File Sizes - Individual modules may exceed maintainability thresholds
Mixed Responsibilities - Single files handling multiple concerns
Poor Module Cohesion - Related functionality scattered across files
Navigation Difficulty - Developers struggle to find relevant code
Testing Complexity - Large modules require extensive test coverage

2.1 Goals¶

Logical Modularization - Organize code by functional responsibility
File Size Management - Keep individual modules under ~500 lines
Improved Cohesion - Group related functionality together
Better Navigation - Clear module structure for developers
Maintained Compatibility - Preserve existing public API

2.2 Non-Goals (Out of Scope)¶

Changing public API or breaking backward compatibility
Performance optimization (separate concern)
Adding new functionality (handled in other phases)
Changing fundamental architecture patterns

2.3 Assumptions and Constraints¶

All Phase 1-6 functionality exists and is implemented
Modularization should not impact runtime performance
Public API surface remains exactly the same
Import paths for public APIs remain unchanged

🧩 3. Design Overview¶

The modularization extracts DI components into organized submodules with clear responsibilities and boundaries.

graph TB
    subgraph "🎯 Current Structure"
        A[new_container.py<br/>~500+ lines]
        B[annotations.py<br/>~400+ lines]
        C[injection.py<br/>~600+ lines]
        D[scanning.py<br/>~400+ lines]
    end

    subgraph "📦 New Submodules"
        E[container/]
        F[annotations/]
        G[injection/]
        H[scanning/]
    end

    subgraph "🔧 Container Module"
        I[core.py<br/>Core container]
        J[bootstrap.py<br/>Bootstrap logic]
        K[registration.py<br/>Registration API]
        L[resolution.py<br/>Resolution API]
    end

    subgraph "💉 Injection Module"
        M[manager.py<br/>Injection coordination]
        N[constructor.py<br/>Constructor injection]
        O[property.py<br/>Property injection]
        P[method.py<br/>Method wrapping]
    end

    A --> I
    A --> J
    A --> K
    A --> L

    C --> M
    C --> N
    C --> O
    C --> P

    E --> I
    E --> J
    E --> K
    E --> L

    G --> M
    G --> N
    G --> O
    G --> P

    classDef currentNode fill: none, stroke: #f44336, stroke-width: 2px, color: #ffffff
    classDef newNode fill: none, stroke: #4caf50, stroke-width: 2px, color: #ffffff
    classDef submoduleNode fill: none, stroke: #2196f3, stroke-width: 2px, color: #ffffff

    class A,B,C,D currentNode
    class E,F,G,H newNode
    class I,J,K,L,M,N,O,P submoduleNode

3.1 Target Module Structure¶

src/loryth/common/di/
├── __init__.py                    # Public API exports (unchanged)
├── exceptions.py                  # DI exceptions (unchanged)
├── lifecycle.py                   # Service lifecycle (unchanged)
├── container_item.py              # Container item (unchanged)
├── container/
│   ├── __init__.py               # Container submodule exports
│   ├── core.py                   # Core DiContainer class
│   ├── bootstrap.py              # Bootstrap and cycle detection
│   ├── registration.py           # Registration method implementations
│   └── resolution.py             # Resolution method implementations
├── injection/
│   ├── __init__.py               # Injection submodule exports
│   ├── manager.py                # InjectionManager coordination
│   ├── constructor.py            # Constructor injection logic
│   ├── property.py               # Property injection with Autowired
│   ├── method.py                 # Method wrapping and AOP
│   └── descriptors.py            # Autowired and other descriptors
├── annotations/
│   ├── __init__.py               # Annotation submodule exports
│   ├── service.py                # Service and component annotations
│   ├── configuration.py          # Configuration and bean annotations
│   ├── processor.py              # Annotation processor
│   └── config_binding.py         # Configuration value binding
├── scanning/
│   ├── __init__.py               # Scanning submodule exports
│   ├── package_scanner.py        # Package and module scanning
│   ├── filters.py                # Annotation and pattern filters
│   ├── application.py            # Application bootstrap
│   └── discovery.py              # Class discovery utilities
└── configuration/
    ├── __init__.py               # Configuration submodule exports
    ├── providers.py              # Config providers (YAML, JSON, TOML)
    ├── loader.py                 # Configuration loading
    └── binder.py                 # Configuration value binding

📐 4. Implementation Details¶

4.1 Container Submodule Organization¶

# src/loryth/common/di/container/__init__.py
"""Container submodule with core DI functionality."""

from .core import DiContainer
from .bootstrap import BootstrapManager, DependencyCycleDetector
from .registration import RegistrationMixin
from .resolution import ResolutionMixin

__all__ = [
    'DiContainer',
    'BootstrapManager',
    'DependencyCycleDetector',
    'RegistrationMixin',
    'ResolutionMixin'
]

# src/loryth/common/di/container/core.py
"""Core DiContainer class with composition of mixins."""

from typing import Any, TypeVar
from ..lifecycle import ServiceLifecycle
from ..container_item import DiContainerItem
from .registration import RegistrationMixin
from .resolution import ResolutionMixin
from .bootstrap import BootstrapManager

T = TypeVar('T')

class DiContainer(RegistrationMixin, ResolutionMixin):
    """
    Enhanced dependency injection container with lifecycle management.

    Combines registration, resolution, and bootstrap capabilities through
    composition of focused mixins.
    """

    def __init__(self) -> None:
        super().__init__()
        self._items: list[DiContainerItem[Any]] = []
        self._cache: dict[type, Any] = {}
        self._dependencies: dict[type, list[type]] = {}
        self._bootstrap_manager = BootstrapManager(self)
        self._is_bootstrapped = False

    def bootstrap(self) -> None:
        """Trigger container bootstrap process."""
        if not self._is_bootstrapped:
            self._bootstrap_manager.bootstrap()
            self._is_bootstrapped = True

    def clear(self) -> None:
        """Clear all registrations and cache."""
        self._items.clear()
        self._cache.clear()
        self._dependencies.clear()
        self._is_bootstrapped = False

# src/loryth/common/di/container/registration.py
"""Registration method implementations."""

from typing import Any, Callable, TypeVar
from ..lifecycle import ServiceLifecycle
from ..container_item import DiContainerItem
from ..exceptions import DiRegistrationError

T = TypeVar('T')

class RegistrationMixin:
    """Mixin providing registration methods for DiContainer."""

    def register_singleton[T](self, service_type: type[T], implementation_type: type[T]) -> None:
        """Register a service as singleton."""
        self._validate_registration(service_type, implementation_type)
        self._analyze_and_track_dependencies(service_type, implementation_type)

        provider = self._create_singleton_provider(implementation_type)
        item = DiContainerItem(service_type, provider, ServiceLifecycle.SINGLETON)
        self._items.append(item)

    def register_transient[T](self, service_type: type[T], implementation_type: type[T]) -> None:
        """Register a service as transient."""
        self._validate_registration(service_type, implementation_type)
        self._analyze_and_track_dependencies(service_type, implementation_type)

        provider = self._create_transient_provider(implementation_type)
        item = DiContainerItem(service_type, provider, ServiceLifecycle.TRANSIENT)
        self._items.append(item)

    # Additional registration methods...

    def _validate_registration(self, service_type: type, implementation_type: type) -> None:
        """Validate registration compatibility."""
        # Implementation moved from main container
        pass

    def _create_singleton_provider(self, implementation_type: type) -> Callable:
        """Create provider for singleton services."""
        # Implementation moved from main container
        pass

4.2 Injection Submodule Organization¶

# src/loryth/common/di/injection/__init__.py
"""Injection submodule with automatic dependency injection."""

from .manager import InjectionManager
from .constructor import ConstructorInjector
from .property import PropertyInjector, Autowired
from .method import MethodWrapper
from .descriptors import AutowiredDescriptor

__all__ = [
    'InjectionManager',
    'ConstructorInjector',
    'PropertyInjector',
    'Autowired',
    'MethodWrapper',
    'AutowiredDescriptor'
]

# src/loryth/common/di/injection/manager.py
"""Injection manager coordinating all injection phases."""

from typing import TypeVar, Any
from ..container_item import DiContainerItem
from .constructor import ConstructorInjector
from .property import PropertyInjector
from .method import MethodWrapper

T = TypeVar('T')

class InjectionManager:
    """Coordinates all phases of dependency injection."""

    def __init__(self, container) -> None:
        self._container = container
        self._constructor_injector = ConstructorInjector(container)
        self._property_injector = PropertyInjector(container)
        self._method_wrapper = MethodWrapper(container)

    def create_with_injection[T](self, item: DiContainerItem[T]) -> T:
        """Execute full injection pipeline."""
        # Phase 1: Constructor injection
        if self._should_use_constructor_injection(item):
            instance = self._constructor_injector.create_with_injection(item)
        else:
            instance = item.provider(self._container)

        # Phase 2: Property injection
        self._property_injector.inject_properties(instance)

        # Phase 3: Method wrapping
        wrapped_instance = self._method_wrapper.wrap_methods(instance)

        return wrapped_instance

# src/loryth/common/di/injection/property.py
"""Property injection implementation."""

from typing import TypeVar, Generic, Any
from ..exceptions import InjectionError

T = TypeVar('T')

class Autowired(Generic[T]):
    """Property descriptor for automatic dependency injection."""

    def __init__(self, service_type: type[T] | None = None) -> None:
        self._service_type = service_type
        self._attr_name: str | None = None

    # Implementation moved from main injection module

class PropertyInjector:
    """Handles property injection for Autowired properties."""

    def __init__(self, container) -> None:
        self._container = container

    def inject_properties(self, instance: Any) -> None:
        """Inject dependencies into Autowired properties."""
        # Implementation moved from main injection module

4.3 Backward Compatibility Layer¶

# src/loryth/common/di/__init__.py
"""DI system public API - maintains backward compatibility."""

# Core container (unchanged public API)
from .container import DiContainer

# Lifecycle and items (unchanged)
from .lifecycle import ServiceLifecycle
from .container_item import DiContainerItem

# Exceptions (unchanged)
from .exceptions import (
    DiError,
    DependencyNotFoundError,
    DependencyCycleError,
    DiRegistrationError,
    DiResolutionError
)

# Injection system (maintain public imports)
from .injection import (
    InjectionManager,
    ConstructorInjector,
    PropertyInjector,
    Autowired,
    MethodWrapper
)

# Annotations (maintain public imports)
from .annotations import (
    Service, Component, Singleton, Transient,
    Configuration, Bean, ConfigValue,
    service, component, singleton, transient,
    configuration, bean
)

# Scanning (maintain public imports)
from .scanning import (
    ApplicationAnnotation, Application,
    PackageScanner, ScanningConfiguration,
    ApplicationBootstrap,
    application
)

# Configuration (maintain public imports)
from .configuration import (
    ConfigProvider,
    YamlConfigProvider,
    JsonConfigProvider,
    TomlConfigProvider,
    ConfigurationLoader,
    ConfigurationBinder
)

# Preserve all existing public API exports
__all__ = [
    # Core
    'DiContainer',
    'ServiceLifecycle',
    'DiContainerItem',

    # Exceptions
    'DiError',
    'DependencyNotFoundError',
    'DependencyCycleError',
    'DiRegistrationError',
    'DiResolutionError',

    # Injection
    'InjectionManager',
    'ConstructorInjector',
    'PropertyInjector',
    'Autowired',
    'MethodWrapper',

    # Annotations
    'Service', 'Component', 'Singleton', 'Transient',
    'Configuration', 'Bean', 'ConfigValue',
    'service', 'component', 'singleton', 'transient',
    'configuration', 'bean',

    # Scanning
    'Application', 'PackageScanner', 'ScanningConfiguration',
    'ApplicationBootstrap', 'application',

    # Configuration
    'ConfigProvider', 'YamlConfigProvider', 'JsonConfigProvider',
    'TomlConfigProvider', 'ConfigurationLoader', 'ConfigurationBinder'
]

4.4 Migration Strategy¶

class ModularizationMigration:
    """Manages migration to modularized structure."""

    EXTRACTION_PLAN = {
        # Container extraction
        "new_container.py": {
            "DiContainer.__init__": "container/core.py",
            "DiContainer.register_*": "container/registration.py",
            "DiContainer.resolve*": "container/resolution.py",
            "DiContainer.bootstrap": "container/bootstrap.py",
            "BootstrapManager": "container/bootstrap.py",
            "DependencyCycleDetector": "container/bootstrap.py"
        },

        # Injection extraction
        "injection.py": {
            "InjectionManager": "injection/manager.py",
            "ConstructorInjector": "injection/constructor.py",
            "PropertyInjector": "injection/property.py",
            "Autowired": "injection/property.py",
            "MethodWrapper": "injection/method.py"
        },

        # Annotations extraction
        "annotations.py": {
            "Service, Component, etc.": "annotations/service.py",
            "Configuration, Bean, etc.": "annotations/configuration.py",
            "AnnotationProcessor": "annotations/processor.py",
            "ConfigurationBinder": "annotations/config_binding.py"
        },

        # Scanning extraction
        "scanning.py": {
            "PackageScanner": "scanning/package_scanner.py",
            "ScanningFilter": "scanning/filters.py",
            "ApplicationBootstrap": "scanning/application.py",
            "ModuleLoader": "scanning/discovery.py"
        }
    }

    def validate_extraction(self, source_file: str, target_module: str) -> bool:
        """Validate that extraction maintains functionality."""
        # Check that all dependencies are preserved
        # Verify that public API remains unchanged
        # Ensure no circular dependencies introduced
        pass

⚙️ 5. Semantics¶

5.1 Module Responsibilities¶

container/: Core container functionality, registration, resolution, bootstrap
injection/: All forms of dependency injection (constructor, property, method)
annotations/: DI annotation system and processing
scanning/: Package scanning and application bootstrap
configuration/: Configuration loading and binding

5.2 Import Rules¶

Public API Unchanged: All existing imports continue to work
Internal Reorganization: Internal module structure is implementation detail
Submodule Access: Internal code can import from submodules directly
No Circular Dependencies: Submodules must have clear dependency hierarchy

5.3 File Size Guidelines¶

Target Size: Individual modules should be under 500 lines
Logical Cohesion: Related functionality grouped together
Single Responsibility: Each module has clear, focused purpose

🧪 6. Examples¶

6.1 Unchanged Public API¶

# All existing imports continue to work exactly the same
from loryth.common.di import Container, ServiceLifecycle
from loryth.common.di import Autowired, service, component
from loryth.common.di import Application, application

# Usage remains identical
container = Container()
container.register_singleton(UserService, UserService)


@service
class EmailService:
    user_service: Annotated[UserService, Autowired()]


@application(packages=["myapp.services"])
class MyApp:
    pass

6.2 Internal Module Organization¶

# Internal imports use new modular structure
# (These are implementation details, not public API)

# Container module
from loryth.common.di.container import Container, BootstrapManager
from loryth.common.di.container.registration import RegistrationMixin
from loryth.common.di.container.resolution import ResolutionMixin

# Injection module
from loryth.common.di.injection import InjectionManager
from loryth.common.di.injection.constructor import ConstructorInjector
from loryth.common.di.injection.property import PropertyInjector, Autowired

# Annotations module
from loryth.common.di.annotations.service import Service, Component
from loryth.common.di.annotations.processor import AnnotationProcessor

6.3 Development Benefits¶

# Developers can navigate to focused modules
# Instead of searching through 1000+ line files

# Find registration logic
# -> container/registration.py (~150 lines)

# Find property injection logic
# -> injection/property.py (~200 lines)

# Find annotation processing logic
# -> annotations/processor.py (~250 lines)

# Each module has single, clear responsibility

✅ 7. Acceptance Criteria¶

❓ 8. Open Questions¶

8.1 Modularization Strategy Questions¶

Granularity Balance: How should we balance module granularity:
Should we create many small modules or fewer larger ones?
Should related functionality be grouped even if it creates larger modules?
Should we optimize for discoverability or file size?
Responsibility Boundaries: How should we define module boundaries:
Should modules be organized by feature or by architectural layer?
Should cross-cutting concerns be in separate modules or distributed?
Should we allow some overlap between module responsibilities?
Migration Strategy: How should we approach the modularization migration:
Should we modularize incrementally or all at once?
Should we maintain both old and new structures temporarily?
Should we provide migration utilities for internal code?

8.2 Architecture and Design Questions¶

Dependency Management: How should submodules handle dependencies:
Should submodules import from each other or only from parent?
Should we enforce strict dependency hierarchies?
Should we allow bidirectional dependencies for closely related modules?
Code Duplication: How should we handle shared functionality:
Should common utilities be in a shared module or duplicated?
Should we create base classes to avoid duplication?
Should we prefer composition or inheritance for shared behavior?
Testing Strategy: How should testing adapt to modularized structure:
Should tests follow the same module structure?
Should we test modules in isolation or integration?
Should module boundaries influence test organization?

8.3 Performance and Compatibility Questions¶

Import Performance: How should modularization affect import performance:
Should we optimize for lazy loading or eager loading?
Should submodule imports be cached or computed on demand?
Should we measure and monitor import performance impact?
Backward Compatibility: How strictly should we maintain compatibility:
Should internal implementation details remain stable?
Should we support undocumented import paths that may exist?
Should we provide migration warnings for internal imports?

8.4 Development Experience Questions¶

IDE Support: How should modularization improve development experience:
Should module structure optimize for IDE navigation?
Should we provide IDE-specific organization hints?
Should module names support autocomplete and discovery?
Documentation: How should modularization be documented:
- Should we document internal module structure?
- Should we provide architectural diagrams for module relationships?
- Should module documentation include usage guidelines?

8.5 Future Extensibility Questions¶

Extension Points: How should modularization support future extensions:
- Should module structure accommodate unknown future features?
- Should we provide plugin interfaces at module boundaries?
- Should modules be designed for independent versioning?
Maintenance: How should modularized structure affect maintenance:
- Should modules have dedicated maintainers or teams?
- Should module changes require cross-module compatibility testing?
- Should we provide tooling for cross-module refactoring?
Evolution: How should the modular structure evolve over time:
- Should module boundaries be stable or allow reorganization?
- Should we version the module structure separately from functionality?
- Should module evolution be coordinated with public API changes?

🚀 9. Rollout and Migration¶

9.1 Implementation Phases¶

Phase 7a: Design and planning - Finalize module structure and responsibilities - Create extraction mapping from current files to new modules - Design backward compatibility preservation strategy

Phase 7b: Submodule creation - Create new submodule directory structure - Extract code into appropriate modules - Implement backward compatibility imports

Phase 7c: Validation and cleanup - Validate that all functionality is preserved - Ensure test suites continue to pass - Clean up any temporary compatibility code

9.2 Migration Timeline¶

Week 1: Design and Planning
- Finalize module structure
- Create detailed extraction plan
- Set up module scaffolding

Week 2: Container and Injection Modules
- Extract container functionality
- Extract injection functionality
- Validate core functionality

Week 3: Annotations and Scanning Modules
- Extract annotation system
- Extract scanning functionality
- Validate complete system

Week 4: Testing and Cleanup
- Comprehensive testing
- Performance validation
- Documentation updates

⚠️ 10. Risks and Mitigations¶

10.1 Backward Compatibility Risk¶

Risk: Modularization breaks existing code that depends on internal imports Mitigation: Comprehensive compatibility testing, maintain public API exactly

10.2 Performance Impact Risk¶

Risk: Additional import overhead from modular structure Mitigation: Performance testing, optimize critical import paths

10.3 Circular Dependency Risk¶

Risk: Poor module organization creates circular dependencies Mitigation: Clear dependency hierarchy design, validation tooling

📈 11. Progress¶

Legend: [x] = complete, [ ] = pending

📝 12. Changelog¶

2025-10-02: Initial draft technical design created

📖 13. Glossary¶

Modularization: Process of organizing code into logical, cohesive modules
Submodule: Module within a larger module that handles specific functionality
Backward Compatibility: Ensuring existing code continues to work unchanged
Code Extraction: Moving code from one module to another while preserving functionality
Module Responsibility: The specific functionality and concerns handled by a module