Skip to content

ADR-0012: Embedding Factory vs Vector Index Ownership

Status

SUPERSEDED by ADR-0015 (Centralized Embedding Store in AI Service)

Date: 2026-02-14

Changelog

Date Revision Description Author
2026-02-14 1.0 Initial version souperk
2026-02-15 1.1 Update status to reflect implementation souperk
2026-02-15 1.2 Superseded by ADR-0015 souperk

Implementation Status

Data model and API endpoints are implemented. ConversationVector entity and repository, EmbeddingProvider protocol, embedding generation endpoint (POST /embeddings/generate), and conversation search endpoint (POST /conversations/search with cosine similarity) are all in place. The pgvector migration (v06-embedding.sql) creates the table and IVFFlat index.

Not yet implemented: - KB service document vectors (KB's responsibility) - Embedding model versioning strategy

See also: ADR-0014 — real embedding provider classes (OpenAICompatibleEmbeddingProvider, OllamaEmbeddingProvider) are now implemented but not yet wired into the embedding pipeline. The global EmbeddingProvider DI singleton still uses StubEmbeddingProvider; embedding resolution will be addressed by ADR-0015.

Context

The platform needs vector embedding capabilities for two distinct use cases:

  1. Conversation Search: Finding semantically similar past conversations within a space (owned by the AI service)
  2. Document Search: Finding relevant knowledge base articles and documents (owned by the KB service)

Both require generating embeddings via an external model provider and storing vectors for similarity search. The question is where the embedding generation capability and vector storage should live.

Key factors: - The AI service already integrates with model providers (which also offer embedding APIs) - The KB service has its own document ingestion pipeline and search requirements - pgvector is used for vector storage, and vectors live in the shared loryth database - Embedding model choice may vary between use cases (conversation embeddings vs document embeddings) - Vector dimensions must match between storage and query time

Decision

We will split embedding responsibilities along domain ownership lines:

  1. AI Service Owns Conversation Vectors: The ai_conversation_vector table stores embeddings for conversation messages. The AI service generates these embeddings via its EmbeddingProvider abstraction and stores them in its own table. Conversation similarity search (POST /conversations/search) is an AI service endpoint.

  2. KB Service Owns Document Vectors: The KB service manages its own document embeddings in its existing vector infrastructure. Document embeddings are generated as part of the KB ingestion pipeline.

  3. Shared Embedding Provider Abstraction: The EmbeddingProvider protocol (defined in the AI service's domain layer) provides a clean interface for embedding generation. The KB service may use the AI service as an internal embedding API or implement its own provider — this is a deployment decision, not an architectural one.

  4. pgvector in Shared Database: Both ai_conversation_vector and KB's vector tables live in the shared loryth database. Table prefixes (ai_, kb_) prevent collisions. The vector extension is created once at database initialization.

  5. Fixed Dimension (1536): V1 uses a fixed 1536-dimensional vector (matching OpenAI's text-embedding-3-small and many other models). The dimension is encoded in the table schema and IVFFlat index.

Consequences

Positive

  • Clear domain ownership: each service owns and queries its own vectors
  • No cross-service vector queries needed at runtime
  • Each service can optimize its embedding strategy independently (different models, chunk sizes, refresh cadences)
  • AI service's EmbeddingProvider abstraction is reusable if other services need embedding generation

Negative

  • Potential duplication if both services embed the same content (e.g., a KB document referenced in a conversation)
  • Fixed 1536 dimensions limits model choice; changing dimension requires migration
  • Two separate vector indexes means no unified "search everything" capability without a federation layer

Neutral

  • Future work may add an embedding microservice if demand grows beyond two consumers
  • The IVFFlat index strategy works well up to ~1M vectors per table; may need HNSW for larger scale
  • Embedding model versioning is not yet addressed; re-embedding after model change is a future concern

Alternatives Considered

  1. Centralized Embedding Service
  2. Description: A dedicated microservice that generates and stores all embeddings, exposing a unified search API
  3. Pros: Single source of truth for all vectors, unified search across domains, model management in one place
  4. Cons: Cross-domain coupling through shared service, single point of failure for all search, complex access control (which service can search what)

  5. Why not chosen: Premature centralization; only two consumers exist today and their embedding needs differ significantly

  6. AI Service Generates All Embeddings, Services Store Their Own

  7. Description: AI service exposes POST /embeddings/generate as an internal API; KB and other services call it and store results locally
  8. Pros: Single embedding model configuration, consistent vector quality
  9. Cons: AI service becomes a bottleneck for all embedding generation, tight runtime coupling

  10. Why not chosen: Creates unnecessary runtime dependency; services should be able to operate independently

  11. KB Service Owns All Vectors

  12. Description: All embeddings (conversation and document) stored in KB's vector tables
  13. Pros: Single vector search implementation, KB already has document processing expertise
  14. Cons: AI service depends on KB for conversation search (wrong domain ownership), KB becomes responsible for AI-specific data
  15. Why not chosen: Violates domain boundary; conversation vectors are an AI concern

References