Engineering Knowledge Assistant

Engineering organizations maintain knowledge across scattered systems — code repositories, API documentation, Confluence pages, production dashboards, and incident post-mortems. Finding a specific answer often means searching five different tools, none of which understand the semantic relationship between a code change, the API it broke, and the Confluence page that documented the old behaviour.

The goal was to build a single assistant that could ingest all these sources, index them into a searchable knowledge graph, and answer natural-language questions by retrieving the most relevant chunks and synthesising them through an LLM — without hallucinating non-existent APIs or code.

The system had to work reliably enough to be useful in a real engineering workflow, with clear failure modes and no silent data loss.

• ›LLM responses must never fabricate APIs, endpoints, or code signatures — retrieval results are the only source of truth, the LLM is a summariser, not a generator of new engineering facts
• ›Chat responses must stream tokens progressively — no multi-second pauses while the full response is generated server-side
• ›Ingestion must be async and durable — a failure in the embedding step must not lose the raw document
• ›Authentication must support both OAuth2 (production) and email+password (local dev) without dual code paths
• ›The frontend must render correctly for keyboard-only users — hover-reveal action groups are a common failure point
• ›91 audit findings at baseline (24 HIGH, 47 MEDIUM, 20 LOW) must be addressed before the system is production-ready

I split the system into two independent deployables: a Spring Boot reactive backend and a Next.js App Router frontend, communicating over HTTP and SSE. The backend owns the data layer, LLM routing, and ingestion; the frontend owns the user experience, streaming visualisation, and dashboard surface.

The backend is modular by domain: auth (JWT filtering + OAuth2 success handling), chat (streaming + citation extraction + context building), ingestion (connectors → chunking → embedding → vector store), retrieval (hybrid search + reranking + semantic cache), graph (Neo4j entity extraction and traversal), and admin (user management, feedback collection). Each module owns its persistence, its reactive chains compose through WebFlux, and blocking operations (JPA, LLM HTTP calls) are isolated on boundedElastic schedulers.

The frontend follows the same domain-sliced pattern under src/features/: chat, search, sources, graph, analytics, observability, admin, and settings. Server state flows through TanStack Query 5 with centralised query keys; client state (auth tokens, UI preferences) lives in Zustand 5 with localStorage persistence. Chat uses a custom SSE reader that hand-reassembles chunked data: lines across TCP segment boundaries — a common bug in streaming implementations that I confirmed and fixed during the audit.

The ingestion pipeline is Kafka-driven: ingestion requests are published to a topic, consumed by a stateless processor that downloads, parses, chunks (code-aware + heading-aware splitters), embeds (OpenAI or Voyage), and upserts into pgvector in batch writes. Neo4j graph population is optional and disabled by default. The architecture ensures the pipeline can be scaled horizontally by adding consumers, and failures at any stage are recoverable without data loss.

After the initial implementation, I ran two comprehensive code audits that identified 91 issues. These were fixed across 27 files in 5 commits: build foundation (Java 21 LTS, Spring Boot GA, dependency pinning), security hardening (JWT role validation, cross-user conversation ownership, removal of default secrets, OAuth2 secure delivery), data integrity (reactive Redis cache, repository-layer persistence, missing database indexes, structured error logging), frontend robustness (Zustand persist consolidation, SSE buffer fix, accessibility compliance), and the addition of email+password authentication for local development.

• ›Backend: 12 Gradle modules (eka-auth, eka-chat, eka-ingestion, eka-retrieval, eka-embedding, eka-graph, eka-web, eka-app, eka-common, eka-test, plus two more). Reactive chain composed via WebFlux; blocking JPA/LLM calls on boundedElastic schedulers
• ›Frontend: 114 source files across 18 directories. Feature-sliced modules under src/features/. Radix UI primitives with Tailwind CSS v4 + CVA for component styling
• ›SSE streaming: custom ReadableStream reader that buffers incomplete data: lines and reassembles on TCP chunk boundaries — verified fix through the 91-audit remediation (Phase 2, item 2f)
• ›Audit remediation: 5 logical commits touching 27 backend+frontend files. Build → Security → Data Integrity → Frontend Auth → Tests. ./gradlew build green, zero warnings
• ›Email+password auth: BCryptPasswordEncoder; POST /api/v1/auth/login and /register; V9 migration adds password_hash column; existing OAuth2 users get a clear error hint if they try email login

Engineering is the act of choosing what to give up. These are the deliberate tradeoffs I made and why.

• 01A reactive architecture is only as reactive as its most blocking dependency. The ConversationCacheService was using a blocking Redis template inside a reactive chain — the audit caught it because the symptom was intermittent, not a crash.
• 02Multi-module Gradle projects need dependency management as much as they need domain modelling. Pinning Spring Boot 3.5.0 → 3.4.5 GA and removing the milestone repo wasn't ideological — 3.5.0-M5 dependencies were pulling in incompatible transitive versions that only surfaced at runtime.
• 03Ownership enforcement at the controller level is a rubber stamp without a test that exercises the cross-user path. The audit found the gap before anyone exploited it, and the fix was a single verifyOwnership() method — but it required changing every endpoint signature.
• 04Default secrets in application.yml are a blind spot because they work perfectly in local dev and silently ship to production. Removing the default and letting the JVM fail at startup is the only safe pattern.
• 05SSE streaming is harder than it looks. The TCP chunk boundary bug (splitting a \n\n across two reads) would corrupt every Nth message under load — the fix was one line (`buffer = parts.pop() ?? ''`) that the audit identified by reasoning about the buffer reassembly logic, not by reproducing the timing.
• 06Zustand persist middleware eliminates an entire class of bugs where localStorage reads happen before the React hydration pass. Consolidating four files of manual getItem/setItem calls into one store was the smallest diff with the largest correctness impact in the frontend audit.
• 07The transactional outbox pattern is overkill for this system's scale, but the principle — don't dual-write — informed the Kafka ingestion pipeline design. Ingestion requests are published to a topic; the consumer acknowledges after the chunk is persisted. A crash between consume and acknowledge means at-least-once delivery, which the upsert handles.
• 08Two code audits caught issues the initial implementation missed not because of carelessness, but because the second pass reads the code from a different altitude — line-by-line reviews find buffer split bugs; structural reviews find missing ownership checks. Both altitudes are necessary.