Interview Tracker API

Job searching is genuinely hard to track at scale — dozens of applications, multiple interview rounds per company, follow-up reminders, and a need to measure conversion funnel quality over time. Existing tools either lack depth (spreadsheets) or lock important features behind expensive paywalls without the flexibility to self-host.

I wanted to build the backend I would actually use: one that modelled the full job-search lifecycle, exposed a clean REST API the frontend could consume, and demonstrated the kind of architecture a senior backend role would expect — domain isolation, observable failure modes, zero-downtime deployments, and a credible security model.

Production-grade means accepting the same constraints a real team would impose before going live.

• ›Zero-downtime schema evolution: every Flyway migration must be backward-compatible with the previous image still running (rolling-deploy policy enforced by a custom Gradle lint task)
• ›Domain isolation: Spring Modulith ArchUnit gate — cross-module imports outside declared allowedDependencies fail the build
• ›Auth hardness: HttpOnly-cookie JWTs, per-token Redis blacklist, token-version bulk revocation, TOTP + WebAuthn as second factors
• ›AI guardrails: CV tailoring via Groq LLM must never fabricate employers, skills, or dates — four enforced filter layers
• ›Billing correctness: webhook idempotency on (provider, event_id); 402 → PaywallError pipeline all the way to the UI modal
• ›Test gate: ≥55% line coverage enforced by JaCoCo in every CI run; Testcontainers Postgres for PG-specific repository tests
• ›Minimal external dependency: core CRUD (create, read, update, delete) survives Groq LLM, Redis, and ntfy.sh outages — degradation paths for each are load-bearing, not aspirational

I structured the codebase as nine Spring Modulith domain modules — iam, applications, profile, notifications, billing, analytics, logos, bootstrap, and the AI sub-modules (coach, coverletter) — each with an explicit allowedDependencies declaration. ArchUnit validates the dependency graph on every build; any accidental cross-module coupling fails the test suite before it reaches review.

Security is layered in order of cost: JWT signature + expiry (in-process, zero network), Redis token blacklist (O(1) SET lookup), then users.token_version comparison (DB row, cached 30s). This means a single-session revoke hits only Redis, while a 'logout all devices' bumps the version column and invalidates everything instantly without enumerating tokens.

The AI pipeline uses Spring AI's OpenAI-compatible client pointed at Groq. Four guardrails enforce the no-fabrication promise: only summary, skill order, and per-role/project bullets are LLM-rewriteable; skill output is intersected with the user's real skill list; experience bullet rewrites are accepted only when keyed to a real experience ID; the service fails soft (returns base content) if Groq is down.

• ›9 domain modules with ArchUnit-enforced isolation; named interfaces for cross-module SPI (iam::security, profile::dto)
• ›Three-layer auth: JWT sig → Redis blacklist → token_version column; Spring Security 7 resource-server adapters replace hand-rolled filter
• ›Transactional outbox (V15) + Spring Modulith @ApplicationModuleListener for durable async delivery (password-reset emails, magic-link emails)
• ›41 Flyway migrations with zero-downtime policy: CONCURRENTLY indexes in separate files, nullable-before-constrained pattern, Gradle lint gate
• ›Resilience4j circuit breakers on every outbound HTTP client (LLM, ntfy, logo CDN, job-link import); same @HttpExchange declarative interface pattern throughout
• ›Rate limiting: sliding-window Redis counters per user; jd-analysis-daily bucket shared across three endpoints via matching policy name
• ›Transactional outbox with DLQ: outbox_messages table (V15) drained via FOR UPDATE SKIP LOCKED with exponential backoff (30 s → 1 h max, 6 attempts → FAILED / DLQ). Micrometer counters (outbox.attempts: dead_lettered/retried/dispatched) alert in Grafana
• ›HikariCP connection pool (configurable, default 50) with virtual-thread-awareness. Optional read-replica routing via DB_READER_URL — @Transactional(readOnly=true) queries route to the reader pool through ReadOnlyRoutingDataSource + LazyConnectionDataSourceProxy
• ›ShedLock distributed scheduling: ReminderScheduler, OutboxWorker, OutboxCleanupScheduler, and CacheRefreshScheduler all guarded by ShedLock so multi-pod deploys never double-fire
• ›Cache stampede prevention: HotKeys tracks every userDetails cache access; CacheRefreshScheduler pre-refreshes hot entries at 25 s before the 30 s TTL expires. SubscriptionService uses ConcurrentHashMap<…, CompletableFuture<>> for in-flight request coalescing on tier cache misses
• ›BillingProvider SPI: Lemon Squeezy is the default merchant of record; swapping to Stripe requires zero controller changes — implement the BillingProvider interface and re-deploy. RateLimitOverride SPI (iam::security named interface) lets billing flex per-user rate limits: 3/day FREE → 50/day PRO for jd-analysis-daily
• ›Virtual threads everywhere: spring.threads.virtual.enabled=true on JDK 25. A bounded llmExecutor (ThreadPoolTaskExecutor) handles Groq AI calls (virtual threads can't block on long-running LLM responses without pinning); Tomcat request threads use virtual threads natively
• ›Frontend telemetry correlation: the browser's Grafana Faro RUM sends a W3C traceparent header on every API call; the backend's Micrometer OTLP exporter joins browser→backend→Groq LLM spans into the same trace ID

Engineering is the act of choosing what to give up. These are the deliberate tradeoffs I made and why.

• 01ArchUnit build gates are the only way to keep module boundaries honest over time. Code review doesn't catch imports; the compiler does.
• 02Two revocation strategies answer two different questions — don't conflate 'kill this session' with 'kill everything'. Doing both with one mechanism means doing one of them badly.
• 03The LLM no-fabrication guardrail is load-bearing, not aspirational. Without the skill intersection and ID whitelist, the first edge-case Groq response would have shipped a lie onto a real CV.
• 04Zero-downtime migrations are a discipline problem before they are a tooling problem. The Gradle lint task forces the conversation in the PR, not at 2am during a deploy.
• 05Fail-soft on external dependencies (LLM, Google Calendar, ntfy) keeps the core path working when third parties are degraded. Every outbound call has a circuit breaker and a graceful degradation path.
• 06The transactional outbox + DLQ pattern is not optional once a single async side-effect matters. Without it, the 'JVM crashed between commit and listener' gap is real and silent. With it, every failed message is a persistent row you can replay, alert on, and audit — the same observability you'd ask of a queueing system, without adding a queueing system.
• 07A read-replica topology that activates via an env var is cheaper than building for scale from day one and cheaper than a migration later. The code has the seam; the deployment doesn't pay for it until the seam is needed.
• 08SPIs beat feature flags for provider swaps. A BillingProvider interface means Lemon Squeezy → Stripe is a new implementation class, not a config toggle that both providers' code paths have to live under.