ENGINEERING IMPLEMENTATION · 2026
Runreviq — Governed Operational Evidence
An access-controlled local workspace for finding applicable runbook passages, reviewing source revisions, and evaluating retrieval quality without presenting unverified guidance as fact.
View GitHub ↗Engineering problem
Operational instructions vary by service, software version, environment, approval state and access rights. Returning an attractive match is unsafe if its applicability or authorization is uncertain.
My contribution
Designed and implemented the personal prototype: PostgreSQL evidence controls, documented review and publication workflows, deterministic retrieval, regression evaluation, and a staged Java backend migration.
Technical approach
- Modelled approved revisions, role-based evidence access, document applicability and source fingerprints
- Built independent review, publication and withdrawal workflows for the runbook corpus
- Implemented exact-passage references and conservative structured conflict handling
- Used external incident questions and controlled retrieval experiments to diagnose lexical misses
- Developed Java service adapters and parity checks as part of a staged migration from the working Python prototype
Key decision and trade-off
Decision
Make authorization and document applicability hard gates before ranking passages.
Why
Ranking alone cannot make a forbidden or wrong-version instruction safe.
Trade-off
Conservative filters can withhold relevant evidence, so relevance evaluation is required alongside access-control checks.
Outcome
A local, reviewable evidence-retrieval prototype with documented authorization controls and benchmark failure cases; the semantic retrieval and Java runtime cutover are not complete.
Show technical details ↓Hide technical details ↑Constraints, alternatives, edge cases, ownership breakdown, lessons.
Constraints
- Authorization must be enforced before passages are exposed
- Unreviewed and superseded instructions must not silently become current guidance
- Evaluations must report false positives and missed evidence, not only successful matches
- No semantic contradiction detector or generated AI answer should be claimed before implementation
Failure cases and edge cases
- Authorized but superseded runbook revisions
- Missing incident context that changes applicability
- Conflicting manually normalized instructions
- Relevant passages missed by lexical retrieval
- An unrelated procedure admitted by an experimental relaxed lexical rule
Full technology list
- Python
- FastAPI
- Java
- Spring Boot
- PostgreSQL
- OIDC
What I learned
A strong evaluation includes negative controls, exact expected passages and proof that access restrictions remain intact.
What I would improve
Add carefully guarded semantic retrieval and independent holdouts before considering AI-generated explanations.