Pablan, as it stands
Self-hosted knowledge management for SMEs: a split-screen Markdown editor whose sections an LLM refines while you write, and RAG question answering over the documents that result. FastAPI + Postgres/pgvector on the back, SvelteKit on the front, everything OpenAI-compatible and self-hostable. Squashed into a single commit; the development history stays local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CA43ZJda8Rbp2hKXNy8f6b
This commit is contained in:
ProfessorNova
@@ -0,0 +1,24 @@
|
||||
# Architecture diagrams
|
||||
|
||||
Hand-authored SVG, one per subject. The SVG **is** the artifact: no diagram
|
||||
toolchain, no build step, and it renders anywhere a browser or an IDE preview
|
||||
does. Each file carries its own stylesheet and follows the reader's light or
|
||||
dark mode through `prefers-color-scheme`.
|
||||
|
||||
**Maintenance rule:** diagrams are as-is documentation like the rest of
|
||||
`docs/`. When a change lands that affects them, the diagram is edited in the
|
||||
same change: a stale diagram is a bug. Where docs and code disagree, the code
|
||||
wins and the doc gets fixed in the same commit.
|
||||
|
||||
**Editing:** every box is a `<rect>` plus its `<text>` lines at explicit
|
||||
coordinates on a plain grid, so a label change is a text edit and a new box is
|
||||
a copied block with new numbers. Keep the shared `<style>` block identical
|
||||
across the five files so they stay one visual system.
|
||||
|
||||
| Diagram | Shows |
|
||||
|---|---|
|
||||
| [data-model.svg](data-model.svg) | ER diagram of all 13 tables, FK semantics, uniques (mirrors the actual migrations) |
|
||||
| [components.svg](components.svg) | System components and their dependencies; built vs. planned (dashed) |
|
||||
| [auth-sequence.svg](auth-sequence.svg) | Login (argon2, session row, httpOnly cookie) and per-request validation incl. the 401 to redirect branch |
|
||||
| [queue-sequence.svg](queue-sequence.svg) | Job queue: claim via FOR UPDATE SKIP LOCKED, handler inside the open claim transaction, backoff, crash safety |
|
||||
| [retrieval-sequence.svg](retrieval-sequence.svg) | Hybrid retrieval, the text-only fallback, and the similarity path: permission CTE first in every one of them |
|
||||
@@ -0,0 +1,158 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1290" width="1000" height="1290" role="img" aria-label="Login and every page request">
|
||||
<title>Login and every page request</title>
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--bg: #fbfaf9;
|
||||
--surface: #ffffff;
|
||||
--sunken: #f2f0ed;
|
||||
--ink: #1a1815;
|
||||
--muted: #63605a;
|
||||
--line: #c5bfb6;
|
||||
--line-soft: #e5e1db;
|
||||
--accent: #b8770a;
|
||||
--secondary: #b04a00;
|
||||
--note: #faf3e6;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #0f0e0d;
|
||||
--surface: #1a1817;
|
||||
--sunken: #232120;
|
||||
--ink: #efece8;
|
||||
--muted: #a49d94;
|
||||
--line: #4a443d;
|
||||
--line-soft: #332f2b;
|
||||
--accent: #e6b422;
|
||||
--secondary: #e08030;
|
||||
--note: #241f16;
|
||||
}
|
||||
}
|
||||
text { font-family: ui-sans-serif, system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif; fill: var(--ink); }
|
||||
.t-title { font-size: 15px; font-weight: 650; }
|
||||
.t-sub { font-size: 11.5px; fill: var(--muted); }
|
||||
.t-node { font-size: 12.5px; font-weight: 600; }
|
||||
.t-body { font-size: 11.5px; }
|
||||
.t-muted { font-size: 11px; fill: var(--muted); }
|
||||
.t-edge { font-size: 10.5px; fill: var(--muted); }
|
||||
.t-mono { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 11px; }
|
||||
.box { fill: var(--surface); stroke: var(--line); stroke-width: 1.2; }
|
||||
.box-sunken { fill: var(--sunken); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.box-accent { fill: var(--surface); stroke: var(--accent); stroke-width: 1.6; }
|
||||
.note { fill: var(--note); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.group { fill: none; stroke: var(--line-soft); stroke-width: 1.2;
|
||||
stroke-dasharray: 5 4; }
|
||||
.lifeline { stroke: var(--line); stroke-width: 1; stroke-dasharray: 4 4; }
|
||||
.edge { stroke: var(--line); stroke-width: 1.3; fill: none; }
|
||||
.edge-accent { stroke: var(--secondary); stroke-width: 1.5; fill: none; }
|
||||
.edge-soft { stroke: var(--line); stroke-width: 1.1; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.edge-soft-accent { stroke: var(--secondary); stroke-width: 1.3; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.planned { stroke-dasharray: 6 4; opacity: 0.62; }
|
||||
</style>
|
||||
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--line)"/>
|
||||
</marker>
|
||||
<marker id="arrow-accent" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--secondary)"/>
|
||||
</marker>
|
||||
<marker id="arrow-open" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9" fill="none" stroke="var(--line)"
|
||||
stroke-width="1.3"/>
|
||||
</marker>
|
||||
</defs>
|
||||
<rect width="1000" height="1290" fill="var(--bg)"/>
|
||||
<text x="28" y="30" class="t-title" text-anchor="start">Login and every page request</text>
|
||||
<text x="28" y="50" class="t-sub" text-anchor="start">Server-side sessions: the cookie token IS the auth_sessions row id.</text>
|
||||
<rect x="98" y="74" width="96" height="30" rx="8" class="box" />
|
||||
<text x="146" y="93" class="t-node" text-anchor="middle">Browser</text>
|
||||
<line x1="146" y1="104" x2="146" y2="1272" class="lifeline"/>
|
||||
<rect x="316.2" y="74" width="131.6" height="30" rx="8" class="box" />
|
||||
<text x="382" y="93" class="t-node" text-anchor="middle">SvelteKit server</text>
|
||||
<line x1="382" y1="104" x2="382" y2="1272" class="lifeline"/>
|
||||
<rect x="570" y="74" width="96" height="30" rx="8" class="box" />
|
||||
<text x="618" y="93" class="t-node" text-anchor="middle">FastAPI</text>
|
||||
<line x1="618" y1="104" x2="618" y2="1272" class="lifeline"/>
|
||||
<rect x="806" y="74" width="96" height="30" rx="8" class="box" />
|
||||
<text x="854" y="93" class="t-node" text-anchor="middle">Postgres</text>
|
||||
<line x1="854" y1="104" x2="854" y2="1272" class="lifeline"/>
|
||||
<line x1="28" y1="138" x2="972" y2="138" class="edge-soft"/>
|
||||
<rect x="336.325" y="127" width="327.35" height="22" rx="11" class="box-sunken" />
|
||||
<text x="500" y="142" class="t-muted" text-anchor="middle">Login: the browser talks to FastAPI directly (D8)</text>
|
||||
<text x="382" y="170" class="t-edge" text-anchor="middle">POST /api/auth/login {email, password}</text>
|
||||
<line x1="146" y1="187" x2="618" y2="187" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="736" y="209" class="t-edge" text-anchor="middle">SELECT user by email</text>
|
||||
<line x1="618" y1="226" x2="854" y2="226" class="edge" marker-end="url(#arrow)"/>
|
||||
<rect x="644" y="268" width="321.2" height="40" rx="6" class="note" />
|
||||
<text x="657" y="285" class="t-body" text-anchor="start">Unknown email still burns a dummy argon2 verify,</text>
|
||||
<text x="657" y="298" class="t-body" text-anchor="start">so timing reveals nothing about who exists.</text>
|
||||
<text x="382" y="326" class="t-edge" text-anchor="middle">401 {detail, code: invalid_credentials}</text>
|
||||
<line x1="618" y1="343" x2="146" y2="343" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<text x="736" y="407" class="t-edge" text-anchor="middle">INSERT auth_sessions</text>
|
||||
<text x="736" y="420" class="t-edge" text-anchor="middle">(id = cookie token, expires_at = now + 14d)</text>
|
||||
<line x1="618" y1="437" x2="854" y2="437" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="382" y="459" class="t-edge" text-anchor="middle">200 user + Set-Cookie pablan_session</text>
|
||||
<text x="382" y="472" class="t-edge" text-anchor="middle">(httpOnly, SameSite=Lax, Secure per PABLAN_COOKIE_SECURE)</text>
|
||||
<line x1="618" y1="489" x2="146" y2="489" class="edge-soft-accent" marker-end="url(#arrow-accent)"/>
|
||||
<line x1="28" y1="533" x2="972" y2="533" class="edge-soft"/>
|
||||
<rect x="336.325" y="522" width="327.35" height="22" rx="11" class="box-sunken" />
|
||||
<text x="500" y="537" class="t-muted" text-anchor="middle">Every page request: handle = sequence(auth, i18n)</text>
|
||||
<text x="264" y="565" class="t-edge" text-anchor="middle">GET /some-page (Cookie: pablan_session, Accept-Language)</text>
|
||||
<line x1="146" y1="582" x2="382" y2="582" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="500" y="604" class="t-edge" text-anchor="middle">GET /api/auth/me (absolute URL, cookie forwarded)</text>
|
||||
<line x1="382" y1="621" x2="618" y2="621" class="edge" marker-end="url(#arrow)"/>
|
||||
<rect x="408" y="637" width="351.95" height="40" rx="6" class="note" />
|
||||
<text x="421" y="654" class="t-body" text-anchor="start">The base URL must be absolute: a relative server-side</text>
|
||||
<text x="421" y="667" class="t-body" text-anchor="start">fetch never reaches the proxy.</text>
|
||||
<text x="736" y="695" class="t-edge" text-anchor="middle">SELECT auth_sessions + user, check expires_at</text>
|
||||
<line x1="618" y1="712" x2="854" y2="712" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="500" y="760" class="t-edge" text-anchor="middle">200 user (incl. locale)</text>
|
||||
<line x1="618" y1="777" x2="382" y2="777" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<rect x="408" y="793" width="444.2" height="53" rx="6" class="note" />
|
||||
<text x="421" y="810" class="t-body" text-anchor="start">Auth runs FIRST and stashes user.locale in a WeakMap keyed by the</text>
|
||||
<text x="421" y="823" class="t-body" text-anchor="start">Request: the locale strategy only receives the request, and a second</text>
|
||||
<text x="421" y="836" class="t-body" text-anchor="start">/me would ask a question we already asked.</text>
|
||||
<rect x="408" y="858" width="450.35" height="40" rx="6" class="note" />
|
||||
<text x="421" y="875" class="t-body" text-anchor="start">i18n: paraglideMiddleware resolves userPreference > cookie ></text>
|
||||
<text x="421" y="888" class="t-body" text-anchor="start">Accept-Language > base, then stamps %lang% / %dir% into the document.</text>
|
||||
<text x="264" y="916" class="t-edge" text-anchor="middle">render with locals.user, in the resolved locale</text>
|
||||
<line x1="382" y1="933" x2="146" y2="933" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<text x="500" y="997" class="t-edge" text-anchor="middle">401 {code: not_authenticated}</text>
|
||||
<line x1="618" y1="1014" x2="382" y2="1014" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<rect x="408" y="1030" width="438.05" height="27" rx="6" class="note" />
|
||||
<text x="421" y="1047" class="t-body" text-anchor="start">No user, so the locale falls back to the cookie or Accept-Language.</text>
|
||||
<text x="264" y="1075" class="t-edge" text-anchor="middle">303 redirect to /login</text>
|
||||
<line x1="382" y1="1092" x2="146" y2="1092" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<line x1="28" y1="1136" x2="972" y2="1136" class="edge-soft"/>
|
||||
<rect x="437.8" y="1125" width="124.4" height="22" rx="11" class="box-sunken" />
|
||||
<text x="500" y="1140" class="t-muted" text-anchor="middle">Ending a session</text>
|
||||
<rect x="290.2" y="1162" width="419.6" height="92" rx="6" class="note" />
|
||||
<text x="303.2" y="1179" class="t-body" text-anchor="start">Logout deletes the session row: revoked everywhere, instantly.</text>
|
||||
<text x="303.2" y="1192" class="t-body" text-anchor="start"></text>
|
||||
<text x="303.2" y="1205" class="t-body" text-anchor="start">Password change (POST /api/account/password): verify the current</text>
|
||||
<text x="303.2" y="1218" class="t-body" text-anchor="start">hash, store the new one, then DELETE auth_sessions WHERE</text>
|
||||
<text x="303.2" y="1231" class="t-body" text-anchor="start">user_id = me AND id <> my session. Other devices are logged out,</text>
|
||||
<text x="303.2" y="1244" class="t-body" text-anchor="start">this one keeps its cookie.</text>
|
||||
<rect x="28" y="242" width="944" height="125" rx="8" class="group" />
|
||||
<rect x="28" y="242" width="38.45" height="20" rx="6" class="box-sunken" />
|
||||
<text x="47.225" y="256" class="t-muted" text-anchor="middle">alt</text>
|
||||
<text x="78.45" y="256" class="t-muted" text-anchor="start">unknown email or wrong password</text>
|
||||
<rect x="28" y="375" width="944" height="138" rx="8" class="group" />
|
||||
<rect x="28" y="375" width="44.6" height="20" rx="6" class="box-sunken" />
|
||||
<text x="50.3" y="389" class="t-muted" text-anchor="middle">else</text>
|
||||
<text x="84.6" y="389" class="t-muted" text-anchor="start">credentials valid</text>
|
||||
<rect x="28" y="728" width="944" height="229" rx="8" class="group" />
|
||||
<rect x="28" y="728" width="38.45" height="20" rx="6" class="box-sunken" />
|
||||
<text x="47.225" y="742" class="t-muted" text-anchor="middle">alt</text>
|
||||
<text x="78.45" y="742" class="t-muted" text-anchor="start">session valid</text>
|
||||
<rect x="28" y="965" width="944" height="151" rx="8" class="group" />
|
||||
<rect x="28" y="965" width="44.6" height="20" rx="6" class="box-sunken" />
|
||||
<text x="50.3" y="979" class="t-muted" text-anchor="middle">else</text>
|
||||
<text x="84.6" y="979" class="t-muted" text-anchor="start">missing, expired or invalid</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 11 KiB |
@@ -0,0 +1,179 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1240 880" width="1240" height="880" role="img" aria-label="Pablan components">
|
||||
<title>Pablan components</title>
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--bg: #fbfaf9;
|
||||
--surface: #ffffff;
|
||||
--sunken: #f2f0ed;
|
||||
--ink: #1a1815;
|
||||
--muted: #63605a;
|
||||
--line: #c5bfb6;
|
||||
--line-soft: #e5e1db;
|
||||
--accent: #b8770a;
|
||||
--secondary: #b04a00;
|
||||
--note: #faf3e6;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #0f0e0d;
|
||||
--surface: #1a1817;
|
||||
--sunken: #232120;
|
||||
--ink: #efece8;
|
||||
--muted: #a49d94;
|
||||
--line: #4a443d;
|
||||
--line-soft: #332f2b;
|
||||
--accent: #e6b422;
|
||||
--secondary: #e08030;
|
||||
--note: #241f16;
|
||||
}
|
||||
}
|
||||
text { font-family: ui-sans-serif, system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif; fill: var(--ink); }
|
||||
.t-title { font-size: 15px; font-weight: 650; }
|
||||
.t-sub { font-size: 11.5px; fill: var(--muted); }
|
||||
.t-node { font-size: 12.5px; font-weight: 600; }
|
||||
.t-body { font-size: 11.5px; }
|
||||
.t-muted { font-size: 11px; fill: var(--muted); }
|
||||
.t-edge { font-size: 10.5px; fill: var(--muted); }
|
||||
.t-mono { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 11px; }
|
||||
.box { fill: var(--surface); stroke: var(--line); stroke-width: 1.2; }
|
||||
.box-sunken { fill: var(--sunken); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.box-accent { fill: var(--surface); stroke: var(--accent); stroke-width: 1.6; }
|
||||
.note { fill: var(--note); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.group { fill: none; stroke: var(--line-soft); stroke-width: 1.2;
|
||||
stroke-dasharray: 5 4; }
|
||||
.lifeline { stroke: var(--line); stroke-width: 1; stroke-dasharray: 4 4; }
|
||||
.edge { stroke: var(--line); stroke-width: 1.3; fill: none; }
|
||||
.edge-accent { stroke: var(--secondary); stroke-width: 1.5; fill: none; }
|
||||
.edge-soft { stroke: var(--line); stroke-width: 1.1; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.edge-soft-accent { stroke: var(--secondary); stroke-width: 1.3; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.planned { stroke-dasharray: 6 4; opacity: 0.62; }
|
||||
</style>
|
||||
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--line)"/>
|
||||
</marker>
|
||||
<marker id="arrow-accent" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--secondary)"/>
|
||||
</marker>
|
||||
<marker id="arrow-open" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9" fill="none" stroke="var(--line)"
|
||||
stroke-width="1.3"/>
|
||||
</marker>
|
||||
</defs>
|
||||
<rect width="1240" height="880" fill="var(--bg)"/>
|
||||
<text x="30" y="32" class="t-title" text-anchor="start">Components</text>
|
||||
<text x="30" y="52" class="t-sub" text-anchor="start">One deployment: browser, one proxy, two app processes, one database, and whatever speaks the OpenAI API.</text>
|
||||
<rect x="40" y="80" width="170" height="38" rx="7" class="box" />
|
||||
<text x="51" y="99" class="t-node" text-anchor="start">Browser</text>
|
||||
<rect x="40" y="148" width="1160" height="46" rx="7" class="box" />
|
||||
<text x="51" y="167" class="t-node" text-anchor="start">Reverse proxy: Caddy</text>
|
||||
<text x="51" y="182" class="t-muted" text-anchor="start">/api/* to the backend, everything else to the frontend; TLS for PABLAN_DOMAIN</text>
|
||||
<line x1="125" y1="118" x2="125" y2="146" class="edge" marker-end="url(#arrow)"/>
|
||||
<line x1="240" y1="194" x2="240" y2="240" class="edge" marker-end="url(#arrow)"/>
|
||||
<line x1="860" y1="194" x2="860" y2="240" class="edge" marker-end="url(#arrow)"/>
|
||||
<rect x="40" y="250" width="410" height="330" rx="10" class="group" />
|
||||
<rect x="54" y="239" width="284.4" height="22" rx="11" class="box-sunken" />
|
||||
<text x="196.2" y="254" class="t-muted" text-anchor="middle">SvelteKit frontend (no DB, no auth logic)</text>
|
||||
<rect x="56" y="272" width="378" height="48" rx="7" class="box" />
|
||||
<text x="67" y="291" class="t-node" text-anchor="start">hooks.server.ts</text>
|
||||
<text x="67" y="306" class="t-muted" text-anchor="start">sequence(auth gate, paraglide i18n); cookie passthrough</text>
|
||||
<rect x="56" y="332" width="378" height="48" rx="7" class="box" />
|
||||
<text x="67" y="351" class="t-node" text-anchor="start">typed API client</text>
|
||||
<text x="67" y="366" class="t-muted" text-anchor="start">openapi-fetch over generated types, never hand-written</text>
|
||||
<rect x="56" y="392" width="183" height="62" rx="7" class="box" />
|
||||
<text x="67" y="411" class="t-node" text-anchor="start">chat</text>
|
||||
<text x="67" y="426" class="t-muted" text-anchor="start">/chat, /chat/[id],</text>
|
||||
<text x="67" y="439" class="t-muted" text-anchor="start">one ChatView, SSE</text>
|
||||
<rect x="251" y="392" width="183" height="62" rx="7" class="box" />
|
||||
<text x="262" y="411" class="t-node" text-anchor="start">writing editor</text>
|
||||
<text x="262" y="426" class="t-muted" text-anchor="start">documents/[id]/edit,</text>
|
||||
<text x="262" y="439" class="t-muted" text-anchor="start">CodeMirror + refine SSE</text>
|
||||
<rect x="56" y="466" width="183" height="62" rx="7" class="box" />
|
||||
<text x="67" y="485" class="t-node" text-anchor="start">documents, people,</text>
|
||||
<text x="67" y="500" class="t-muted" text-anchor="start">admin, account</text>
|
||||
<text x="67" y="513" class="t-muted" text-anchor="start"></text>
|
||||
<rect x="251" y="466" width="183" height="62" rx="7" class="box" />
|
||||
<text x="262" y="485" class="t-node" text-anchor="start">paraglide messages</text>
|
||||
<text x="262" y="500" class="t-muted" text-anchor="start">de source, en in sync;</text>
|
||||
<text x="262" y="513" class="t-muted" text-anchor="start">app.css design tokens</text>
|
||||
<rect x="520" y="250" width="690" height="350" rx="10" class="group" />
|
||||
<rect x="534" y="239" width="444.4" height="22" rx="11" class="box-sunken" />
|
||||
<text x="756.2" y="254" class="t-muted" text-anchor="middle">FastAPI backend (single worker: queue and metrics are per process)</text>
|
||||
<rect x="536" y="272" width="658" height="48" rx="7" class="box" />
|
||||
<text x="547" y="291" class="t-node" text-anchor="start">api/ routers</text>
|
||||
<text x="547" y="306" class="t-muted" text-anchor="start">auth, account, people, documents, templates, admin, conversations (SSE), authoring (SSE)</text>
|
||||
<rect x="536" y="334" width="160" height="58" rx="7" class="box" />
|
||||
<text x="547" y="353" class="t-node" text-anchor="start">auth/</text>
|
||||
<text x="547" y="368" class="t-muted" text-anchor="start">argon2,</text>
|
||||
<text x="547" y="381" class="t-muted" text-anchor="start">auth_sessions</text>
|
||||
<rect x="708" y="334" width="160" height="58" rx="7" class="box" />
|
||||
<text x="719" y="353" class="t-node" text-anchor="start">modes/</text>
|
||||
<text x="719" y="368" class="t-muted" text-anchor="start">registry, query;</text>
|
||||
<text x="719" y="381" class="t-muted" text-anchor="start">yields ModeEvents</text>
|
||||
<rect x="880" y="334" width="160" height="58" rx="7" class="box" />
|
||||
<text x="891" y="353" class="t-node" text-anchor="start">authoring/</text>
|
||||
<text x="891" y="368" class="t-muted" text-anchor="start">template schema,</text>
|
||||
<text x="891" y="381" class="t-muted" text-anchor="start">section refinement</text>
|
||||
<rect x="1044" y="334" width="150" height="58" rx="7" class="box" />
|
||||
<text x="1055" y="353" class="t-node" text-anchor="start">ingestion/</text>
|
||||
<text x="1055" y="368" class="t-muted" text-anchor="start">jobs queue</text>
|
||||
<text x="1055" y="381" class="t-muted" text-anchor="start">+ handlers</text>
|
||||
<rect x="536" y="406" width="658" height="52" rx="7" class="box" />
|
||||
<text x="547" y="425" class="t-node" text-anchor="start">rag/</text>
|
||||
<text x="547" y="440" class="t-muted" text-anchor="start">chunking, indexing, permissions, retrieval (hybrid + text-only + similarity)</text>
|
||||
<rect x="536" y="470" width="494" height="52" rx="7" class="box" />
|
||||
<text x="547" y="489" class="t-node" text-anchor="start">llm/client.py</text>
|
||||
<text x="547" y="504" class="t-muted" text-anchor="start">the ONLY caller of an endpoint: chat_stream, chat_json, embed, probe</text>
|
||||
<rect x="1042" y="470" width="152" height="52" rx="7" class="box" />
|
||||
<text x="1053" y="489" class="t-node" text-anchor="start">llm/overrides.py</text>
|
||||
<text x="1053" y="504" class="t-muted" text-anchor="start">env bootstrap,</text>
|
||||
<text x="1053" y="517" class="t-muted" text-anchor="start">then the DB wins</text>
|
||||
<rect x="536" y="534" width="323" height="48" rx="7" class="box" />
|
||||
<text x="547" y="553" class="t-node" text-anchor="start">config, log, metrics</text>
|
||||
<text x="547" y="568" class="t-muted" text-anchor="start">content never reaches a log line</text>
|
||||
<rect x="871" y="534" width="323" height="48" rx="7" class="box planned" />
|
||||
<text x="882" y="553" class="t-node" text-anchor="start">ee_hooks.py</text>
|
||||
<text x="882" y="568" class="t-muted" text-anchor="start">optional pablan_ee import; core never imports ee/</text>
|
||||
<line x1="616" y1="320" x2="616" y2="332" class="edge-soft" marker-end="url(#arrow)"/>
|
||||
<line x1="783" y1="320" x2="783" y2="332" class="edge-soft" marker-end="url(#arrow)"/>
|
||||
<line x1="950" y1="320" x2="950" y2="332" class="edge-soft" marker-end="url(#arrow)"/>
|
||||
<line x1="1111" y1="320" x2="1111" y2="332" class="edge-soft" marker-end="url(#arrow)"/>
|
||||
<line x1="783" y1="392" x2="783" y2="404" class="edge" marker-end="url(#arrow)"/>
|
||||
<line x1="950" y1="392" x2="950" y2="404" class="edge" marker-end="url(#arrow)"/>
|
||||
<line x1="1111" y1="392" x2="1111" y2="404" class="edge" marker-end="url(#arrow)"/>
|
||||
<line x1="616" y1="458" x2="616" y2="468" class="edge" marker-end="url(#arrow)"/>
|
||||
<line x1="950" y1="458" x2="950" y2="468" class="edge" marker-end="url(#arrow)"/>
|
||||
<path d="M 450 296 H 520" class="edge-accent" marker-end="url(#arrow)"/>
|
||||
<text x="485" y="288" class="t-edge" text-anchor="middle">/api/auth/me</text>
|
||||
<path d="M 450 356 H 520" class="edge-accent" marker-end="url(#arrow)"/>
|
||||
<text x="485" y="348" class="t-edge" text-anchor="middle">same origin</text>
|
||||
<rect x="40" y="760" width="560" height="70" rx="7" class="box" />
|
||||
<text x="51" y="779" class="t-node" text-anchor="start">PostgreSQL + pgvector</text>
|
||||
<text x="51" y="794" class="t-muted" text-anchor="start">documents (Markdown, the source of truth), chunks (embedding + tsvector),</text>
|
||||
<text x="51" y="807" class="t-muted" text-anchor="start">auth_sessions, conversations, jobs, llm_settings, templates</text>
|
||||
<rect x="640" y="760" width="560" height="70" rx="7" class="box" />
|
||||
<text x="651" y="779" class="t-node" text-anchor="start">OpenAI-compatible endpoints</text>
|
||||
<text x="651" y="794" class="t-muted" text-anchor="start">three roles, each its own base_url + key + model:</text>
|
||||
<text x="651" y="807" class="t-muted" text-anchor="start">chat, utility, embedding (bge-m3). llama.cpp locally, any cloud API in production</text>
|
||||
<line x1="560" y1="600" x2="560" y2="758" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="566" y="668" class="t-edge" text-anchor="start">every table</text>
|
||||
<line x1="1185" y1="600" x2="1185" y2="758" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="1179" y="745" class="t-edge" text-anchor="end">llm/client.py only</text>
|
||||
<rect x="660" y="630" width="240" height="78" rx="7" class="box" />
|
||||
<text x="671" y="649" class="t-node" text-anchor="start">templates/</text>
|
||||
<text x="671" y="664" class="t-muted" text-anchor="start">blueprint catalog, de + en;</text>
|
||||
<text x="671" y="677" class="t-muted" text-anchor="start">inert until an admin adds one</text>
|
||||
<rect x="920" y="630" width="240" height="78" rx="7" class="box" />
|
||||
<text x="931" y="649" class="t-node" text-anchor="start">help/</text>
|
||||
<text x="931" y="664" class="t-muted" text-anchor="start">built-in help pages, German;</text>
|
||||
<text x="931" y="677" class="t-muted" text-anchor="start">re-imported on every start</text>
|
||||
<path d="M 780 630 V 604" class="edge-soft" marker-end="url(#arrow)"/>
|
||||
<path d="M 1040 630 V 604" class="edge-soft" marker-end="url(#arrow)"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 12 KiB |
@@ -0,0 +1,381 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1290 1090" width="1290" height="1090" role="img" aria-label="Pablan data model">
|
||||
<title>Pablan data model</title>
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--bg: #fbfaf9;
|
||||
--surface: #ffffff;
|
||||
--sunken: #f2f0ed;
|
||||
--ink: #1a1815;
|
||||
--muted: #63605a;
|
||||
--line: #c5bfb6;
|
||||
--line-soft: #e5e1db;
|
||||
--accent: #b8770a;
|
||||
--secondary: #b04a00;
|
||||
--note: #faf3e6;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #0f0e0d;
|
||||
--surface: #1a1817;
|
||||
--sunken: #232120;
|
||||
--ink: #efece8;
|
||||
--muted: #a49d94;
|
||||
--line: #4a443d;
|
||||
--line-soft: #332f2b;
|
||||
--accent: #e6b422;
|
||||
--secondary: #e08030;
|
||||
--note: #241f16;
|
||||
}
|
||||
}
|
||||
text { font-family: ui-sans-serif, system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif; fill: var(--ink); }
|
||||
.t-title { font-size: 15px; font-weight: 650; }
|
||||
.t-sub { font-size: 11.5px; fill: var(--muted); }
|
||||
.t-node { font-size: 12.5px; font-weight: 600; }
|
||||
.t-body { font-size: 11.5px; }
|
||||
.t-muted { font-size: 11px; fill: var(--muted); }
|
||||
.t-edge { font-size: 10.5px; fill: var(--muted); }
|
||||
.t-mono { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 11px; }
|
||||
.box { fill: var(--surface); stroke: var(--line); stroke-width: 1.2; }
|
||||
.box-sunken { fill: var(--sunken); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.box-accent { fill: var(--surface); stroke: var(--accent); stroke-width: 1.6; }
|
||||
.note { fill: var(--note); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.group { fill: none; stroke: var(--line-soft); stroke-width: 1.2;
|
||||
stroke-dasharray: 5 4; }
|
||||
.lifeline { stroke: var(--line); stroke-width: 1; stroke-dasharray: 4 4; }
|
||||
.edge { stroke: var(--line); stroke-width: 1.3; fill: none; }
|
||||
.edge-accent { stroke: var(--secondary); stroke-width: 1.5; fill: none; }
|
||||
.edge-soft { stroke: var(--line); stroke-width: 1.1; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.edge-soft-accent { stroke: var(--secondary); stroke-width: 1.3; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.planned { stroke-dasharray: 6 4; opacity: 0.62; }
|
||||
</style>
|
||||
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--line)"/>
|
||||
</marker>
|
||||
<marker id="arrow-accent" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--secondary)"/>
|
||||
</marker>
|
||||
<marker id="arrow-open" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9" fill="none" stroke="var(--line)"
|
||||
stroke-width="1.3"/>
|
||||
</marker>
|
||||
|
||||
<marker id="many" viewBox="0 0 12 12" refX="11" refY="6" markerWidth="11"
|
||||
markerHeight="11" orient="auto">
|
||||
<path d="M 1 1 L 11 6 M 1 6 L 11 6 M 1 11 L 11 6" fill="none"
|
||||
stroke="var(--line)" stroke-width="1.2"/>
|
||||
</marker>
|
||||
<marker id="one" viewBox="0 0 12 12" refX="2" refY="6" markerWidth="11"
|
||||
markerHeight="11" orient="auto">
|
||||
<path d="M 3 1 L 3 11" fill="none" stroke="var(--line)" stroke-width="1.4"/>
|
||||
</marker>
|
||||
</defs>
|
||||
<rect width="1290" height="1090" fill="var(--bg)"/>
|
||||
<text x="30" y="32" class="t-title" text-anchor="start">Data model</text>
|
||||
<text x="30" y="52" class="t-sub" text-anchor="start">Markdown in documents.content_md is the source of truth; chunks are a disposable derivative.</text>
|
||||
<path d="M 90 167 V 200" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="98" y="187.5" class="t-edge" text-anchor="start">employs</text>
|
||||
<path d="M 90 356 V 404" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="98" y="384" class="t-edge" text-anchor="start">logs in</text>
|
||||
<path d="M 130 356 V 520" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="138" y="442" class="t-edge" text-anchor="start">starts</text>
|
||||
<path d="M 90 625 V 653" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="98" y="643" class="t-edge" text-anchor="start">contains</text>
|
||||
<path d="M 530 303 V 404" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="538" y="374.5" class="t-edge" text-anchor="start">grants</text>
|
||||
<path d="M 590 303 V 520" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="598" y="441" class="t-edge" text-anchor="start">audited by</text>
|
||||
<path d="M 650 303 V 724" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="658" y="543" class="t-edge" text-anchor="start">derived into</text>
|
||||
<path d="M 410 136 H 422 V 130 H 470" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="462" y="130" class="t-edge" text-anchor="end">owns</text>
|
||||
<path d="M 410 158 H 434 V 464 H 470" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="462" y="152" class="t-edge" text-anchor="end">granted to</text>
|
||||
<path d="M 410 284 H 446 V 182 H 470" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="462" y="278" class="t-edge" text-anchor="end">authors</text>
|
||||
<path d="M 410 328 H 470 V 658 H 470" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="462" y="322" class="t-edge" text-anchor="end">acted</text>
|
||||
<path d="M 30 306 H 18 V 950 H 470" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text x="26" y="940" class="t-edge" text-anchor="start">is asked / answers</text>
|
||||
<path d="M 850 270 H 866 V 950 H 850" class="edge" marker-end="url(#many)" marker-start="url(#one)"/>
|
||||
<text transform="rotate(-90 870 700)" x="870" y="700" class="t-edge" text-anchor="middle">is questioned by</text>
|
||||
<rect x="30" y="96" width="380" height="71" rx="8" class="box" />
|
||||
<rect x="30" y="96" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="42" y="115" class="t-node" text-anchor="start">departments</text>
|
||||
<text x="42" y="137" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="182" y="137" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="137" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="42" y="154" class="t-mono" text-anchor="start">name</text>
|
||||
<text x="182" y="154" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="154" class="t-muted" text-anchor="start">unique</text>
|
||||
<rect x="30" y="200" width="380" height="156" rx="8" class="box" />
|
||||
<rect x="30" y="200" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="42" y="219" class="t-node" text-anchor="start">users</text>
|
||||
<text x="42" y="241" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="182" y="241" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="241" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="42" y="258" class="t-mono" text-anchor="start">email</text>
|
||||
<text x="182" y="258" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="258" class="t-muted" text-anchor="start">unique</text>
|
||||
<text x="42" y="275" class="t-mono" text-anchor="start">name</text>
|
||||
<text x="182" y="275" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="42" y="292" class="t-mono" text-anchor="start">role</text>
|
||||
<text x="182" y="292" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="292" class="t-muted" text-anchor="start">member | admin</text>
|
||||
<text x="42" y="309" class="t-mono" text-anchor="start">locale</text>
|
||||
<text x="182" y="309" class="t-mono" text-anchor="start">varchar</text>
|
||||
<text x="245" y="309" class="t-muted" text-anchor="start">de | en, null = browser</text>
|
||||
<text x="42" y="326" class="t-mono" text-anchor="start">password_hash</text>
|
||||
<text x="182" y="326" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="326" class="t-muted" text-anchor="start">argon2</text>
|
||||
<text x="42" y="343" class="t-mono" text-anchor="start">department_id</text>
|
||||
<text x="182" y="343" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="343" class="t-muted" text-anchor="start">FK, null, SET NULL</text>
|
||||
<rect x="30" y="404" width="380" height="88" rx="8" class="box" />
|
||||
<rect x="30" y="404" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="42" y="423" class="t-node" text-anchor="start">auth_sessions</text>
|
||||
<text x="42" y="445" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="182" y="445" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="445" class="t-muted" text-anchor="start">PK, = cookie token</text>
|
||||
<text x="42" y="462" class="t-mono" text-anchor="start">user_id</text>
|
||||
<text x="182" y="462" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="462" class="t-muted" text-anchor="start">FK, CASCADE</text>
|
||||
<text x="42" y="479" class="t-mono" text-anchor="start">expires_at</text>
|
||||
<text x="182" y="479" class="t-mono" text-anchor="start">timestamp</text>
|
||||
<text x="245" y="479" class="t-muted" text-anchor="start">TTL 14d, configurable</text>
|
||||
<rect x="30" y="520" width="380" height="105" rx="8" class="box" />
|
||||
<rect x="30" y="520" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="42" y="539" class="t-node" text-anchor="start">conversations</text>
|
||||
<text x="42" y="561" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="182" y="561" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="561" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="42" y="578" class="t-mono" text-anchor="start">mode</text>
|
||||
<text x="182" y="578" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="578" class="t-muted" text-anchor="start">query | insight</text>
|
||||
<text x="42" y="595" class="t-mono" text-anchor="start">status</text>
|
||||
<text x="182" y="595" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="595" class="t-muted" text-anchor="start">active | completed</text>
|
||||
<text x="42" y="612" class="t-mono" text-anchor="start">user_id</text>
|
||||
<text x="182" y="612" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="612" class="t-muted" text-anchor="start">FK, CASCADE</text>
|
||||
<rect x="30" y="653" width="380" height="122" rx="8" class="box" />
|
||||
<rect x="30" y="653" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="42" y="672" class="t-node" text-anchor="start">messages</text>
|
||||
<text x="42" y="694" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="182" y="694" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="694" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="42" y="711" class="t-mono" text-anchor="start">conversation_id</text>
|
||||
<text x="182" y="711" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="245" y="711" class="t-muted" text-anchor="start">FK, CASCADE</text>
|
||||
<text x="42" y="728" class="t-mono" text-anchor="start">role</text>
|
||||
<text x="182" y="728" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="245" y="728" class="t-muted" text-anchor="start">user/assistant/system</text>
|
||||
<text x="42" y="745" class="t-mono" text-anchor="start">content</text>
|
||||
<text x="182" y="745" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="42" y="762" class="t-mono" text-anchor="start">meta</text>
|
||||
<text x="182" y="762" class="t-mono" text-anchor="start">jsonb</text>
|
||||
<text x="245" y="762" class="t-muted" text-anchor="start">citations, fallback code</text>
|
||||
<rect x="470" y="96" width="380" height="207" rx="8" class="box" />
|
||||
<rect x="470" y="96" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="482" y="115" class="t-node" text-anchor="start">documents</text>
|
||||
<text x="482" y="137" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="622" y="137" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="137" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="482" y="154" class="t-mono" text-anchor="start">title</text>
|
||||
<text x="622" y="154" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="482" y="171" class="t-mono" text-anchor="start">status</text>
|
||||
<text x="622" y="171" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="685" y="171" class="t-muted" text-anchor="start">draft | published | archived</text>
|
||||
<text x="482" y="188" class="t-mono" text-anchor="start">is_builtin</text>
|
||||
<text x="622" y="188" class="t-mono" text-anchor="start">bool</text>
|
||||
<text x="685" y="188" class="t-muted" text-anchor="start">shipped help page</text>
|
||||
<text x="482" y="205" class="t-mono" text-anchor="start">visibility</text>
|
||||
<text x="622" y="205" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="685" y="205" class="t-muted" text-anchor="start">public | department |</text>
|
||||
<text x="685" y="222" class="t-muted" text-anchor="start">restricted</text>
|
||||
<text x="482" y="239" class="t-mono" text-anchor="start">content_md</text>
|
||||
<text x="622" y="239" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="685" y="239" class="t-muted" text-anchor="start">THE source of truth</text>
|
||||
<text x="482" y="256" class="t-mono" text-anchor="start">meta</text>
|
||||
<text x="622" y="256" class="t-mono" text-anchor="start">jsonb</text>
|
||||
<text x="685" y="256" class="t-muted" text-anchor="start">template, context, summary</text>
|
||||
<text x="482" y="273" class="t-mono" text-anchor="start">author_id</text>
|
||||
<text x="622" y="273" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="273" class="t-muted" text-anchor="start">FK, null, SET NULL</text>
|
||||
<text x="482" y="290" class="t-mono" text-anchor="start">department_id</text>
|
||||
<text x="622" y="290" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="290" class="t-muted" text-anchor="start">FK, null, SET NULL</text>
|
||||
<rect x="470" y="404" width="380" height="88" rx="8" class="box" />
|
||||
<rect x="470" y="404" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="482" y="423" class="t-node" text-anchor="start">doc_permissions</text>
|
||||
<text x="482" y="445" class="t-mono" text-anchor="start">document_id</text>
|
||||
<text x="622" y="445" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="445" class="t-muted" text-anchor="start">PK, FK, CASCADE</text>
|
||||
<text x="482" y="462" class="t-mono" text-anchor="start">department_id</text>
|
||||
<text x="622" y="462" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="462" class="t-muted" text-anchor="start">PK, FK, CASCADE</text>
|
||||
<text x="482" y="479" class="t-mono" text-anchor="start">level</text>
|
||||
<text x="622" y="479" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="685" y="479" class="t-muted" text-anchor="start">read</text>
|
||||
<rect x="470" y="520" width="380" height="173" rx="8" class="box" />
|
||||
<rect x="470" y="520" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="482" y="539" class="t-node" text-anchor="start">document_events</text>
|
||||
<text x="482" y="561" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="622" y="561" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="561" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="482" y="578" class="t-mono" text-anchor="start">document_id</text>
|
||||
<text x="622" y="578" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="578" class="t-muted" text-anchor="start">FK, CASCADE</text>
|
||||
<text x="482" y="595" class="t-mono" text-anchor="start">actor_id</text>
|
||||
<text x="622" y="595" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="595" class="t-muted" text-anchor="start">FK, null, SET NULL</text>
|
||||
<text x="482" y="612" class="t-mono" text-anchor="start">action</text>
|
||||
<text x="622" y="612" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="685" y="612" class="t-muted" text-anchor="start">created | edited | published |</text>
|
||||
<text x="685" y="629" class="t-muted" text-anchor="start">archived | visibility_changed |</text>
|
||||
<text x="685" y="646" class="t-muted" text-anchor="start">review_requested / _resolved</text>
|
||||
<text x="482" y="663" class="t-mono" text-anchor="start">content_md</text>
|
||||
<text x="622" y="663" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="685" y="663" class="t-muted" text-anchor="start">snapshot AFTER the event</text>
|
||||
<text x="482" y="680" class="t-mono" text-anchor="start">title / visibility / meta</text>
|
||||
<text x="685" y="680" class="t-muted" text-anchor="start">snapshot</text>
|
||||
<rect x="470" y="724" width="380" height="156" rx="8" class="box" />
|
||||
<rect x="470" y="724" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="482" y="743" class="t-node" text-anchor="start">chunks</text>
|
||||
<text x="482" y="765" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="622" y="765" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="765" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="482" y="782" class="t-mono" text-anchor="start">document_id</text>
|
||||
<text x="622" y="782" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="782" class="t-muted" text-anchor="start">FK, CASCADE</text>
|
||||
<text x="482" y="799" class="t-mono" text-anchor="start">chunk_index</text>
|
||||
<text x="622" y="799" class="t-mono" text-anchor="start">int</text>
|
||||
<text x="685" y="799" class="t-muted" text-anchor="start">unique per document</text>
|
||||
<text x="482" y="816" class="t-mono" text-anchor="start">content</text>
|
||||
<text x="622" y="816" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="482" y="833" class="t-mono" text-anchor="start">embedding</text>
|
||||
<text x="622" y="833" class="t-mono" text-anchor="start">vector</text>
|
||||
<text x="685" y="833" class="t-muted" text-anchor="start">1024 bge-m3, HNSW cosine</text>
|
||||
<text x="482" y="850" class="t-mono" text-anchor="start">tsv</text>
|
||||
<text x="622" y="850" class="t-mono" text-anchor="start">tsvector</text>
|
||||
<text x="685" y="850" class="t-muted" text-anchor="start">generated german, GIN</text>
|
||||
<text x="482" y="867" class="t-mono" text-anchor="start">meta</text>
|
||||
<text x="622" y="867" class="t-mono" text-anchor="start">jsonb</text>
|
||||
<text x="685" y="867" class="t-muted" text-anchor="start">heading path, filters</text>
|
||||
<rect x="470" y="904" width="380" height="156" rx="8" class="box" />
|
||||
<rect x="470" y="904" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="482" y="923" class="t-node" text-anchor="start">review_requests</text>
|
||||
<text x="482" y="945" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="622" y="945" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="945" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="482" y="962" class="t-mono" text-anchor="start">document_id</text>
|
||||
<text x="622" y="962" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="962" class="t-muted" text-anchor="start">FK, CASCADE</text>
|
||||
<text x="482" y="979" class="t-mono" text-anchor="start">requester_id</text>
|
||||
<text x="622" y="979" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="979" class="t-muted" text-anchor="start">FK, null, SET NULL</text>
|
||||
<text x="482" y="996" class="t-mono" text-anchor="start">reviewer_id</text>
|
||||
<text x="622" y="996" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="996" class="t-muted" text-anchor="start">FK, null, grants read+edit</text>
|
||||
<text x="482" y="1013" class="t-mono" text-anchor="start">question</text>
|
||||
<text x="622" y="1013" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="685" y="1013" class="t-muted" text-anchor="start">null, "still 14 days?"</text>
|
||||
<text x="482" y="1030" class="t-mono" text-anchor="start">resolved_at</text>
|
||||
<text x="622" y="1030" class="t-mono" text-anchor="start">timestamp</text>
|
||||
<text x="685" y="1030" class="t-muted" text-anchor="start">null while open</text>
|
||||
<text x="482" y="1047" class="t-mono" text-anchor="start">resolved_by_id</text>
|
||||
<text x="622" y="1047" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="685" y="1047" class="t-muted" text-anchor="start">FK, null, who checked it</text>
|
||||
<rect x="890" y="96" width="380" height="105" rx="8" class="box" />
|
||||
<rect x="890" y="96" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="902" y="115" class="t-node" text-anchor="start">templates</text>
|
||||
<text x="902" y="137" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="1042" y="137" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="1105" y="137" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="902" y="154" class="t-mono" text-anchor="start">name</text>
|
||||
<text x="1042" y="154" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="902" y="171" class="t-mono" text-anchor="start">version</text>
|
||||
<text x="1042" y="171" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="1105" y="171" class="t-muted" text-anchor="start">schema version, e.g. 1.0</text>
|
||||
<text x="902" y="188" class="t-mono" text-anchor="start">config</text>
|
||||
<text x="1042" y="188" class="t-mono" text-anchor="start">jsonb</text>
|
||||
<text x="1105" y="188" class="t-muted" text-anchor="start">authoring template</text>
|
||||
<rect x="890" y="226" width="380" height="173" rx="8" class="box" />
|
||||
<rect x="890" y="226" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="902" y="245" class="t-node" text-anchor="start">llm_settings</text>
|
||||
<text x="902" y="267" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="1042" y="267" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="1105" y="267" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="902" y="284" class="t-mono" text-anchor="start">role</text>
|
||||
<text x="1042" y="284" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="1105" y="284" class="t-muted" text-anchor="start">unique: chat/utility/embed</text>
|
||||
<text x="902" y="301" class="t-mono" text-anchor="start">base_url</text>
|
||||
<text x="1042" y="301" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="1105" y="301" class="t-muted" text-anchor="start">seeded from env at start</text>
|
||||
<text x="902" y="318" class="t-mono" text-anchor="start">model</text>
|
||||
<text x="1042" y="318" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="1105" y="318" class="t-muted" text-anchor="start">seeded from env at start</text>
|
||||
<text x="902" y="335" class="t-mono" text-anchor="start">api_key</text>
|
||||
<text x="1042" y="335" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="1105" y="335" class="t-muted" text-anchor="start">plaintext, never returned</text>
|
||||
<text x="902" y="352" class="t-mono" text-anchor="start">base_url_from_env</text>
|
||||
<text x="1042" y="352" class="t-mono" text-anchor="start">bool</text>
|
||||
<text x="1105" y="352" class="t-muted" text-anchor="start">provenance, drives reset</text>
|
||||
<text x="902" y="369" class="t-mono" text-anchor="start">model_from_env</text>
|
||||
<text x="1042" y="369" class="t-mono" text-anchor="start">bool</text>
|
||||
<text x="1105" y="369" class="t-muted" text-anchor="start">provenance, drives reset</text>
|
||||
<text x="902" y="386" class="t-mono" text-anchor="start">api_key_from_env</text>
|
||||
<text x="1042" y="386" class="t-mono" text-anchor="start">bool</text>
|
||||
<text x="1105" y="386" class="t-muted" text-anchor="start">provenance, drives reset</text>
|
||||
<rect x="890" y="430" width="380" height="88" rx="8" class="box" />
|
||||
<rect x="890" y="430" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="902" y="449" class="t-node" text-anchor="start">prompt_settings</text>
|
||||
<text x="902" y="471" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="1042" y="471" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="1105" y="471" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="902" y="488" class="t-mono" text-anchor="start">key</text>
|
||||
<text x="1042" y="488" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="1105" y="488" class="t-muted" text-anchor="start">unique, e.g. query_system</text>
|
||||
<text x="902" y="505" class="t-mono" text-anchor="start">content</text>
|
||||
<text x="1042" y="505" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="1105" y="505" class="t-muted" text-anchor="start">full replacement text</text>
|
||||
<rect x="890" y="530" width="380" height="156" rx="8" class="box" />
|
||||
<rect x="890" y="530" width="380" height="28" rx="8" class="box-sunken" />
|
||||
<text x="902" y="549" class="t-node" text-anchor="start">jobs</text>
|
||||
<text x="902" y="571" class="t-mono" text-anchor="start">id</text>
|
||||
<text x="1042" y="571" class="t-mono" text-anchor="start">uuid</text>
|
||||
<text x="1105" y="571" class="t-muted" text-anchor="start">PK</text>
|
||||
<text x="902" y="588" class="t-mono" text-anchor="start">type</text>
|
||||
<text x="1042" y="588" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="902" y="605" class="t-mono" text-anchor="start">payload</text>
|
||||
<text x="1042" y="605" class="t-mono" text-anchor="start">jsonb</text>
|
||||
<text x="902" y="622" class="t-mono" text-anchor="start">status</text>
|
||||
<text x="1042" y="622" class="t-mono" text-anchor="start">string</text>
|
||||
<text x="1105" y="622" class="t-muted" text-anchor="start">pending/running/done/failed</text>
|
||||
<text x="902" y="639" class="t-mono" text-anchor="start">run_after</text>
|
||||
<text x="1042" y="639" class="t-mono" text-anchor="start">timestamp</text>
|
||||
<text x="1105" y="639" class="t-muted" text-anchor="start">idx (status, run_after)</text>
|
||||
<text x="902" y="656" class="t-mono" text-anchor="start">attempts</text>
|
||||
<text x="1042" y="656" class="t-mono" text-anchor="start">int</text>
|
||||
<text x="902" y="673" class="t-mono" text-anchor="start">last_error</text>
|
||||
<text x="1042" y="673" class="t-mono" text-anchor="start">text</text>
|
||||
<text x="1105" y="673" class="t-muted" text-anchor="start">sanitized, never content</text>
|
||||
<rect x="890" y="904" width="380" height="156" rx="8" class="note" />
|
||||
<text x="904" y="926" class="t-node" text-anchor="start">Reading the lines</text>
|
||||
<text x="904" y="945" class="t-muted" text-anchor="start">A bar is the one side, a fan the many side.</text>
|
||||
<text x="904" y="958" class="t-muted" text-anchor="start">Optionality is on the column instead: a note</text>
|
||||
<text x="904" y="971" class="t-muted" text-anchor="start">saying null (and SET NULL vs CASCADE) is what</text>
|
||||
<text x="904" y="984" class="t-muted" text-anchor="start">actually matters when a row is deleted.</text>
|
||||
<text x="904" y="997" class="t-muted" text-anchor="start">templates, llm_settings, prompt_settings and</text>
|
||||
<text x="904" y="1010" class="t-muted" text-anchor="start">jobs stand alone: config and work, not knowledge.</text>
|
||||
<text x="904" y="1029" class="t-muted" text-anchor="start">An open review_request is not a document status:</text>
|
||||
<text x="904" y="1042" class="t-muted" text-anchor="start">it can hang on a draft or on one published long ago,</text>
|
||||
<text x="904" y="1055" class="t-muted" text-anchor="start">and it marks the document everywhere it appears.</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 26 KiB |
@@ -0,0 +1,126 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 810" width="860" height="810" role="img" aria-label="Background jobs: one Postgres table, no broker">
|
||||
<title>Background jobs: one Postgres table, no broker</title>
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--bg: #fbfaf9;
|
||||
--surface: #ffffff;
|
||||
--sunken: #f2f0ed;
|
||||
--ink: #1a1815;
|
||||
--muted: #63605a;
|
||||
--line: #c5bfb6;
|
||||
--line-soft: #e5e1db;
|
||||
--accent: #b8770a;
|
||||
--secondary: #b04a00;
|
||||
--note: #faf3e6;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #0f0e0d;
|
||||
--surface: #1a1817;
|
||||
--sunken: #232120;
|
||||
--ink: #efece8;
|
||||
--muted: #a49d94;
|
||||
--line: #4a443d;
|
||||
--line-soft: #332f2b;
|
||||
--accent: #e6b422;
|
||||
--secondary: #e08030;
|
||||
--note: #241f16;
|
||||
}
|
||||
}
|
||||
text { font-family: ui-sans-serif, system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif; fill: var(--ink); }
|
||||
.t-title { font-size: 15px; font-weight: 650; }
|
||||
.t-sub { font-size: 11.5px; fill: var(--muted); }
|
||||
.t-node { font-size: 12.5px; font-weight: 600; }
|
||||
.t-body { font-size: 11.5px; }
|
||||
.t-muted { font-size: 11px; fill: var(--muted); }
|
||||
.t-edge { font-size: 10.5px; fill: var(--muted); }
|
||||
.t-mono { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 11px; }
|
||||
.box { fill: var(--surface); stroke: var(--line); stroke-width: 1.2; }
|
||||
.box-sunken { fill: var(--sunken); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.box-accent { fill: var(--surface); stroke: var(--accent); stroke-width: 1.6; }
|
||||
.note { fill: var(--note); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.group { fill: none; stroke: var(--line-soft); stroke-width: 1.2;
|
||||
stroke-dasharray: 5 4; }
|
||||
.lifeline { stroke: var(--line); stroke-width: 1; stroke-dasharray: 4 4; }
|
||||
.edge { stroke: var(--line); stroke-width: 1.3; fill: none; }
|
||||
.edge-accent { stroke: var(--secondary); stroke-width: 1.5; fill: none; }
|
||||
.edge-soft { stroke: var(--line); stroke-width: 1.1; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.edge-soft-accent { stroke: var(--secondary); stroke-width: 1.3; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.planned { stroke-dasharray: 6 4; opacity: 0.62; }
|
||||
</style>
|
||||
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--line)"/>
|
||||
</marker>
|
||||
<marker id="arrow-accent" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--secondary)"/>
|
||||
</marker>
|
||||
<marker id="arrow-open" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9" fill="none" stroke="var(--line)"
|
||||
stroke-width="1.3"/>
|
||||
</marker>
|
||||
</defs>
|
||||
<rect width="860" height="810" fill="var(--bg)"/>
|
||||
<text x="28" y="30" class="t-title" text-anchor="start">Background jobs: one Postgres table, no broker</text>
|
||||
<text x="28" y="50" class="t-sub" text-anchor="start">FOR UPDATE SKIP LOCKED, and the handler runs inside the claim transaction.</text>
|
||||
<rect x="114" y="74" width="96" height="30" rx="8" class="box" />
|
||||
<text x="162" y="93" class="t-node" text-anchor="middle">Queue loop</text>
|
||||
<line x1="162" y1="104" x2="162" y2="792" class="lifeline"/>
|
||||
<rect x="382" y="74" width="96" height="30" rx="8" class="box" />
|
||||
<text x="430" y="93" class="t-node" text-anchor="middle">Postgres</text>
|
||||
<line x1="430" y1="104" x2="430" y2="792" class="lifeline"/>
|
||||
<rect x="648.7" y="74" width="98.6" height="30" rx="8" class="box" />
|
||||
<text x="698" y="93" class="t-node" text-anchor="middle">Job handler</text>
|
||||
<line x1="698" y1="104" x2="698" y2="792" class="lifeline"/>
|
||||
<text x="296" y="158" class="t-edge" text-anchor="middle">BEGIN</text>
|
||||
<line x1="162" y1="175" x2="430" y2="175" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="296" y="197" class="t-edge" text-anchor="middle">SELECT job WHERE status=pending AND run_after<=now()</text>
|
||||
<text x="296" y="210" class="t-edge" text-anchor="middle">ORDER BY run_after LIMIT 1 FOR UPDATE SKIP LOCKED</text>
|
||||
<line x1="162" y1="227" x2="430" y2="227" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="296" y="275" class="t-edge" text-anchor="middle">ROLLBACK</text>
|
||||
<line x1="162" y1="292" x2="430" y2="292" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="208" y="316" class="t-edge" text-anchor="start">wait one poll interval</text>
|
||||
<path d="M 162.0 319 h 34 v 22 h -34" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="430" y="403" class="t-edge" text-anchor="middle">handler(db, job), INSIDE the open claim transaction</text>
|
||||
<line x1="162" y1="420" x2="698" y2="420" class="edge-accent" marker-end="url(#arrow-accent)"/>
|
||||
<text x="296" y="468" class="t-edge" text-anchor="middle">status=done, attempts+1, COMMIT</text>
|
||||
<text x="296" y="481" class="t-edge" text-anchor="middle">(the handler's writes commit atomically with the job)</text>
|
||||
<line x1="162" y1="498" x2="430" y2="498" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="296" y="562" class="t-edge" text-anchor="middle">ROLLBACK (the handler's writes go too)</text>
|
||||
<line x1="162" y1="579" x2="430" y2="579" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="296" y="601" class="t-edge" text-anchor="middle">follow-up tx: attempts+1, last_error (sanitized),</text>
|
||||
<text x="296" y="614" class="t-edge" text-anchor="middle">run_after = now + 30s * 2^(attempts-1)</text>
|
||||
<line x1="162" y1="631" x2="430" y2="631" class="edge" marker-end="url(#arrow)"/>
|
||||
<rect x="456" y="647" width="222.8" height="27" rx="6" class="note" />
|
||||
<text x="469" y="664" class="t-body" text-anchor="start">After 5 attempts: status=failed.</text>
|
||||
<rect x="214.05" y="734" width="431.9" height="40" rx="6" class="note" />
|
||||
<text x="227.05" y="751" class="t-body" text-anchor="start">Crash mid-job: the open transaction aborts, the row lock releases,</text>
|
||||
<text x="227.05" y="764" class="t-body" text-anchor="start">and the job is still pending and claimable after the restart.</text>
|
||||
<rect x="28" y="243" width="804" height="120" rx="8" class="group" />
|
||||
<rect x="28" y="243" width="38.45" height="20" rx="6" class="box-sunken" />
|
||||
<text x="47.225" y="257" class="t-muted" text-anchor="middle">alt</text>
|
||||
<text x="78.45" y="257" class="t-muted" text-anchor="start">no claimable job</text>
|
||||
<rect x="28" y="436" width="804" height="86" rx="8" class="group" />
|
||||
<rect x="28" y="436" width="38.45" height="20" rx="6" class="box-sunken" />
|
||||
<text x="47.225" y="450" class="t-muted" text-anchor="middle">alt</text>
|
||||
<text x="78.45" y="450" class="t-muted" text-anchor="start">handler succeeds</text>
|
||||
<rect x="28" y="530" width="804" height="164" rx="8" class="group" />
|
||||
<rect x="28" y="530" width="44.6" height="20" rx="6" class="box-sunken" />
|
||||
<text x="50.3" y="544" class="t-muted" text-anchor="middle">else</text>
|
||||
<text x="84.6" y="544" class="t-muted" text-anchor="start">handler raises</text>
|
||||
<rect x="28" y="371" width="804" height="339" rx="8" class="group" />
|
||||
<rect x="28" y="371" width="44.6" height="20" rx="6" class="box-sunken" />
|
||||
<text x="50.3" y="385" class="t-muted" text-anchor="middle">else</text>
|
||||
<text x="84.6" y="385" class="t-muted" text-anchor="start">job claimed, row lock held</text>
|
||||
<rect x="28" y="126" width="804" height="600" rx="8" class="group" />
|
||||
<rect x="28" y="126" width="44.6" height="20" rx="6" class="box-sunken" />
|
||||
<text x="50.3" y="140" class="t-muted" text-anchor="middle">loop</text>
|
||||
<text x="84.6" y="140" class="t-muted" text-anchor="start">every poll interval, default 1s</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 7.7 KiB |
@@ -0,0 +1,157 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1040 1356" width="1040" height="1356" role="img" aria-label="Retrieval: hybrid search, and the similarity path next to it">
|
||||
<title>Retrieval: hybrid search, and the similarity path next to it</title>
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--bg: #fbfaf9;
|
||||
--surface: #ffffff;
|
||||
--sunken: #f2f0ed;
|
||||
--ink: #1a1815;
|
||||
--muted: #63605a;
|
||||
--line: #c5bfb6;
|
||||
--line-soft: #e5e1db;
|
||||
--accent: #b8770a;
|
||||
--secondary: #b04a00;
|
||||
--note: #faf3e6;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #0f0e0d;
|
||||
--surface: #1a1817;
|
||||
--sunken: #232120;
|
||||
--ink: #efece8;
|
||||
--muted: #a49d94;
|
||||
--line: #4a443d;
|
||||
--line-soft: #332f2b;
|
||||
--accent: #e6b422;
|
||||
--secondary: #e08030;
|
||||
--note: #241f16;
|
||||
}
|
||||
}
|
||||
text { font-family: ui-sans-serif, system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif; fill: var(--ink); }
|
||||
.t-title { font-size: 15px; font-weight: 650; }
|
||||
.t-sub { font-size: 11.5px; fill: var(--muted); }
|
||||
.t-node { font-size: 12.5px; font-weight: 600; }
|
||||
.t-body { font-size: 11.5px; }
|
||||
.t-muted { font-size: 11px; fill: var(--muted); }
|
||||
.t-edge { font-size: 10.5px; fill: var(--muted); }
|
||||
.t-mono { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 11px; }
|
||||
.box { fill: var(--surface); stroke: var(--line); stroke-width: 1.2; }
|
||||
.box-sunken { fill: var(--sunken); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.box-accent { fill: var(--surface); stroke: var(--accent); stroke-width: 1.6; }
|
||||
.note { fill: var(--note); stroke: var(--line-soft); stroke-width: 1; }
|
||||
.group { fill: none; stroke: var(--line-soft); stroke-width: 1.2;
|
||||
stroke-dasharray: 5 4; }
|
||||
.lifeline { stroke: var(--line); stroke-width: 1; stroke-dasharray: 4 4; }
|
||||
.edge { stroke: var(--line); stroke-width: 1.3; fill: none; }
|
||||
.edge-accent { stroke: var(--secondary); stroke-width: 1.5; fill: none; }
|
||||
.edge-soft { stroke: var(--line); stroke-width: 1.1; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.edge-soft-accent { stroke: var(--secondary); stroke-width: 1.3; fill: none;
|
||||
stroke-dasharray: 5 4; }
|
||||
.planned { stroke-dasharray: 6 4; opacity: 0.62; }
|
||||
</style>
|
||||
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--line)"/>
|
||||
</marker>
|
||||
<marker id="arrow-accent" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9 z" fill="var(--secondary)"/>
|
||||
</marker>
|
||||
<marker id="arrow-open" viewBox="0 0 10 10" refX="9" refY="5"
|
||||
markerWidth="8" markerHeight="8" orient="auto-start-reverse">
|
||||
<path d="M 0 1 L 9 5 L 0 9" fill="none" stroke="var(--line)"
|
||||
stroke-width="1.3"/>
|
||||
</marker>
|
||||
</defs>
|
||||
<rect width="1040" height="1356" fill="var(--bg)"/>
|
||||
<text x="28" y="30" class="t-title" text-anchor="start">Retrieval: hybrid search, and the similarity path next to it</text>
|
||||
<text x="28" y="50" class="t-sub" text-anchor="start">The permission CTE is part of the one statement. There is no search without a user.</text>
|
||||
<rect x="75.3" y="74" width="151.4" height="30" rx="8" class="box" />
|
||||
<text x="151" y="93" class="t-node" text-anchor="middle">Caller (mode / API)</text>
|
||||
<line x1="151" y1="104" x2="151" y2="1338" class="lifeline"/>
|
||||
<rect x="298.2" y="74" width="197.6" height="30" rx="8" class="box" />
|
||||
<text x="397" y="93" class="t-node" text-anchor="middle">rag/retrieval + similarity</text>
|
||||
<line x1="397" y1="104" x2="397" y2="1338" class="lifeline"/>
|
||||
<rect x="577.2" y="74" width="131.6" height="30" rx="8" class="box" />
|
||||
<text x="643" y="93" class="t-node" text-anchor="middle">llm/client.embed</text>
|
||||
<line x1="643" y1="104" x2="643" y2="1338" class="lifeline"/>
|
||||
<rect x="841" y="74" width="96" height="30" rx="8" class="box" />
|
||||
<text x="889" y="93" class="t-node" text-anchor="middle">Postgres</text>
|
||||
<line x1="889" y1="104" x2="889" y2="1338" class="lifeline"/>
|
||||
<line x1="28" y1="138" x2="1012" y2="138" class="edge-soft"/>
|
||||
<rect x="417.825" y="127" width="204.35" height="22" rx="11" class="box-sunken" />
|
||||
<text x="520" y="142" class="t-muted" text-anchor="middle">search(): hybrid, the default</text>
|
||||
<text x="274" y="170" class="t-edge" text-anchor="middle">search(db, query, user=..., top_k=5)</text>
|
||||
<line x1="151" y1="187" x2="397" y2="187" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="520" y="209" class="t-edge" text-anchor="middle">embed([query])</text>
|
||||
<line x1="397" y1="226" x2="643" y2="226" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="520" y="248" class="t-edge" text-anchor="middle">query vector (1024, bge-m3)</text>
|
||||
<line x1="643" y1="265" x2="397" y2="265" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<text x="643" y="287" class="t-edge" text-anchor="middle">ONE statement:</text>
|
||||
<line x1="397" y1="304" x2="889" y2="304" class="edge-accent" marker-end="url(#arrow-accent)"/>
|
||||
<rect x="423" y="320" width="419.6" height="53" rx="6" class="note" />
|
||||
<text x="436" y="337" class="t-body" text-anchor="start">1. CTE "allowed": the LIVE documents table. status=published AND</text>
|
||||
<text x="436" y="350" class="t-body" text-anchor="start"> (public | own department | doc_permissions grant | author).</text>
|
||||
<text x="436" y="363" class="t-body" text-anchor="start"> Never chunk meta, so a change applies without reindexing.</text>
|
||||
<rect x="423" y="385" width="364.25" height="40" rx="6" class="note" />
|
||||
<text x="436" y="402" class="t-body" text-anchor="start">2. vector candidates: top-20 by cosine distance (HNSW),</text>
|
||||
<text x="436" y="415" class="t-body" text-anchor="start"> joined to allowed.</text>
|
||||
<rect x="423" y="437" width="395" height="40" rx="6" class="note" />
|
||||
<text x="436" y="454" class="t-body" text-anchor="start">3. FTS candidates: top-20 by ts_rank_cd over</text>
|
||||
<text x="436" y="467" class="t-body" text-anchor="start"> websearch_to_tsquery('german', query), joined to allowed.</text>
|
||||
<rect x="423" y="489" width="450.35" height="27" rx="6" class="note" />
|
||||
<text x="436" y="506" class="t-body" text-anchor="start">4. RRF merge: score = sum of 1/(60+rank), ORDER BY score LIMIT top_k.</text>
|
||||
<text x="643" y="534" class="t-edge" text-anchor="middle">rows: chunk, document, title, meta, score,</text>
|
||||
<text x="643" y="547" class="t-edge" text-anchor="middle">vector distance, fts rank</text>
|
||||
<line x1="889" y1="564" x2="397" y2="564" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<text x="274" y="586" class="t-edge" text-anchor="middle">SearchResult[] with citation metadata</text>
|
||||
<text x="274" y="599" class="t-edge" text-anchor="middle">(title, heading_path, score, vector_distance, fts_match)</text>
|
||||
<line x1="397" y1="616" x2="151" y2="616" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<rect x="304.05" y="632" width="431.9" height="53" rx="6" class="note" />
|
||||
<text x="317.05" y="649" class="t-body" text-anchor="start">No-answer signal: the top result has fts_match=false AND</text>
|
||||
<text x="317.05" y="662" class="t-body" text-anchor="start">vector_distance >= 0.45 (calibrated for bge-m3). Callers treat the</text>
|
||||
<text x="317.05" y="675" class="t-body" text-anchor="start">whole set as low confidence rather than citing it.</text>
|
||||
<line x1="28" y1="709" x2="1012" y2="709" class="edge-soft"/>
|
||||
<rect x="353.25" y="698" width="333.5" height="22" rx="11" class="box-sunken" />
|
||||
<text x="520" y="713" class="t-muted" text-anchor="middle">text_search(): the same query with no model at all</text>
|
||||
<text x="274" y="741" class="t-edge" text-anchor="middle">text_search(db, query, user=...)</text>
|
||||
<line x1="151" y1="758" x2="397" y2="758" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="643" y="780" class="t-edge" text-anchor="middle">SAME allowed CTE, then the german tsvector GIN index alone</text>
|
||||
<line x1="397" y1="797" x2="889" y2="797" class="edge" marker-end="url(#arrow)"/>
|
||||
<rect x="300.975" y="813" width="438.05" height="53" rx="6" class="note" />
|
||||
<text x="313.975" y="830" class="t-body" text-anchor="start">The fallback for a dead embedding endpoint: keywords, not meaning,</text>
|
||||
<text x="313.975" y="843" class="t-body" text-anchor="start">so the user is told it was a search without a model. No embed call,</text>
|
||||
<text x="313.975" y="856" class="t-body" text-anchor="start">which is why it works when nothing runs behind the LLM config.</text>
|
||||
<line x1="28" y1="890" x2="1012" y2="890" class="edge-soft"/>
|
||||
<rect x="390.15" y="879" width="259.7" height="22" rx="11" class="box-sunken" />
|
||||
<text x="520" y="894" class="t-muted" text-anchor="middle">rag/similarity: threshold, not ranking</text>
|
||||
<text x="274" y="922" class="t-edge" text-anchor="middle">similar_chunks(db, text, user=..., max_distance=<one of two constants>)</text>
|
||||
<line x1="151" y1="939" x2="397" y2="939" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="520" y="961" class="t-edge" text-anchor="middle">embed([text])</text>
|
||||
<line x1="397" y1="978" x2="643" y2="978" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="643" y="1000" class="t-edge" text-anchor="middle">SAME allowed CTE, then pure vector:</text>
|
||||
<text x="643" y="1013" class="t-edge" text-anchor="middle">ORDER BY cosine distance LIMIT top_k</text>
|
||||
<line x1="397" y1="1030" x2="889" y2="1030" class="edge" marker-end="url(#arrow)"/>
|
||||
<text x="643" y="1052" class="t-edge" text-anchor="middle">rows with a distance that is ALWAYS present</text>
|
||||
<line x1="889" y1="1069" x2="397" y2="1069" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<text x="274" y="1091" class="t-edge" text-anchor="middle">SimilarChunk[] filtered to distance <= max_distance</text>
|
||||
<text x="274" y="1104" class="t-edge" text-anchor="middle">(similar_documents groups per document, keeping the min)</text>
|
||||
<line x1="397" y1="1121" x2="151" y2="1121" class="edge-soft" marker-end="url(#arrow-open)"/>
|
||||
<rect x="288.675" y="1137" width="462.65" height="92" rx="6" class="note" />
|
||||
<text x="301.675" y="1154" class="t-body" text-anchor="start">Why not the RRF path: its score is a fusion rank, not a similarity,</text>
|
||||
<text x="301.675" y="1167" class="t-body" text-anchor="start">and vector_distance is null for FTS-only hits. A threshold needs a</text>
|
||||
<text x="301.675" y="1180" class="t-body" text-anchor="start">comparable number. The permission filter is shared; only the ranking</text>
|
||||
<text x="301.675" y="1193" class="t-body" text-anchor="start">differs. Two calibrated constants exist, CAPTURE_CONTEXT_MAX_DISTANCE</text>
|
||||
<text x="301.675" y="1206" class="t-body" text-anchor="start">(loose) and DUPLICATE_MAX_DISTANCE (tight); exclude_builtin drops</text>
|
||||
<text x="301.675" y="1219" class="t-body" text-anchor="start">Pablan's own help pages, because a capture asks what the COMPANY knows.</text>
|
||||
<rect x="294.825" y="1241" width="450.35" height="79" rx="6" class="note" />
|
||||
<text x="307.825" y="1258" class="t-body" text-anchor="start">Document search (GET /api/documents/search) is the same call with a</text>
|
||||
<text x="307.825" y="1271" class="t-body" text-anchor="start">larger top_k, grouped per document: one code path, so browsing and</text>
|
||||
<text x="307.825" y="1284" class="t-body" text-anchor="start">chat can never disagree on access. Query mode surfaces the run as SSE</text>
|
||||
<text x="307.825" y="1297" class="t-body" text-anchor="start">state events (searching, results(count) | no_answer, answering).</text>
|
||||
<text x="307.825" y="1310" class="t-body" text-anchor="start">Counts only, never the query text or a passage (rule 12).</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 12 KiB |
Reference in New Issue
Block a user