Obsidian
Obsidian – AI Governance and Audit Platform
Every LLM call, audited and on budget.
A governance layer for LLM apps. Each query is screened for personal data and prompt injection, kept within its agent’s budget, routed to a suitable model and logged with a trust score.
- Type
- Hackathon build
- Team
- Team of 4
- Core stack
- Next.js 15 · React 19 · TypeScript
Problem and solution
The problem
Teams adding LLM features rarely see what each call costs, which model answered or whether a prompt leaked personal data. Budgets overrun quietly, and there is no record to show a reviewer.
The solution
Obsidian sits between the application and the model. It screens each query, enforces a per-agent budget, switches to a cheaper model when costs climb, and records every decision in an audit log that the dashboard turns into spend, latency and trust metrics.
How it’s built
- 01Next.js dashboardOverview, events, analytics and trust score
- 02API proxy routeForwards dashboard calls to the backend
- 03FastAPI + cascadeflowPolicy, budget and routing decisions
- 04Groq modelsConfigurable default and fallback models
What it does
Governance
- Policy checksFlags card numbers, SSNs and emails, and blocks known prompt-injection phrases.
- Budget capsHard limits per agent; queries stop when the budget runs out.
- Auto-remediationMoves a category to a cheaper model when costs escalate.
- Agent isolationSeparate budgets, logs and scores for each agent.
Visibility
- Audit trailModel, cost, latency and action for every query.
- Trust scoreCompliance 40%, cost efficiency 30% and recall 30%, shown with its formula.
- ROI viewActual spend against a worst-case baseline.
- Slack alertsAdmins hear about compliance stops and low budgets.
Integration
- OpenAI-compatible gatewayA /v1/chat/completions endpoint behind an API key.
- Terminal dashboardA CLI with a live cost chart and event log.
- Ask ObsidianTry a query and watch the policy decide.
Under the hood
Budget checked before the call
Built on the cascadeflow harness, which checks the budget before each model call, so a query that would overspend never reaches the model.
Cost-aware routing
Queries are classified by category and routed between a default model and a cheaper fallback on Groq (Llama 3.3 70B and 3.1 8B out of the box), and every automatic switch is logged.
Drop-in adoption
Because the gateway speaks the OpenAI chat API, an existing client can route through Obsidian by changing its base URL.
Explainable scoring
The trust score is shown with its weighted formula and recall checks instead of as an opaque number.
Built with
- Next.js 15
- React 19
- TypeScript
- Tailwind CSS
- Recharts
- Framer Motion
- Python
- FastAPI
- Pydantic
- cascadeflow
- Groq API
- Node.js CLI
- blessed-contrib
- Slack webhooks
- Vercel
My part
I built the Ask Obsidian query page, the dark dashboard redesign with its Platform and Health topology pages, and the terminal CLI. On the backend I added the OpenAI-compatible gateway and the admin endpoints, and fixed PII and jailbreak blocking.