According to a recent LinkedIn post from Supabase, the company has open-sourced Supabase Evals, a framework designed to benchmark how effectively AI coding agents work with its platform. The post indicates that agents such as Claude Code, Codex, and OpenCode are increasingly central to building with Supabase via its MCP server, CLI, agent skills, and documentation.
The LinkedIn post explains that Supabase previously relied on manual testing and intuition to assess agent performance, but Supabase Evals now provides a more systematic measurement. The tool runs agents against realistic tasks, including creating database schemas, debugging Edge Functions, and correcting Row Level Security policies, and then scores their actions using deterministic checks combined with LLM-based judgment.
According to the post, all evaluation scenarios operate against real Supabase environments, ensuring agents interact with production-like MCP and CLI workflows. Scenarios are said to be derived from actual user pain points, including support tickets, bug reports, and GitHub issues, and results can be browsed and filtered by model, product, stage, or evaluation type.
For investors, the open-sourcing of Supabase Evals suggests an effort to strengthen the reliability and transparency of AI-assisted development on the platform, which may enhance user trust and adoption among developers. By formalizing and sharing its evaluation methodology, Supabase could improve ecosystem engagement and position itself more competitively in the increasingly AI-driven developer tooling market.
The post also implies that making Supabase Evals open source may encourage community contributions, potentially accelerating improvements to both the scenarios and scoring methods. This community-driven enhancement loop could support product quality, reduce support overhead tied to common issues, and ultimately contribute to a more scalable, efficient growth trajectory for the company.