TipRanks
Advertisement

Turing Highlights New Benchmark for LLM-Based Procedural Database Programming

Turing Highlights New Benchmark for LLM-Based Procedural Database Programming

According to a recent LinkedIn post from Turing, the company is highlighting new research conducted with Oracle on PLSQLBench, a benchmark designed to evaluate whether large language models can write executable PL/SQL programs. The post notes that existing benchmarks focus on general code generation or text-to-SQL, while real enterprise database work involves stored procedures, cursors, exception handling, and interaction with live schemas.

The company’s LinkedIn post explains that PLSQLBench is positioned as the first benchmark tailored specifically to procedural database programming, comprising 2,865 instances, 2,594 single-turn tasks, and 271 multi-turn conversations totaling 978 turns. The benchmark is built on enterprise-style Spider 2 databases, schema-grounded Spider tasks, and MBPP-derived procedural problems to better reflect tasks developers perform in production environments.

According to the post, PLSQLBench measures capabilities such as writing procedures, handling exceptions, grounding work in real schemas, and iterative refinement of code, which are critical for deploying reliable database applications. Tests of eight LLMs reportedly revealed recurring weaknesses in schema grounding, fidelity to the PL/SQL dialect, procedural control flow, exception handling, and consistency across multi-turn interactions.

The LinkedIn post suggests that tool-augmented agents were able to close part of the performance gap on schema-grounded tasks, although significant limitations remain in generating robust procedural database code. The work has been accepted to the EMNLP industry track 2026 in Budapest, indicating peer-reviewed recognition that may enhance Turing’s credibility in enterprise AI tooling and specialized benchmarks for software development.

For investors, this research effort could signal Turing’s strategic focus on high-value, enterprise-grade AI applications rather than only generic code generation. If PLSQLBench becomes a reference standard for evaluating LLMs in procedural database programming, Turing may strengthen its position in the AI developer tools ecosystem and deepen relationships with large database and cloud vendors such as Oracle.

The post also points to ongoing performance gaps in current LLMs, which may create demand for improved models, evaluation frameworks, and supporting tools that Turing could help provide. Over time, successful commercialization of such benchmarks and related products could support pricing power, integration opportunities with enterprise clients, and differentiation in a crowded AI infrastructure and services market.

Disclaimer & DisclosureReport an Issue

1