Enhancing LLM-Based API Test Generation via Constraint-Aware Prompting: An Empirical Study

Abstract

Automated API testing is crucial for ensuring the re- liability of modern software architectures. While Large Language Models (LLMs) demonstrate significant potential in generating API tests from OpenAPI specifications, existing baseline methods often yield extremely low execution success rates when applied to complex, industrial-grade ecosystems like Java Spring Boot. In this paper, we conduct an empirical study reproducing the RestTSLLM methodology to identify the root causes of these failures. Our findings indicate that these failures primarily stem from the LLM’s lack of environmental context-awareness and its inability to adhere to strict HTTP protocol semantics. To address these limitations, we propose the Constraint-Enhanced Prompting Framework. This approach shifts human intervention from downstream code post-processing to upstream prompt design by introducing three core constraint pillars: Dynamic Context, Protocol Semantics, and Regex Strictness. Evaluated on the widely adopted Spring PetClinic project using the DeepSeek- V3 model, our enhanced framework successfully eliminated generation hallucinations, achieving a 100% execution pass rate in the evaluated test suites. Furthermore, the highly constrained LLM effectively served as a strict API contract auditor, detecting latent real-world vulnerabilities in the System Under Test (SUT), including Specification-to-Implementation drift and unhandled server exceptions.