Building a Performance Testing Framework with AI Agents: Lessons Learned
Joni Saarinen
About this talk
This talk shares lessons from developing a Locust-based performance testing framework with AI at SOK. The project would not realistically have been possible at this scope and speed without AI. The focus is not the tool, but how the development actually worked.
It examines two connected workflows: developing the framework itself and developing the test scripts that use it, both using an AI-agent-driven approach. The talk explores how people and AI agents shared the work, with engineers guiding the goals, decisions, and desired behavior.
The work showed that AI-generated code can appear convincing while hiding faulty assumptions and important gaps. Making expectations explicit, testing behavior, and reviewing the results became essential parts of the development process.
The goal is to share honest lessons for anyone building or maintaining a codebase with an AI agent: what slowed the work, what would be done differently, and what proved surprising.
Speaker
Joni has over 15 years of experience in software testing, covering roles from manual testing to test management. In recent years, his work has focused on test automation, DevOps, and AI-assisted software development. More recently, he has applied AI agents to software development, test automation, and performance testing.
Joni has worked in consulting and product development environments, with domain experience in retail, telecom, insurance, banking, and industrial sectors. He takes a practical approach to testing and values making quality, behavior, and results visible. He is particularly interested in automation in all its forms, including helping people do better work through intelligent and automated assistance.