Can LLM Agents Learn to Test? Maintaining Scoped Experience for Mobile GUI Testing
Story 12, from arXiv
This paper explores how LLM-based testers can retain useful testing experience while avoiding outdated or irrelevant information, using a maintenance contract for reusable test routes and locators in mobile GUI testing.
Key points
- LLM-based testers can retain useful experience by assigning explicit applicability scopes to test routes and locators.
- A maintenance contract for AdaptiTest includes mechanisms for confirmation, revalidation, and withdrawal of test experience.
- The design changes focus on retrieving context, not model weights, and the evaluation uses read-only Android workflows.
- A six-task subset shows 18/18 completed runs with memory versus 3/18 without, but a single-agent control also solves the task.
- The study examines real-update hazard records, distinguishing stale-locator observations from policy-based withholding.
- The evidence supports a bounded design and case study, but does not establish general efficacy or superiority over other systems.
Why it matters
Software engineers who use AI for testing may find value in understanding how LLMs can retain and reuse testing experience, especially in dynamic environments like mobile GUI testing where interfaces frequently change.
Counterpoint
The study does not establish general efficacy or superiority over independent GUI-agent systems, and the evidence is limited to a bounded design and case study.
Sources
This summary was written by a language model and may contain errors. Read the original for the full story.