Are We Just Applying LLMs? What SE Research Studies, Underexplores, and Could Do Next in the LLM Era
Story 18, from arXiv
A study of 4,790 SE research papers and 244 researcher responses shows that AI is reshaping software engineering research, but the field is reorganizing around AI without critically questioning its impact.
Key points
- AI now appears in over half of SE research papers at major conferences like ICSE, FSE, and ASE.
- AI accounts for 93.1% of publication growth in SE research since 2018, but some areas are contracting while others expand.
- Most AI papers apply or evaluate LLMs, rather than interrogate the paradigm or develop alternatives.
- The FoSE survey shows researchers are more concerned about AI-driven change than they are willing to address it.
- The LLM wave has shifted SE tools more than problem boundaries, with new problems arising from integrating LLMs into the SE process.
Why it matters
As a software engineer using AI tools daily, understanding how AI is reshaping research and practice can help you anticipate shifts in tooling, problem-solving approaches, and the evolving landscape of software engineering.
Try this
Review recent SE research papers to see how AI is being applied in your domain, and consider whether you're solving new problems or reusing existing ones with LLMs.
Counterpoint
The study highlights a lack of critical engagement with AI's role in SE, suggesting that the field may be adopting AI without fully understanding its implications or limitations.
Sources
- Are We Just Applying LLMs? What SE Research Studies, Underexplores, and Could Do Next in the LLM Era (arXiv)
This summary was written by a language model and may contain errors. Read the original for the full story.