Automated Software Engineering Myth
Learning Objectives
- You know of some of the myths and realities of automatic programming systems.
The following is a quote from an article “Automatic Programming: Myths and Prospects” published in 1988, written by Charles Rich and Richard C. Waters, highlighting a “cocktail party” description of the potentials of automatic programming:
“There will be no more programming. The end user, who only needs to know about the application domain, will write a brief requirement for what is wanted. The automatic programming system, which only needs to know about programming, will produce an efficient program satisfying the requirement. Automatic programming systems will have three key features: They will be end-user oriented, communicating directly with end users; they will be general purpose, working as well in one domain as in another; and they will be fully automatic, requiring no human assistance.”
In the light of automatic programming system research up to the point of the release of the article, the article highlights myths and realities about automating programming. The following table summarizes the myths and realities:
| Myth | Reality |
|---|---|
| End-user-oriented automatic programming systems do not need domain knowledge. | Automatic programming systems must be domain experts. They also must have enough contextual information to be able to fill in gaps in possibly vague end-user descriptions. |
| End-user-oriented, general purpose, fully automatic programming is possible. | For general-purpose end-user automatic programming to work, automatic programming systems should be experts in every domain, including ones that have not yet been invented. |
| Requirements can be complete. | Requirements engineering is an iterative process that is used to form an understanding of the problem and possible solutions, and in the process and over time, the requirements can and will change. |
| Programming is a serial process. | Programming is an iterative process that involves dialogue between the end-user and the programmer (in part as requirements change over time). |
| There will be no more programming. | There will be more and more programming as end users will also be involved in programming tasks such as writing macros for spreadsheets and so on. |
| There will be no more programming in the large. | Even if there were “automatic programming systems”, not everyone would be satisfied with them. Thus, there would be at the minimum work on new automatic programming systems and tools that support them. |
While large language models have demonstrated that it is possible to produce code, the idea that programming can be fully automated is still a myth. At the core of the problem is that (1) programming is not a purely serial and mechanical process that requires no interventions and (2) that creating meaningful software that solves the problems of customers requires understanding the domain in which the customers work. End users and software engineers learn about the domain and the problem they are solving as they work within the domain.
Creating meaningful software is much more than writing code and there are a range of things that can go wrong in the process. As a good starting point for what can go wrong, see e.g. Perceived Causes of Software Project Failures - An Analysis of their Relationships.
So far, as we have observed in this course, large language models can be used to assist software engineers in their work. They can be used to generate code, provide code completions, review code, document code, test code, and to check for code vulnerabilities. They can also be used to aid in software engineering tasks, such as requirements engineering and so on.
Large language models are also becoming better at real-world software engineering tasks. As an example, as mentioned in the part on Utility and Trustworthiness of Generated Code, state-of-the-art large language models can already be used to automatically solve some GitHub issues.
In the broader context, it is important to remember that software engineering is not a static process. Software changes over time, even if nothing is done to it. As pointed out by Lehman’s Laws of Software Evolution, there is a need for continuing change, as the software must be continuously worked on, or the customers will become less satisfied with it over time.