Requirements Analysis
Learning Objectives
- You know what requirements analysis is and know ways how large language models could be used to help with requirements analysis.
The requirements that have been collected during requirements gathering phase are studied in the requirements analysis phase. The goal of requirements analysis is to more deeply understand and document the requirements, to identify possible problems and inconsistencies in the requirements, and to assert whether the requirements are feasible to implement and how their completion could be measured.
As an example, considering the “Intelligent Code Autocompletion”-feature of an AI programming assistant proposed by the large language model, there were two features that were identified as requirements:
- Predictive typing for method names, variables, and other code components.
- Context-aware suggestions based on the specific code module or library in use.
The above requirements are rather vague. What does “predictive typing” mean? Does it mean that the AI programming assistant should suggest the next word in a sentence, or should it suggest the next line of code, or should it even actively type in the content? What does “context-aware” mean? Does it mean that the AI programming assistant should suggest code components based on the code that has been written so far, or should it suggest code components based on the code that has been written so far and the code that is likely to be written next? Even the term “other code components” is ambiguous.
Functional and non-functional requirements
Requirements can be classified into functional and non-functional requirements. Functional requirements describe the functionality of the system and what can be done with it, while non-functional requirements quality describe qualities of the system, including usability, safety, testability, maintainability, and so on.
Both of the above requirements for the “Intelligent Code Autocompletion”-feature are functional requirements. They describe the functionality of the system and what can be done with it. No restrictions on the usability of the features are set however.
A classic example of usability of machine learning systems is from weather prediction. Imagine a machine learning model that would be able to predict the weather of the next 24 hours with 99% accuracy. This would be a great achievement. However, if the model would take 24 hours to make the prediction, it would not be very useful.
Non-functional requirements are often more difficult to define than functional requirements. They are often not explicitly stated by the stakeholders, but rather implicitly assumed. For example, the stakeholders of the “Intelligent Code Autocompletion”-feature might assume that the feature should be usable in real-time — this would be a non-functional requirement.
Large language models can be used to elicit suggestions for non-functional requirements. We could, for example, ask to provide possible non-functional requirements related to a requirement set or a feature, which we could then use as a starting point for discussions on non-functional requirements from the stakeholders.
Consider the requirements engineering for an intelligent code autocompletion feature of an AI programming assistant with the following features: (1) Predictive typing for method names, variables, and other code components and (2) Context-aware suggestions based on the specific code module or library in use.
What sorts of non-functional requirements could be present for the above features?
Non-functional requirements (NFRs) describe the system attributes, like performance, reliability, usability, etc., rather than specific functionalities. For an AI programming assistant with the features mentioned, potential non-functional requirements could include:
Performance:
- Response Time: The suggestions should appear in real-time, for instance, within a few milliseconds after the user starts typing.
- Throughput: The system should handle a high number of requests per second, especially in a shared or cloud environment.
Scalability:
- The system should scale with the increasing number of users without degrading performance.
- It should be able to handle larger and more complex codebases as well.
Accuracy:
- Predictions should be accurate, i.e., relevant to the context and based on popular coding patterns.
- False positives, where irrelevant suggestions are shown, should be minimal.
Usability:
- Suggestions should be clear and understandable.
- The user interface should not be cluttered with too many suggestions; there should be a reasonable limit.
- The system should allow users to easily enable/disable the feature or adjust its behavior.
...
These are just some potential non-functional requirements. The exact set would be determined by the specifics of the project, the constraints of the development environment, the target user base, and other contextual factors.
Appropriate detail of requirements
A part of requirements analysis is to seek appropriate detail for the requirements. This means that the requirements should be detailed enough to be implemented, but not too detailed. If the requirements are too detailed, they might be too specific and not generalizable. On the other hand, if the requirements are not detailed enough, they might be too vague and not implementable.
In practice, it is often not straightforward to find the appropriate level of detail for requirements. It is often a matter of trial and error, and the level of detail might also depend on the expertise of the team.
There are a few acronyms that can be used to help with this. One of them is SMART, which stands for specific, measurable, achievable, relevant, and time-bound, while another is INVEST, which stands for independent, negotiable, valuable, estimable, small, and testable.
Both of these acronyms can be used to help with requirements analysis. For example, the “Intelligent Code Autocompletion”-feature could be analyzed with the help of the SMART and INVEST acronyms. Let’s first look into whether the requirements are SMART:
- Specific: The requirements are not very specific. They do not specify what “predictive typing” means, what “context-aware” means, and what “other code components” means.
- Measurable: It is not clear how the requirements could be measured.
- Achievable: The requirements are achievable (i.e., it is possible to implement such functionality).
- Relevant: The requirements are relevant and related to the “Intelligent Code Autocompletion”-feature.
- Time-bound: It is not clear when the requirements should be implemented.
We already see problems in the requirements, but for completeness, let’s look into whether the requirements are INVESTable:
- Independent: On the surface-level the requirements are independent and do not depend on other requirements, but in practice they could require other functionality.
- Negotiable: The requirements are negotiable and could be changed.
- Valuable: The requirements are valuable and are related to the “Intelligent Code Autocompletion”-feature.
- Estimable: As there are ambiguities, it is hard to estimate the effort required to implement them.
- Small: The requirements are not small and could/should be likely divided into smaller parts.
- Testable: It is unclear how the requirements should be tested.
In practice, however, documenting and analyzing requirements is often not straightforward. There are often ambiguities and inconsistencies in the requirements, dependencies between requirements, requirements that are not feasible to implement, requirements that are not testable, and so on. Identifying these is a key part of learning about the domain.
Describing expected system functionality
There are a variety of ways to describe expected system functionality. Two of the more common ways are use cases and user stories, which are often used in agile software development. Both of them have their uses, also depending on the expertise of the team.
Use cases
Use cases are a way to describe and document the functionality of a system from the perspective of the user. They are typically written in a structured way, with a description of the use case, the actors involved, the preconditions and postconditions, the main scenario, alternative scenarios, and exceptions.
As an example, the following outlines a use case describing a possible autocompletion functionality for the “Intelligent Code Autocompletion”-feature:
Use case: Autocomplete code |
---|
Actor: User |
Description: The user wants to autocomplete code. |
Preconditions: The user has started typing code. |
Postconditions: The user has completed typing code. |
Main scenario: |
1. The user starts typing code. |
2. The system suggests code components. |
3. The user selects a code component. |
4. The system inserts the code component into the code. |
Alternative scenarios: |
1. The user starts typing code. |
2. The system suggests code components. |
3. The user does not select a code component. |
4. The system does not insert a code component into the code. |
Exceptions: |
1. The user starts typing code. |
2. The system does not suggest code components. |
3. The user does not select a code component. |
4. The system does not insert a code component into the code. |
Notes: The user can start typing code at any time. The user can select a code component at any time. The system can suggest code components at any time. The system can insert a code component into the code at any time. |
User stories
User stories are typically written in a more informal way, including also information that outlines what the value or objective of particular functionality is. They are typically written in the form of “As a role, I want goal so that benefit.” Here, the role is the user or stakeholder of the system, the goal is the functionality that is desired, and the benefit is the value or objective of the functionality.
As an example, the above use case could be written as a user story as follows: “As a developer, I want to autocomplete code so that I can write code faster.” As you notice, the user story is much shorter and more informal than the use case. It is also more focused on the value of the functionality, rather than the functionality itself.
Both use cases and user stories can also be brainstormed and created with the help of large language models.
Please provide a structured use case for a requirement "Context-aware suggestions based on the specific code module or library in use." of an intelligent AI programming assistant. Use the terms use case, actor, description, preconditions, postconditions, and main scenario.
Use Case: Context-aware Code Suggestions
Actor: Developer
Description:
The intelligent AI programming assistant provides context-aware suggestions to the Developer based on the specific code module or library currently in use. This aims to enhance the productivity and efficiency of the Developer by offering relevant code snippets, functions, and documentation tailored to the code context.
Preconditions:
- The Developer has the AI programming assistant tool installed and activated in their development environment.
- The Developer is writing or editing code in a recognized programming language.
- The programming environment has access to the library or module's documentation or a knowledge base where the AI assistant can pull suggestions from.
Postconditions:
- The Developer receives relevant and context-aware suggestions pertaining to the library or module they are working with.
- Developer's code remains unchanged unless they accept and implement a suggestion.
Main Scenario:
- The Developer starts writing code using a specific module or library.
- The AI programming assistant identifies the module or library in use by analyzing the import statements or equivalent in the code.
- Based on the current context (i.e., the functions or methods being used or the code patterns identified), the AI assistant searches its knowledge base or the documentation of the library/module.
- The AI assistant generates a list of context-aware suggestions. These can include:
- Relevant functions or methods from the module or library.
- Common coding patterns or practices associated with the current context.
- Direct links to relevant parts of the module or library's documentation.
- Potential issues or warnings related to the current usage of the library/module.
- These suggestions are presented to the Developer in a non-intrusive manner, such as a sidebar, tooltip, or pop-up.
...
In the above, we prompt a large lagnuage model for a use case for the context-aware suggestions of code. The key parts of a use case are present, and we could continue prompting for further use cases. Below, we prompt for a user story for the same requirement.
Please provide a structured user story for a requirement "Context-aware suggestions based on the specific code module or library in use." of an intelligent AI programming assistant. Use the template "as a <user>, I want <goal> so that <benefit>".
Sure, here's a structured user story for the requirement:
Title: Context-Aware Code Suggestions
User Story:
As a software developer, I want the AI programming assistant to provide context-aware suggestions based on the specific code module or library I'm using, So that I can write code more efficiently, ensure it's aligned with best practices, and reduce potential bugs or integration issues.
Acceptance Criteria:
- When a developer starts typing a function or method associated with a recognized library or module, the AI assistant should identify the context.
- The suggestions provided should be relevant to the module or library in use, rather than general coding suggestions.
- The AI assistant should prioritize suggestions based on the popularity or best practices associated with the library or module.
...
Interestingly, above, we also observe that the large language model provides an acceptance criteria for the user story. This is a good example of how large language models can be used to help with requirements analysis.
Try asking the LLM to generate use cases for the requirements of an application of your choice. Start with simply asking for use cases, without mentioning e.g. requirements, and explore refining the prompt and seeing how the generated use cases and their format changes.
Once done, try creating use cases for the requirements of an application that does not make any sense. For example, a broccoli or salad peeling machine. What do you notice?
As outlined by Martin Fowler, both use cases and user stories can be useful, but it might not be that both are not needed:
”… Some teams might use use cases early on to build a narrative picture, and then break down into stories for planning. Others go direct to stories. Others might just do use cases and annotate the use case text to show what features get done when.” — Martin Fowler: What is the difference between a UseCase and XP’s UserStory?
Priorities and relative importance
A part of requirements analysis is also to prioritize the requirements and to determine their relative importance. This is important as it helps to determine which requirements should be implemented first and which requirements are more important than others. One way of prioritizing requirements is to use the MoSCoW method, which is a prioritization technique that is often used in agile software development, which stands for must have, should have, could have, and won’t have — essentially, requirements are classified into the four categories based on their importance.
When identifying requirements, it is often meaningful to form an in-depth understanding of the most important requirements first, and then move on to the less important requirements. The same holds also for implementation — it is meaningful to first implement the most important parts of the system, and then move on to the less important parts.
Large language models can also to some extent be used to prioritize requirements. As an example, see the article Prioritizing Software Requirements Using Large Language Models.