Software development processes

Table of Contents

Software processes

    This case study is intended to serve as a study material for the course Technology of System Development for Software IT BSc students at the Faculty of Informatics, Unversity of Debrecen. The primary goal of the case study is to show the major steps of analysis and design activities of the software engineering lifecycle. The reader is expected to know the basics of the object-oriented (OO) paradigm and have some experience in programming in at least one OO language. However, the case study is intended to be independent of any concrete programming languages since it is out of the scope of this study material to deal with implementation details.

    Knowledge about some guidelines, principles and patterns of software design might also be required. Students learn those pieces of background information in the prerequisite course.

    Our running example will be a Book and DVD store. This example was chosen in order to be general enough for showing the most challenges that the specification and design of information systems have. The main purpose of the system is to allow the customers to browse and search products in an online warehouse, and they should also be able to place orders for those products. Detailed requirements will be discussed later, in the chapter entitled Requirements engineering.

    The artifacts that are produced throughout the stages of development are documentations that are illustrated by models. Due to the rigid nature of some development documentations (i.e., requirements specification document or software design document) we will not include those documentations in their full details; instead, our focus is on the process of establishing the models describing the system of the case study. The models will predominantly be described using the Unified Modeling Language (UML) standard which is not only a de facto standard for describing models of software systems but has also become a de iure standard under reference numbers ISO/IEC 19505-1 and 19505-2. The current version of UML by the time of writing is 2.4.1 therefore we will use this version, however, version 2.5—which will have a major change in the organization of the specification—is under development.

    This document is organized as follows. The rest of this chapter provides a very short overview of software process models and an outline of the UML 2.4.1 standard. Chapter 2 discusses the major issues of requirements engineering and provides a detailed view of the requirements of our case study. Chapter 3 introduces few high-level system modeling steps that will guide the reader to the architectural and detailed design covered in Chapter 4 and 5. The built-in extensibility mechanism of UML (called profiles) are detailed in Chapter 6 along with some examples of tailoring the design of the system to specific platforms. The last chapter is concerned with the emerging field of service-oriented architectures and discusses issues on how to expose system functionalities as services that can later be orchastrated into more complex services thus providing means of cross-organizational integration of software services.

    Software processes

    A software process can be defined as a set of related activities that will lead to the creation a software product. Some systems are developed from scratch while other systems are developed based on existing systems by extending them. Another group of systems are created when existing components are integrated and configured.

    The four fundamental activities of software engineering are:

    1. Software specification. Software's functionality, required product attributes and operational constraints are defined.

    2. Software design and implementation. The software is designed and programmed in order to meet the specification.

    3. Software validation. Software must be validated to ensure that it delivers the required functionalities, meets the required product attributes and fulfills the constraints that users (customers) want so briefly it is checking whether the system fulfills its goals.

    4. Software evolution. The development of software is not finished with deployment but the system should evolve to meet the changing needs.

    These are those processes that you will inevitably meet regardless of which software process is applied in your project. Of course, these processes are complex processes: they include sub-processes or sub-activities like requirements management, architectural design, interface design, unit testing, acceptance testing, etc. Some supporting activities like configuration management or documantation is also needed when creating software.

    When describing software processes, the activities involved in those processes should be discussed like designing the user interface or specifying a data structure. Ordering of activities are also important from process view so we can refine our previous definition on software processes that will include not only the set of activities that should be executed during development but the ordering of those activities are important, as well. Software process descriptions, however, are not limited to describe activities and ordering of them but they might also include information on

    1. products or artifacts that are the results of a process activity,

    2. roles that reflect the responsibilities of the people involved in the process, and

    3. pre- and postconditions which define logical statements that must be true before or after a process activity has been performed or an artifact has been created.

    A Software Requirements Specification document is an example of an artifact that is the result of the specification activity; programmer or project manager are examples of roles; while a precondition of integration testing might be that all the components that should be integrated are accessible and they are unit tested individually.

    Important

    Software processes are complex and, like all intellectual and creative processes, rely on people making decisions and judgments. There is no silver bullet that means that there is not an ideal process of developing software. Processes vary based on their domain, application type, devloping organization, etc. Majority of organizations have developed their own software development processes that are standardized at an organizational level. Processes have evolved to get the best out of the people's capabilities within an organization and the specific characteristics of the systems to be developed.

    Sommerville describes three categories of software process models (that can be, however, often used together). In fact, these are generic models that are not definitive descriptions of processes but are seen as abstractions. So you can think of them like software process frameworks that can and should be extended and/or adapted in order to create specific software engineering processes. These generic processes are:

    1. Waterfall model which takes the four fundamental process activities described above and represents them as separate stages like requirements specification, software design, implementation, testing, and so on.

    2. Incremental development which interleaves specification, development, and validation activities. The system is developed as a series of versions (increments), with each version adding functionality to the previous one.

    3. Reuse-oriented development which is, instead of creating systems from scratch, focuses on the integration of existing components into a system so this is based on the existence of a significant number of reusable components.

    Software processes can be categorized as either plan-driven or agile processes. Plan-driven processes are processes where all of the process activities are planned in advance and progress is measured against this plan while in agile processes, planning is incremental and it is easier to change the process to reflect changing customer requirements.

    Waterfall model

    The waterfall model is an example of a plan-driven process—in principle, you must plan and schedule all of the process activities before starting work on them. As it is seen on the figure, phases are cascaded from one phase to another, that is while it is known as the ‘waterfall model’.

    Figure 1.1. Waterfall model

    The principal stages of the waterfall model directly reflect the fundamental development activities:

    1. Requirements analysis and definition. The system’s services, constraints, and goals are established by consultation with system users. They are then defined in detail and serve as a system specification.

    2. System and software design. The systems design process allocates the requirements to either hardware or software systems by establishing an overall system architecture. Software design involves identifying and describing the fundamental software system abstractions and their relationships.

    3. Implementation and unit testing. During this stage, the software design is realized as a set of programs or program units. Unit testing involves verifying that each unit meets its specification.

    4. Integration and system testing. The individual program units or programs are integrated and tested as a complete system to ensure that the software requirements have been met. After testing, the software system is delivered to the customer.

    5. Operation and maintenance. Normally (although not necessarily), this is the longest life cycle phase. The system is installed and put into practical use. Maintenance involves correcting errors which were not discovered in earlier stages of the life cycle, improving the implementation of system units and enhancing the system’s services as new requirements are discovered.

    At least in principle, the result (artifact) of each phase is one or more documents that should be approved and the subsequent phase should not be started until the previous phase has completely been finished. In practice, however, these stages overlap and feed information to each other. During design, problems with requirements can be identified, during coding some of the design problems can be found, etc. The software process therefore is not a simple linear model but involves feedback from one phase to another as it is depicted on the figure. Documents produced in each phase may then have to be modified to reflect the changes made.

    In principle, the waterfall model should only be applied when requirements are well understood and unlikely to change radically during development as this model has a relatively rigid structure which makes it relatively hard to accomodate change when the process in underway.

    An important variant of the waterfall model is formal system development, where the specification is described using a mathematical model which will later be refined into executable code (possibly through a series of other models) using consistency-preserving mathematical transformations. If the mathematical transformations applied are correct, we have a strong argument that a program generated in this way is consistent with its specification.

    Incremental development

    Incremental development is based on the idea of developing an initial implementation, exposing this to user feedback and evolving it through several versions until an acceptable system has been developed. Activities of specification, development and validation are not separated but interleaved. There are also lots of feedback across those activities.

    Figure 1.2. Icremental development

    Incremental software development, which is a fundamental part of agile approaches, is better than a waterfall approach for most business, e-commerce, and personal systems. Incremental development reflects the way that we solve problems. We rarely work out a complete problem solution in advance but move toward a solution in a series of steps, backtracking when we realize that we have made a mistake. By developing the software incrementally, it is cheaper and easier to make changes in the software as it is being developed.

    Each system increment captures a piece of the functionality that is needed by the customer. Generally, the early increments of the system should include the most important or most urgently required functionality. If this is the case then customers can evaluate the system at a relatively early stage in the development to see if it delivers what is required. If not, then only the current increment has to be changed and, possibly, new functionality defined for later increments. Compared to the waterfall model, incremental development has three important benefits:

    1. There is a reduced cost of dealing with changing requirements since the amount of analysis and documentation activities that has to be redone is much less than is required when using the waterfall model.

    2. It is much easier to get customer feedback on the work done. In waterfall-like processes (at least, theoretically) customers first meet the system when it is fully developed and tested and is deployed. If something is not acceptable than it is a real problem (that is the reason why waterfall should only be applied when requirements are well-understood). In iterative development, customers regularly receive a (not fully functional) version of the system that can be commented which provides valueable feedback.

    3. More rapid delivery and deployment is possible so customers can get value out of the software earlier than is possible with the waterfall process. In incremental development it is allowed (and advised) to deliver useful functionalities even in early stages of the development.

    Incremental development in some form is now the most common approach for the development of application systems. This approach can be either plan-driven, agile, or, more usually, a mixture of these approaches. In a plan-driven approach, the system increments are identified in advance; if an agile approach is adopted, only the early increments are identified but the development of later increments depends on progress and customer priorities.

    Reuse-oriented software engineering

    Reuse is one of the most important concepts of today's software engineering since it can not only save a given amount of work when existing components providing a given funtionality are reused but existing components might have lots of testing received so far so we can possibly build more reliable systems based on them. Nowadays more and more software projects apply reuse to some extent, however, some of them relies more on reused components than others.

    Reuse often happens in an informal way when people working on the project know of designs or code that are similar to what is required. They look for these, modify them as needed, and incorporate them into their system. This is basically the application of patterns in the development process.

    Figure 1.3. Reuse-oriented development


    A general process model for reuse-oriented software engineering is shown in the above figure. Although the initial requirements specification stage and the validation stage are comparable with other software processes, the intermediate stages in a reuse-oriented process are different. These stages are:

    1. Component analysis. Based on the requirements specification, components that implement (some part of) the specification are looked for. In the most of the cases there is no exact match and the components that may be used only provide some of the functionality required.

    2. Requirements modification. During this stage, the requirements are analyzed using information about the components that have been discovered. They are then modified to reflect the available components. Where modifications are impossible, the component analysis activity may be re-entered to search for alternative solutions.

    3. System design with reuse. During this phase, the framework of the system is designed or an existing framework is reused. The architects will perform the design by taking into account the components that are reused and they will organize the framework accordingly. New pieces of software may have to be designed if reusable components are not available.

    4. Development and integration. Software that cannot be externally procured is developed, and the components and commercial-off-the-shelf (COTS) systems are integrated to create the new system. System integration, in this model, may be part of the development process rather than a separate activity.

    There are basically three types of software components that can be used in a reuse-oriented process:

    1. Web services that are developed according to well-known service standards and which will become available for remote invocation.

    2. Collections of objects that are developed as a package to be integrated with a component framework such like .NET or Java EE.

    3. Standalone software systems that are configured for use in a particular environment.

    One of the most important advabtage of reuse-oriented software engineering is the reduced amount of software to be developed and therefore reduced cost and risks. As a consequence, it can also lead to faster delivery. However, compromises must be achieved on requirements which might lead to a system that does not meet the real needs of users. Some control over the system evolution might also be lost as new versions of the reusable components are not under the control of the organization using them (no influence on what functionalities to include/exclude, etc.).

    Process activities

    In this section some of the most important activities and sub-activities of the four general process activities will be discussed very briefly. For a more detailed description of activities, the reader should read some general software engineering textbooks like.

    The four basic process activities of specification, development, validation, and evolution are organized differently in different development processes. In the waterfall model, they are organized in sequence while in incremental development they are interleaved. How these activities are performed might depend on the type of software, people involved in development, etc.

    Specification

    Software specification or requirements engineering is the process of understanding and defining what services are required from the system and identifying the constraints on the system’s operation and development.

    Figure 1.4. Requirements engineering process


    There are four main sub-activities in requirements engineering processes:

    1. Feasibility study when an estimation of whether the identified requirements may be satisfied using current software and hardware technologies is made. A feasibility study should be relatively cheap and quick; it should inform the decision of whether or not to go ahead with a more detailed analysis.

    2. Requirements elicitation and analysis, which is the process of deriving the system requirements through observation of existing systems, discussions with stakeholders, etc. This may involve the development of one or more system models and prototypes that can help us understanding the system to be specified.

    3. Requirements specification, which is the activity of translating the information gathered during the analysis activity into a (formal or informal, depending on the underlying process used) document that defines a set of requirements. Two types of requirements may be included in this document:

      1. User requirements are abstract statements of the system requirements for the customer and end-user of the system;

      2. System requirements are a more detailed description of the functionality to be provided.

    4. Requirements validation is the process of checking the requirements for realism, consistency and completeness. During this process our goal is to reveal errors in the requirements document. When an error is found, the requirements specification document needs to be modified to correct the problems.

    Of course, the activities in the requirements process are not necessarily executed in a strict sequence but the activities of analysis, definition, and specification can be interleaved. In agile methods, such as extreme programming, requirements are developed incrementally according to user priorities and the elicitation of requirements comes from users who are part of the development team.

    Design and implementation

    The implementation phase is the process of converting a system specification into an executable system. It always involves processes of software design and programming but, if an incremental approach to development is used, may also involve refinement of the software specification.

    Figure 1.5. Design process

    The activities needs to be carried out during the design process vary depending on the type of the system needs to be developed. However, for the development of information systems, the main four activities are the following:

    1. Architectural design, where the overall system structure is identified. It involves finding principal components (sometimes called sub-systems or modules) along with their relationships. Distribuution of components (to hardware nodes and to development teams) should also be discussed.

    2. Interface design, where the interfaces among various system components are defined. This interface specification must be unambiguous. With a precise interface, a component can be used without other components having to know how it is implemented. Once interface specifications are agreed, the components can be designed and developed concurrently.

    3. Component design, where we take each system component and design how it will operate. This may be a simple statement of the expected functionality to be implemented, with the specific design left to the programmer.

    4. Database design, where the data structures used by the system are designed and their representation in a database is dealt with. The sub-activities that need to be performed here depends on whether an existing database is to be reused or a new database is to be created.

    Validation

    Software validation or, more generally, verification and validation (V&V) is intended to show that a system both conforms to its specification and that it meets the real needs of the customer or user of the application. Testing, where the system is executed using simulated test data, is an important validation technique but some checking processes, such like inspections or reviews should also be applied during V&V processes.

    Testing has three main stages:

    1. Development testing. The components that compose the system are tested by the people developing the system. Each component is tested independently, without other system components. Components may be simple entities such as functions or object classes, or may be coherent groupings of these entities. Test automation tools, such as JUnit are commonly used.

    2. System testing. System components are integrated to create a complete system. This process is concerned with finding errors that result from unanticipated interactions between components and component interface problems. It is also concerned with showing that the system meets its functional and non-functional requirements, and testing the emergent system properties.

    3. Acceptance testing. This is the final stage in the testing process before the system is accepted for operational use. The system is tested with data supplied by the system customer rather than with simulated test data. Acceptance testing may reveal errors and omissions in the system requirements definition, because the real data exercise the system in different ways from the test data.

    Evolution

    Requirements are always changing, even after the system has been put into its operating environment. Emerging systems with new interfaces that our software needs to communicate with, or business environment changes might result in changing needs.

    Historically, there has always been a split between the process of software development and the process of software evolution (software maintenance). However, this distinction between development and maintenance is increasingly irrelevant. Hardly any software systems are completely new systems and it makes much more sense to see development and maintenance as a continuum. Rather than two separate processes, it is more realistic to think of software engineering as an evolutionary process where software is continually changed over its lifetime in response to changing requirements and customer needs.

    Category