Chapter 12.  Software quality management

Table of Contents
12.1. Process and product quality
12.2. Quality assurance and standards
12.2.1. ISO
12.2.2. Documentation standards
12.3. Quality planning
12.4. Quality control
12.4.1. Quality reviews
12.5. Software measurement and metrics
12.5.1. The measurement process
12.5.2. Product metrics
12.6. Exercises

The quality of software has improved significantly over the past two decades. One reason for this is that companies have used new technologies in their software development process such as object-oriented development, CASE tools, etc. In addition, a growing importance of software quality management and the adoption of quality management techniques from manufacturing can be observed. However, software quality significantly differs from the concept of quality generally used in manufacturing mainly for the next reasons [ 1 ]:

  1. The software specification should reflect the characteristics of the product that the customer wants. However, the development organization may also have requirements such as maintainability that are not included in the specification.

  2. Certain software quality attributes such as maintainability, usability, reliability cannot be exactly specified and measured.

  3. At the early stages of software process it is very difficult to define a complete software specification. Therefore, although software may conform to its specification, users don’t meet their quality expectations.

Software quality management is split into three main activities:

  1. Quality assurance. The development of a framework of organizational procedures and standards that lead to high quality software.

  2. Quality planning. The selection of appropriate procedures and standards from this framework and adapt for a specific software project.

  3. Quality control. Definition of processes ensuring that software development follows the quality procedures and standards.

Quality management provides an independent check on the software and software development process. It ensures that project deliverables are consistent with organizational standards and goals.

12.1.  Process and product quality

It is general, that the quality of the development process directly affects the quality of delivered products. The quality of the product can be measured and the process is improved until the proper quality level is achieved. Figure 12.1. illustrates the process of quality assessment based on this approach.

Process based quality assessment.
Figure 12.1. Process based quality assessment.


In manufacturing systems there is a clear relationship between production process and product quality. However, quality of software is highly influenced by the experience of software engineers. In addition, it is difficult to measure software quality attributes, such as maintainability, reliability, usability, etc., and to tell how process characteristics influence these attributes. However, experience has shown that process quality has a significant influence on the quality of the software.

Process quality management includes the following activities:

  1. Defining process standards.

  2. Monitoring the development process.

  3. Reporting the software.

12.2.  Quality assurance and standards

Quality assurance is the process of defining how software quality can be achieved and how the development organization knows that the software has the required level of quality. The main activity of the quality assurance process is the selection and definition of standards that are applied to the software development process or software product. There are two main types of standards. The product standards are applied to the software product, i.e. output of the software process. The process standards define the processes that should be followed during software development. The software standards are based on best practices and they provide a framework for implementing the quality assurance process.

The development of software engineering project standards is a difficult and time consuming process. National and international bodies such as ANSI and the IEEE develop standards that can be applied to software development projects. Organizational standards, developed by quality assurance teams, should be based on these national and international standards. Table 12.1. shows examples of product and process standards.

Table 12.1. Examples of product and process standards.

Product standards

Process standards

Requirements document structure

Project plan approval process

Method header format

Version release process

Java programming style

Change control process

Change request form

Test recording process

12.2.1.  ISO

ISO 9000 is an international set of standards that can be used in the development of a quality management system in all industries. ISO 9000 standards can be applied to a range of organizations from manufacturing to service industries. ISO 9001 is the most general of these standards. It can be applied to organizations that design, develop and maintain products and develop their own quality processes. A supporting document (ISO 9000-3) interprets ISO 9001 for software development.

The ISO 9001 standard isn’t specific to software development but includes general principles that can be applied to software development projects. The ISO 9001 standard describes various aspects of the quality process and defines the organizational standards and procedures that a company should define and follow during product development. These standards and procedures are documented in an organizational quality manual.

The ISO 9001 standard does not define the quality processes that should be used in the development process. Organizations can develop own quality processes and they can still be ISO 9000 compliant companies. The ISO 9000 standard only requires the definition of processes to be used in a company and it is not concerned with ensuring that these processes provide best practices and high quality of products. Therefore, the ISO 9000 certification doesn’t means exactly that the quality of the software produced by an ISO 9000 certified companies will be better than that software from an uncertified company.

12.2.2.  Documentation standards

Documentation standards in a software project are important because documents can represent the software and the software process. Standardized documents have a consistent appearance, structure and quality, and should therefore be easier to read and understand. There are three types of documentation standards:

  1. Documentation process standards. These standards define the process that should be followed for document production.

  2. Document standards. These standards describe the structure and presentation of documents.

  3. Documents interchange standards. These standards ensure that all electronic copies of documents are compatible.

12.3.  Quality planning

Quality planning is the process of developing a quality plan for a project. The quality plan defines the quality requirements of software and describes how these are to be assessed. The quality plan selects those organizational standards that are appropriate to a particular product and development process. Quality plan has the following parts:

  1. Introduction of product.

  2. Product plans.

  3. Process descriptions.

  4. Quality goals.

  5. Risks and risk management.

The quality plan defines the most important quality attributes for the software and includes a definition of the quality assessment process. Table 12.2. shows generally used software quality attributes that can be considered during the quality planning process.

Table 12.2. Software quality attributes.

Safety

Understandability

Portability

Security

Testability

Usability

Reliability

Adaptability

Reusability

Resilience

Modularity

Efficiency

Robustness

Complexity

Learnability

Maintainability

   

12.4.  Quality control

Quality control provides monitoring the software development process to ensure that quality assurance procedures and standards are being followed. The deliverables from the software development process are checked against the defined project standards in the quality control process. The quality of software project deliverables can be checked by regular quality reviews and/or automated software assessment. Quality reviews are performed by a group of people. They review the software and software process in order to check that the project standards have been followed and that software and documents conform to these standards. Automated software assessment processes the software by a program that compares it to the standards applied to the development project.

12.4.1.  Quality reviews

Quality reviews are the most widely used method of validating the quality of a process or product. They involve a group of people examining part or all of a software process, system, or its associated documentation to discover potential problems. The conclusions of the review are formally recorded and passed to the author for correcting the discovered problems. Table 12.3. describes several types of review, including quality reviews.

Table 12.3. Types of review.

Review type

Principal purpose

Design or program inspections

To detect detailed errors in the requirements, design or code.

Progress reviews

To provide information for management about the overall progress of the project.

Quality reviews

To carry out a technical analysis of product components or documentation to find mismatches between the specification and the component design, code or documentation and to ensure that defined quality standards of the organization have been followed.

12.5.  Software measurement and metrics

Software measurement provides a numeric value for some quality attribute of a software product or a software process. Comparison of these numerical values to each other or to standards draws conclusions about the quality of software or software processes. Software product measurements can be used to make general predictions about a software system and identify anomalous software components.

Software metric is a measurement that relates to any quality attributes of the software system or process. It is often impossible to measure the external software quality attributes, such as maintainability, understandability, etc., directly. In such cases, the external attribute is related to some internal attribute assuming a relationship between them and the internal attribute is measured to predict the external software characteristic. Three conditions must be hold in this case:

  1. The internal attribute must be measured accurately.

  2. A relationship must exist between what we can measure and the external behavioural attribute.

  3. This relationship has to be well understood, has been validated and can be expressed in terms of a mathematical formula.

12.5.1.  The measurement process

A software measurement process as a part of the quality control process is shown in Figure 12.2. The steps of measurement process are the followings:

  1. Select measurements to be made. Selection of measurements that are relevant to answer the questions to quality assessment.

  2. Select components to be assessed. Selection of software components to be measured.

  3. Measure component characteristics. The selected components are measured and the associated software metric values computed.

  4. Identify anomalous measurements. If any metric exhibit high or low values it means that component has problems.

  5. Analyze anomalous components. If anomalous values for particular metrics have been identified these components have to be examined to decide whether the anomalous metric values mean that the quality of the component is compromised.

Generally each of the components of the system is analyzed separately. Anomalous measurements identify components that may have quality problems.

The software measurement process.
Figure 12.2. The software measurement process.


12.5.2.  Product metrics

The software characteristics that can be easily measured such as size do not have a clear and consistent relationship with quality attributes such as understandability and maintainability. Product metrics has two classes:

  1. Dynamic metrics. These metrics (for example execution time) are measured during the execution of a program.

  2. Static metrics. Static metrics are based on measurements made of representations of the system such as the design, program or documentation.

Dynamic metrics can be related to the efficiency and the reliability of a program. Static metrics such as code size are related to software quality attributes such as complexity, understandability, maintainability, etc.

12.6.  Exercises

  1. Give examples of software quality attributes!

  2. What is the objective of product standards?

  3. What is the objective of process standards?

  4. Give examples of product and process standards!

  5. What international standards can be used in software quality assessment?

  6. What are the types of document standard?

  7. What are the main parts of a quality plan?

  8. What are the steps of software measurement process?

  9. Give examples of static software metric!