 | The Software Engineering Process Group
In approaching the software process it is instructive to examine the way processes are handled in other fields. Consider, for example, the difference between a factory and a machine shop. They both may have similar equipment, but they clearly produce vastly different results. In the machine shop the process depends on who is handling the work, while the process in the factory has been designed. In fact, manufacturing processes are typically designed by process specialists before the tools are even ordered and installed.
While there are vast differences between software engineering & the typical factory, software process development also requires some specialists to do the work. Actually the problems of software process change are often complicated by the fact that no one is responsible to make it happen. If software process improvement isn't anybody's job, it is not surprising that it doesn't get done! If it is important enough to do, however, someone must be assigned the responsibility and given the necessary resources. Until this is done, software process development will remain a nice thing to do someday, but never today.
1. Changing the Software Process:
Changing anything as complex as a software process must necessarily involve a host of factors, among which one of the most challenging is the need to change the way the software people do their work. They have typically learned, often through years of experience, how to cope with their many tasks and crises. These professionals must be convinced that changes make sense for them. What is more, software engineering involves complex tasks done under tight schedule pressure. Since change is difficult even without pressure, it is not surprising that software process change is both painfully haphazard.
Software process improvement process is not simple. To achieve tangible results, it needs to be treated like a development task:
Identify the key problems.
Establish priorities.
Define action plan.
Get professional & management agreement.
Assign people.
Provide training & guidance.
Launch implementation.
Track progress.
Fix the inevitable problems.
The task involves a lot of work, which obviously can't be done by any group. The Software Process Engineering Group (SEPG), however, is the focal point for this total effort. Some of the other people who must be involved are senior management, the line projects, SQA, education, finance, administration, and, most important, the software professionals themselves.
2. The Role of the SEPG:
The SEPG has two basic tasks that are done simultaneously: initiating and sustaining process change & supporting normal operations. The SEPG may also serve as the spawning ground for other groups such as technology support, education, cost estimation, standards, and possibly even SQA. Large software groups need professional support in all these areas, and, where it is not available, the SEPG can provide guidance until such functions are established.
Change is an essential part of the software process. While projects must not be disrupted with excessive change, the software process should be viewed as continuous learning. Process change involves learning new methods & accommodating to the changing nature & scale of the problems encountered. The products we build in the future will be far larger & more complex than those of today. Without an established software process, projects will not have an organized experience base
on which to build and must each learn through painful trial & error. A thoughtfully established software process provides a framework for building on the experiences of others.
Change is also needed to bring current software practice up to the level of current knowledge. We typically know how to do our work far better than we are currently doing. Most software problems are caused not by lack of knowledge but by the lack of discipline to apply the best known methods & an inability to effectively handle the myriad of associated process & product detail. Building the disciplined practices to do every software task with precise correctness requires a painstaking improvement program.
Changes are therefore a normal and continuous part of software management, & a key SEPG role is to ensure that these changes are effectively implemented. The rate of change should be aggressive but not disruptive, and it should take advantage of available resources & technology. The key need is for a balanced level of SEPG resources that can be sustained on a permanent basis.
The SEPG should also serve as a consolidating force for the changes that have already been made & should support the projects as they use new methods, standards and technology. This support helps with the adoption of new practices and facilitates their retention in the working fabric of the organization. Without such guidance & support, lasting process improvement is practically impossible.
2.1.The SEPG as a Change Agent:
A change agent provides the energy, enthusiasm, and direction needed to overcome resistance and cause change. [2] The SEPG fills this role by providing the skilled resources, the creativity, and the management leverage needed to make things happen. The decision to make the changes must rest with line management, but the SEPG should provide the technical guidance on what changes are most important. They must be aware of the state of the software process, appreciate the areas needing improvement, and present practical improvement recommendations to management.
When management approves these action plans, the SEPG takes the lead in launching the required efforts, providing leadership in getting them staffed, and supporting the work with needed information, training, and skills. The SEPG also tracks action plan progress and informs management of status and major problems.
While change is essential, it is also important to control the pace of change. If it is too slow, progress will be limited, while too rapid a pace will be disruptive and self-defeating. No group can tolerate perpetual disruption, and no project can operate without reasonable stability. Change must therefore be carefully planned and coordinated with the status and needs of each project.
2.2. The SEPG Sustaining Role:
The continuing role of the SEPG can be divided into six categories:
1. Establish process standards.
2. Maintain the process database.
3. Serve as the focal point for technology insertion.
4. Provide key process education.
5. Provide project consultation.
6. Make periodic assessments and status reports.
3. Establishing Standards:
The SEPG is the focal point for establishing process standards. These include those product-related standards, as well as the process related standards.
Since the SEPG staff cannot have sufficient resources or the resident expertise to develop all these standards, it must establish working teams with members drawn from the most involved groups. With configuration management standards, for example, the SCM group should develop the standard, possibly with SEPG help. SEPG can then assist with reviews, approvals, and implementation.
The SEPG standards role involves the following tasks:
1. The SEPG recommends the priorities for developing standards. There are many potentially valuable standards, but they must be introduced in an orderly way. New standards efforts should generally not be started until the developed standards have been approved and adopted. The existence of proposed but unapproved standards can be both confusing and disruptive. The SEPG should maintain a prioritized list of standards development work.
2. The SEPG should ensure that prior work on the subject is reviewed. With few exceptions, the topics being considered for standardization have been studied by others, and much time and effort can be saved if this work is used as a foundation.
3. The standards development team should include available experts and users. Every standard is a balance between what is theoretically desirable and what can be practically implemented. This balance can only be determined through negotiation between the standardization advocates and the people who must use the result. In the absence of a standards development group, the SEPG should coordinate the formation and chartering of these teams.
4. The final standard review and approval is the most important step. Even when user representatives have participated in the standards development, many conflicting opinions and messy details are involved. A comprehensive review by the prospective users will expose these issues and facilitate the thoughtful technical and management debates required to produce a competent result. Since this review process can be time-consuming and complex, the SEPG should ensure that appropriate tracking and control facilities are used for this purpose.
4. The Process Database:
The process database is the permanent repository for the data gathered on the software engineering process and the resulting products. This data provides the reference for improving estimating accuracy, analyzing productivity, and assessing product quality. It is used by process specialists, project professionals, the quality staff, and management.
The types of information retained in this database are:
Size, cost, and schedule data. This is data on overall process performance. It is used to validate estimates, to generate estimating factors, and to establish contingencies.
Product metrics. These provide basic information on the nature, complexity, structure, and quality of the products. The information is used to assess the results produced and to relate these results to process actions.
Process metrics. These productivity and quality characteristics of detailed process tasks are used to assess task effectiveness and identify the most promising areas for process improvement.
To be most useful, a great deal of information is required for each kind of data. While this is discussed further in Chapter 15, the essential considerations for an effective process database are:
1. The reason for gathering each piece of data must be stated. There is so much potentially useful data that any organization could devote much of its time just to gathering it. This is a mistake, however, because data that is not gathered for a specific purpose is generally too loosely defined to be useful.
2. To define the data, the exact meaning of each field must be specified, together with all the anticipated options and exceptions. Where data is subject to various interpretations, such as lines of code or programmer months, the items to be included must be specified, together with means to identify them (semicolons, department numbers, and so forth). The data specification must be precise enough so a program could be written to gather and process it.
3. Simple and easy-to-use procedures are required for gathering the data, together with tools, forms, training, and assistance.
4. Means are also required for ensuring that the data is obtained in a timely way. The data gathering must be as complete as practical because a database that omits relevant data is just as erroneous as one that includes incorrect data.
5. Means are needed to verify data accuracy. Since data recorded by practicing programmers has been shown to be as much as 50 percent in error, a verification mechanism is essential. [1]
6. Resources and procedures are required for entering the data into the database.
7. Provision must be made for user access to and analysis of the data.
8. Finally, provisions are required to protect and maintain the data. The database must be updated periodically for format or other changes, old data must be purged, periodic back-ups are needed, and someone needs to enhance and correct the database programs.
5. Technology Insertion Focal Point:
Technology support for any reasonably large software engineering organization involves seven activities:
1. A process definition is needed to help identify the software tasks that are both widely enough used and sufficiently common to warrant automated support.
2. A set of requirements is then developed for the needed support.
3. These requirements must be approved by the users, the people who will handle installation and support, and management.
4. A search is then made to identify commercially available tools that meet these needs.
5. An overall technology plan is developed that defines long-term technology needs and establishes a strategy for addressing them. This includes an architectural plan for the support environment and a phased introduction plan (see Chapter 18).
6. A technology support group is established to handle tool and environment installation, provide user assistance, and be the interface to the tool suppliers on problems and enhancements.
7. Education, training, and consultation must also be provided before the tools and methods are installed for general use.
In the absence of a technology support group that provides the above services, the SEPG can help coordinate the technology efforts of the various projects. The SEPG should not, however, attempt to provide all the technology support services. This is a large job that could easily consume the entire SEPGs resources. Either a technology group is formed to carry this load or the projects must fend for themselves with SEPG assistance.
6. Education and Training:
Education could also easily consume all the SEPG resources. The SEPG must serve as the focal point for process education and avoid being saddled with all the teaching responsibilities. Ideally a full-time education group maintains a staff of volunteers, instructors, or consultants to do this job. Where an adequately staffed education group is not available, knowledgeable professionals can generally be found to serve as volunteer instructors.
The SEPG may initially have to teach some of the subjects until qualified instructors can be found or trained. Examples of courses that are often needed are:
Project management methods: how to plan, estimate, and track projects
Software design methods: the use of design languages, object oriented design, and prototyping
Quality management: methods for making quantitative quality estimates and plans
Design and inspections
Most of these topics will generally require considerable post-training consultation before the professionals can be suitably proficient. By assigning professionals who have such experience to work for or with the SEPG as consultants, the entire organization can gain from their knowledge and experience. After these methods are better known and a resident knowledge base has been established, the regular education function can generally handle the teaching responsibilities.
7. Process Consultation:
The key focus of the process group is on improving the practice of software engineering. They do this by working with software practitioners both to provide assistance and to stay current on the projects problems. If they do not stay current, they will not be able to relate to the practicing professionals.
Another reason for SEPG consultation is that few projects can afford their own SEPG group. For projects larger than about 200 or so professionals, a dedicated process capability might make sense, but even then a central group is required to provide cross-organization standards and awareness.
The SEPG can be most helpful to the project by consulting on:
The process data they should gather
The analyses and interpretation of the data gathered
Tuning the standard process to unique project needs
Assisting with the preparation of quality plans
Serving as experienced inspection moderators
Advising on the priority areas for technology insertion
8. Process Status and Assessment:
The SEPG is also responsible for understanding the current state of software practice and alerting management to key problems. This includes:
Awareness of how completely each projects process is defined
Knowledge of how the process is implemented
Judgment on when an assessment would be appropriate
Leadership in conducting assessments
Senior management reviews are conducted at least quarterly. These provide an overview of the state of the software process, a summary of process improvement status and plans, and priority items for management attention. The items required to make such reviews effective are:
1. Goals for process improvement
2. A comparison of the actual process state to prior plans
3. The status of the process improvement actions
4. Identified problems and recommended corrective actions
5. Recommended responsibilities for handling these actions
9. Establishing the SEPG:
The questions to address in setting up an SEPG are:
How big should the effort be?
Where does the staff come from?
Who should head it?
Where should it report?
What are the tasks for initial focus?
What are the key risks?
How is its effectiveness evaluated?
9.1. SEPG Size and Staffing:
While each organization has its own unique needs, a useful initial guide is to aim at full-time assignments to the SEPG of about 2 percent of the software professionals. For organizations of about a hundred or more software professionals, the typical range falls between 1 percent and 3 percent, with the prime determinants being the ability to recruit suitable candidates and the financial constraints of the organization. For smaller organizations, it is essential to have at least one full-time SEPG professional with the part-time support of other professionals on working groups. When the organization is too small to support a full-time professional, the manager must be the de facto process manager with the support of a committee and special working groups.
Staffing the SEPG with competent and experienced professionals is often the most difficult and time-consuming part of establishing this function. Good people are hard to find, and the projects are never willing to give up their best people. If the process improvement efforts have strong management backing, however, some of the better people will often seek these assignments. These are typically experienced professionals who are convinced that there are better ways to do software and would like to participate in finding and installing them. A respected SEPG manager will often hear of these people through the grapevine. Getting them assigned, however, will generally require management support. If not enough qualified people can be obtained in this way, a management tax must be imposed on development. Care is required, however, to guard against getting the lame, the halt, and the tired.
9.2. SEPG Leadership and Reporting:
The SEPG leader must be a knowledgeable manager with a demonstrated ability to make things happen, the respect of the project professionals, and the support of top management. To select this official agent for process change, the key criteria are: [2]
Agents should be enthusiastic about leading the change process. Irwin and Langham point out that enthusiasm can be contagious and people tend to perform better in an optimistic environment than in a wont-work environment. [3]
Agents must be both technically and politically capable of understanding the problems and ensuring that effective solutions are implemented.
Agents need the respect of the people they are to deal with.
Agents must have managements confidence and support or they will not act with the assurance needed to get wide cooperation and acceptance.
The SEPG must not report to line development management or to the SQA organization. In either case their role would likely become one of taking sides in the traditional SQA/development conflicts and their focus on process improvement would suffer. While there is often no natural reporting point in most organizations, there is usually some technical staff function that could provide a neutral home. The SEPG could, for example, report to the same executive reporting point as SQA, the computing center, SCM, or software technology.
9.3. SEPG Priorities and Risks:
The SEPG priorities should roughly follow those given in this book, with initial emphasis on project planning and project management. The most important single guideline is for the SEPG to limit its focus to those tasks it can handle reasonably quickly and effectively. It might, for example, focus entirely on installing an inspection program and defer all other work until this is reasonably underway. The danger of superficially addressing many topics is that none will actually get done.
There are, of course, also key risk situations for the SEPG:
It doesnt have enough full-time capable professionals to do competent work.
The SEPG does not have sufficient management support to convince the projects to support the process improvement efforts.
The SEPG manager is not able to obtain the participation of the most knowledgeable software professionals in the process improvement task groups.
With senior management support, these risks can all be handled, but generally none of them can be handled without it.
9.4. Evaluating the SEPG:
There are many ways to evaluate staff and support groups like the SEPG. Some of the more common methods are to use advisory groups or periodically to poll the users and software professionals for their opinions. While it is probably a good idea to do some of these things, perhaps the best approach is to judge the SEPG on how effectively it applies the software process maturity framework to its own work. This should permit them to present a clear and succinct picture of what they are doing and where they stand against the following criteria:
Level 2: Does the SEPG have a plan for its work, a tracking system, and means to retain and control its work products?
Level 3: Have the SEPG professionals established a basic framework for their own work, including standards, procedures, and a review program?
Level 4: Does the SEPG measure the productivity and quality of its own work? This, for example, might include workload factors for training, consultation, process development, and administration.
Level 5: Do they regularly assess their own activities for improvement opportunities and incorporate them in their working process?
While all of these activities take time and effort to initiate, they will make the SEPG more effective while demonstrating process leadership to the organization. If they are not at least one maturity level ahead of the projects, they may not have the knowledge or conviction to do their job.
10. Summary:
Software process improvement often receives little orderly attention. If it is important enough to do, however, someone must be assigned the responsibility and given the resources to make it happen. Until this is done, it will remain a nice thing to do someday, but never today.
The SEPG has two basic tasks that are done simultaneously: initiating and sustaining process change and supporting normal operations. Process change requires the professionals to learn new methods and to accommodate to the changing nature and scale of the problems they encounter. The SEPG supports this learning while also serving as a consolidating force for the changes that have already been made. As a change agent, it provides the skilled resources, the organizational focus, and the management leverage to make things happen.
The SEPG is the focal point for establishing process standards, and it maintains the process database as a permanent repository for the data gathered on the software engineering process and products. In its role as a technology insertion focal point, the SEPG helps coordinate the projects technology efforts, at least until a permanent technology group is established. Further SEPG roles concern education, consultation, and awareness of the current state of software practice.
A useful initial guide is to assign about 2 percent of the software professionals to the SEPG. Staffing the SEPG with competent and experienced professionals, however, is often the most difficult and time-consuming part of establishing this function. This group should not report to line development management or to the SQA organization. The SEPG could, for example, report to the same executive reporting point as SQA, the computing center, SCM, or advanced software technology.
The initial SEPG priorities should roughly follow those given in this book, with initial emphasis on project planning and project management. The most important single guideline is for the SEPG to.limit its focus to those tasks it can handle reasonably quickly and effectively. The danger of superficially addressing many topics is that none will actually get done.
The major risks in establishing an SEPG are that it will not be adequately staffed, that it will receive insufficient management support to do its job, or that the SEPG manager will not be able to obtain the participation and support of the project leaders and software professionals. While SEPG performance can be evaluated with the traditional means for judging support staffs, probably the best measure is the SEPGs adherence to the software process maturity framework for their own work.
|