ES, IG-99-016, Audit of Advanced X-Ray Astrophysics Facility

AUDIT OF ADVANCED X-RAY
ASTROPHYSICS FACILITY
IG-99-016

Executive Summary

Introduction
The Advanced X-ray Astrophysics Facility⁽¹⁾ (AXAF) is the third of the four "Great Observatories" intended to observe the universe in four electromagnetic spectrum regions: visible, infrared, gamma ray, and x-ray. AXAF is intended to provide unique information based on observations in the x-ray band, on the nature of objects ranging from nearby stars like our Sun to quasars at the edge of the observable universe.
The AXAF program is under the purview of the Office of Space Science. Marshall Space Flight Center (Marshall) has responsibility for managing the day-to-day operations of the AXAF Program including supervision of design, development, prelaunch verification, launch, and orbital verification of the AXAF. The current development cost of the program is about $1.5 billion.
Originally, the AXAF was to be launched during August 1998. In November 1997, the prime contractor informed the AXAF Program Manager that the contractor was not going to meet the scheduled launch date.

Objectives
The overall objective was to evaluate the management response to the initial AXAF launch delay, including procurement and contract administration functions. Specifically, we determined whether:

NASA oversight is sufficient to ensure that schedule, cost, and quality control impacts are minimized because of late delivery of the AXAF.
Contractor performance is adequately monitored and evaluated, and award fees reflect actual performance. Details on the scope and methodology are in Appendix A.

Results of Audit
Overall, NASA responded adequately to the initial AXAF launch delay and has focused additional attention on contractor performance. The AXAF launch delay will increase contract costs by an estimated $28.8 million. The initial delay was caused by problems in software development and inadequate time scheduled for integration and test activities for the AXAF flight and ground software. When software development was identified as a high risk, the AXAF Risk Management Plan was not updated because NASA policy did not require the plan to be updated. Also, NASA did not assign personnel with software expertise at the contractor location. However, when the delivery delay became known, NASA management took action to minimize the impacts and adjusted the contractor award fee to reflect actual performance.

Other Matters of Interest
The prime contractor had not been efficiently processing Certificates of Qualification⁽²⁾ (COQs), which document hardware certification approval and problem reports, prior to shipping the AXAF. The prime contractor had completed only about 44 percent (30 of 67) of the known COQs and 48 percent (135 of 279) of the known problem reports as of February 1998. The AXAF launch could be further delayed if these reports are not processed before shipment. After we brought this issue to management's attention, management placed greater emphasis on processing COQs and problem reports. Therefore, we are making no recommendation on this issue (see Appendix D).

Recommendations
We recommended that management revise the NASA policy to require program managers to update Risk Management Plans as high-risk issues arise, and if NASA management designates software development as a significant risk to a program, management should consider having personnel with software expertise on-site at the contractor's location.

Management's Response
The NASA Chief Engineer forwarded the recommendations to the Program/Project Management Working Group to deliberate and revise NASA policy. The Group will reconvene in late April 1999 and plans to finalize the modification by late summer 1999. If software is identified as a significant management risk, and as part of the risk management plan developed under NASA policy, management may consider on-site software expertise at the contractor's location.

Evaluation of Management's Response
Management's planned actions are responsive to the recommendations. In response to management's comments, we revised the recommendation relating to having personnel with software expertise on site when software-driven programs are designated a significant management risk because it would not be necessary or practical in all cases to assign personnel with software expertise on-site at contractor locations.

FOOTNOTES

1. NASA renamed AXAF the Chandra X-ray Observatory, in honor of the late Indian-American Nobel Laureate Subrahmanyan Chandrasekhar, in December 1998.

2. The COQ is a form which lists the configuration of the hardware and the documentation required to verify that the flight hardware or program critical ground support equipment is qualified for its intended use.

Introduction	The Advanced X-ray Astrophysics Facility⁽¹⁾ (AXAF) is the third of the four "Great Observatories" intended to observe the universe in four electromagnetic spectrum regions: visible, infrared, gamma ray, and x-ray. AXAF is intended to provide unique information based on observations in the x-ray band, on the nature of objects ranging from nearby stars like our Sun to quasars at the edge of the observable universe. The AXAF program is under the purview of the Office of Space Science. Marshall Space Flight Center (Marshall) has responsibility for managing the day-to-day operations of the AXAF Program including supervision of design, development, prelaunch verification, launch, and orbital verification of the AXAF. The current development cost of the program is about $1.5 billion. Originally, the AXAF was to be launched during August 1998. In November 1997, the prime contractor informed the AXAF Program Manager that the contractor was not going to meet the scheduled launch date.
Objectives	The overall objective was to evaluate the management response to the initial AXAF launch delay, including procurement and contract administration functions. Specifically, we determined whether: NASA oversight is sufficient to ensure that schedule, cost, and quality control impacts are minimized because of late delivery of the AXAF. Contractor performance is adequately monitored and evaluated, and award fees reflect actual performance. Details on the scope and methodology are in Appendix A.
Results of Audit	Overall, NASA responded adequately to the initial AXAF launch delay and has focused additional attention on contractor performance. The AXAF launch delay will increase contract costs by an estimated $28.8 million. The initial delay was caused by problems in software development and inadequate time scheduled for integration and test activities for the AXAF flight and ground software. When software development was identified as a high risk, the AXAF Risk Management Plan was not updated because NASA policy did not require the plan to be updated. Also, NASA did not assign personnel with software expertise at the contractor location. However, when the delivery delay became known, NASA management took action to minimize the impacts and adjusted the contractor award fee to reflect actual performance.
Other Matters of Interest	The prime contractor had not been efficiently processing Certificates of Qualification⁽²⁾ (COQs), which document hardware certification approval and problem reports, prior to shipping the AXAF. The prime contractor had completed only about 44 percent (30 of 67) of the known COQs and 48 percent (135 of 279) of the known problem reports as of February 1998. The AXAF launch could be further delayed if these reports are not processed before shipment. After we brought this issue to management's attention, management placed greater emphasis on processing COQs and problem reports. Therefore, we are making no recommendation on this issue (see Appendix D).
Recommendations	We recommended that management revise the NASA policy to require program managers to update Risk Management Plans as high-risk issues arise, and if NASA management designates software development as a significant risk to a program, management should consider having personnel with software expertise on-site at the contractor's location.
Management's Response	The NASA Chief Engineer forwarded the recommendations to the Program/Project Management Working Group to deliberate and revise NASA policy. The Group will reconvene in late April 1999 and plans to finalize the modification by late summer 1999. If software is identified as a significant management risk, and as part of the risk management plan developed under NASA policy, management may consider on-site software expertise at the contractor's location.
Evaluation of Management's Response	Management's planned actions are responsive to the recommendations. In response to management's comments, we revised the recommendation relating to having personnel with software expertise on site when software-driven programs are designated a significant management risk because it would not be necessary or practical in all cases to assign personnel with software expertise on-site at contractor locations.