04-11

Procurement Notice


December 29, 2005

ADDITIONAL COMPETITIVE PROCEDURES FOR NASA ANNOUNCEMENTS OF OPPORTUNITY

 

Background:  NASA’s Broad Agency Announcement (BAA)/Announcement of Opportunity (AO) procurements are conducted as public/private competitions meeting the spirit and intent of the guidelines promulgated by the Competitive Sourcing element of the President’s Management Agenda.  The agency seeks to increase the effectiveness of BAA/AO competitions and raise the efficiency level with which the BAA/AO competitions are conducted.

 

ACQUISITIONS AFFECTED BY CHANGES:  All solicitations released and contracts awarded as a result of the BAA/AO processes.

 

ACTION REQUIRED BY CONTRACTING OFFICERS:  Compliance with the revised guidance addressing proposal preparation procedures as contained in the newly added section 1872.308, and adherence to the clarification language regarding requirements for justifications for other than full and open competition, as contained in the revisions to section 1872.502.

 

CLAUSE CHANGES:  None

 

PARTS AFFECTED:  Changes are made in Part 1872.

 

REPLACEMENT PAGES:  You may use the enclosed pages to replace 72:1, 72:2, 72:7 through 72:32 of the NFS.

 

TYPE OF RULE AND PUBLICATION DATE:  Not Applicable.  This change does not have a significant effect beyond the internal operating procedures of the Agency or have a significant cost or administrative impact on contractors or offerors and therefore does not require codification in the Code of Federal Regulations (CFR) or publication for public comment.

 

HEADQUARTERS CONTACT:  Jamiel Commodore, Contract Management Division, (202) 358-0302, email: jamiel.c.commodore@nasa.gov.

 

 

Tom Luedtke

Assistant Administrator for Procurement

 

Enclosures

 

 

DISTRIBUTION:

  PN List



PART 1872

ACQUISITIONS OF INVESTIGATIONS

 

TABLE OF CONTENTS

 

1872.000                                 Scope of part.

 

SUBPART     1872.1             THE INVESTIGATION ACQUISITION SYSTEM

1872.101                                 General.

1872.102                                 Key features of the system.

1872.103                                 Management responsibilities.

 

SUBPART     1872.2             APPLICABILITY OF THE PROCESS

1872.201                                 General.

1872.202                                 Criteria for determining applicability.

1872.203                                 Applicable programs and activities.

1872.204                                 Approval.

 

SUBPART     1872.3             THE ANNOUNCEMENT OF OPPORTUNITY

1872.301                                 General.

1872.302                                 Preparatory effort.

1872.303                                 Responsibilities.

1872.304                                 Proposal opportunity period.

1872.305                                 Guidelines for announcement of opportunity.

1872.306                                 Announcement of opportunity soliciting foreign participation.

1872.307                                 Guidelines for proposal preparation.

1872.308                                 Proposals submitted by NASA investigators.

 

SUBPART      1872.4             EVALUATION OF PROPOSALS

1872.401                                 General.

1872.402                                 Criteria for evaluation.

1872.403                                 Methods of evaluation.

1872.403-1                              Advisory subcommittee evaluation process.

1872.403-2                              Contractor evaluation process.

1872.403-3                              Government evaluation process.

1872.404                                 Engineering, integration, and management evaluation.

1872.405                                 Program office evaluation.

1872.406                                 Steering committee review.

1872.407                                 Principles to apply.

 

SUBPART     1872.5             THE SELECTION PROCESS

1872.501                                 General.

1872.502                                 Decisions to be made.

1872.503                                 The selection statement.

1872.504                                 Notification of proposers.

1872.505                                 Debriefing.

SUBPART     1872.6             PAYLOAD FORMULATION

1872.601                                 Payload formulation.

 

SUBPART     1872.7             ACQUISITION AND OTHER CONSIDERATIONS

1872.701                                 Early involvement essential.

1872.702                                 Negotiation, discussions and contract award.

1872.703                                 Applications of the Federal Acquisition Regulation (FAR) and the

                                                  NASA FAR Supplement.

1872.704                                 Other administrative and functional requirements.

1872.705                                 Format of announcement of opportunity.

1872.705-1                              Appendix A:  General instructions and provisions.

1872.705-2                              Appendix B:  Guidelines for proposal preparation.

1872.705-3                              Appendix C:  Glossary of terms and abbreviations associated with

                                                  investigations.

 

 

PART 1872

ACQUISITIONS OF INVESTIGATIONS

 

1872.000  Scope of part.

   This part prescribes policies and procedures for the acquisition of investigations.

 

Subpart 1872.1--The Investigation Acquisition System

 

1872.101  General.

   The investi­ga­tion acquisition system encourages the participation of investigators and the selec­tion of investigations which contribute most effectively to the advancement of NASA's scientific and technological objectives.  It is a system separate from the acquisition process, but requiring the same management and discipline to assure compliance with statutory requirements and considerations of equity.

 

1872.102  Key Features of the system. 

   (a)(1) Use of the system commences with the Enterprise Associate Administrator's determination that the investigation acquisition process is appropriate for a program. An Announcement of Opportunity (AO) is disseminated to the interested scientific and technical communities. The AO is a form of broad agency announcement (BAA) (see FAR 35.016 and 1835.016 for general BAA requirements). This solicitation does not specify the investigations to be proposed but solicits investigative ideas which contribute to broad objectives. In order to determine which of the proposals should be selected, a formal competitive evaluation process is utilized. The evaluation for merit is normally made by experts in the fields represented by the proposals. Care should be taken to avoid conflicts of interest. These evaluators may be from NASA, other Government agencies, universities, or the commercial sector. Along with or subsequent to the evaluation for merit, the other factors of the proposals, such as engineering, cost, and integration aspects, are reviewed by specialists in those areas. The evaluation



spread over the number of months most compatible with the possible flight opportunities and the avail­ability of resources necessary to evaluate and fund the proposals.

       (2) The AO may be issued establish­ing a single proposal submission date.  However, the AO could provide that NASA amend the AO to provide for subse­quent dates for submission of proposals, if addi­tional inves­tigations are desired within the AO objec­tives. 

       (3) The AO may provide for an initial submission date with the AO to remain open for submission of additional proposals up to a final cutoff date.  This final date should be related to the availabil­ity of resources necessary to evaluate the continu­ous flow of proposals, the time remaining prior to the flight opportunity(s) contem­plated by the AO, and payload fund­ing and availability.

   (b) Generally, a core payload of investi­gations would be selected from the initial submission of proposals under the above methods of open-ended AOs.  These selec­tions could be final or tentative recog­nizing the need for further definition.  Proposals received by subsequent submis­sion

dates would be considered in the scope of the original AO but would be subject to the opportunities and resources remaining available or the progress being made by prior selected investigations.

   (c) Any proposal, whether received on the initial submission or subsequent submis­sion, requires notification to the investiga­tor and the investigator's institu­tion of the proposal disposition. Some of the proposals will be rejected completely and the investi­gators immediately notified.  The remaining unselected proposals may, if agreeable with the proposers, be held for later consider­ation and funding and the investigator so notified.  However, if an investigator's proposal is considered at a later date, the investigator must be given an opportunity to validate the proposal with the investigator's institution and for updat­ing the cost and other data contained in the original submission prior to a final selec­tion.  In summary, NASA may retain  pro­posals, receiving Category I, II, or III classi­fications (see 1872.403-1(e)), for possible later sponsorship until no longer feasible to consider the proposal.  When this final stage is reached, the investigator must be prompt­ly notified. Proposing investigators not desiring their proposals be held for later consider­ation should be given the opportu­nity to so indicate in their original submis­sions.

 

1872.305  Guidelines for announcement of opportunity.

   (a) The AO should be tailored to the particular needs of the contemplated inves­tigations and be complete in itself.  Each AO will identify the originating program office and be numbered consecutively by calendar year, e.g., OA‑1-95, OA-2-95; OLMSA-1-95; OSS-1-95; etc.  The required format and detailed instructions regarding the contents of the AO are contained in 1872.705.

   (b) The General Instructions and Provi­sions (1872.705-1) are necessary to accom­modate the unique aspects of the AO pro­cess.  Therefore, they must be appended to each AO.

   (c) At the time of issuance, copies of the AO must be furnished to Headquarters, Office of Procurement (Code HS) and Office of Gener­al Counsel (Code GK).

   (d) Proposers should be informed of significant departures from scheduled dates for activities related in the AO.

 

1872.306  Announcement of opportunity soliciting foreign participation.

   Foreign proposals or U.S. proposals with foreign participation shall be treated in accordance with 1835.016-70.  Additional guidelines applicable to foreign proposers are contained in the Management Plan Section of 1872.705-2 and must be included in any Guidelines for Proposal Preparation or otherwise furnished to foreign proposers.

 

1872.307 Guidelines for proposal preparation.

  While not all of the guidelines outlined in 1872.705-2 will be applicable in response to every AO, the investigator should be informed of the relevant infor­mation re­quired.  The proposal may be submitted on a form supplied by the Pro­gram Office.  However, the proposal should be submitted in at least two sections:  (a) Investigation and Technical Plan; and (b) Management and Cost Plan as de­scribed in 1872.705-2.  Investigators shall be required to identify and discuss risk factors and issues throughout the proposal where they are relevant, and describe their approach to managing these risks.

 

1872.308 Proposals submitted by NASA investigators.

   (a) NASA solicits, accepts, and evaluates proposals submitted by NASA Centers in response to an AO.  A NASA investigator may team with one or more non-Government co-investigators.  A NASA investigator may also need to acquire supplies, including instruments and other hardware, and non-research services in support of the proposed investigation.  If a proposal submitted by a NASA Center is selected, formal assembly of the team and acquisition of hardware and support services must be accomplished through the award of new Government contracts, unless existing Government contracts are available.  The award of new Government contracts must comply with procurement laws and regulations.

   (b) In addition to complying with proposal preparation instructions contained in the AO, proposals submitted by NASA Centers should address the following matters. 

       (1) Non-Government co-investigators. 

            (i) The proposal should describe the open and competitive process that was used for selecting proposed team members.  While a formal solicitation is not required, the process should include the following competitive aspects: notice of the opportunity to participate to potential sources, submissions from and/or discussions with potential sources, and objective criteria for selecting team members among interested sources.  If proposed team members are selected without using an open and competitive process, the proposal should contain a full justification consistent with the requirements of FAR Subpart 6.3. 

            (ii) The proposal should also include a representation that the NASA investigator has examined his/her financial interests and has determined that no personal conflict of interest exists.

       (2) Supplies and support services.        

            (i) The proposal should indicate that the supplies or services are available under an existing Government contract; or

            (ii) The proposal should state that the supplies or services will be acquired under a full and open competition; or

            (iii) The proposal should explain the basis of a justification for acquiring the supplies or services noncompetitively (see FAR Subpart 6.3). 

  (c) A selection decision approving the non-Government team members as selected co-investigators satisfies legal and regulatory requirements without further competition or justification (see 1872.702). 

   (d) For the acquisition of supplies, including hardware, and support services by non-Government co-investigators, see 1872.502(a)(4). 

Subpart 1872.4--Evaluation of Proposals

 

1872.401  General.

   (a) The evaluation process consid­ers the aspects of each proposal by the following progressive sorting: 

       (1) A review resulting in a categoriza­tion is performed by using one of the meth­ods or combination of the methods outlined in 1872.403.  The purpose of this initial review is to determine the scientific and/or technological merit of the proposals in the context of the AO objectives. 

       (2) Those proposals which are considered to have the greatest scientific or technological merit are then reviewed in detail for the engineering, management, and cost aspects, usually by the project office at the installation re­sponsible for the project. 

       (3) Final reviews are performed by the program office and the steering committee and are aimed at devel­oping a group of investigations which rep­resent an integrated payload or a well-balanced program of investigation which has the best possibility for meeting the AO’s objectives within programmatic constraints. 

   (b) The importance of considering the interrelationship of the several aspects of the proposals to be reviewed in the pro­cess and the need for carefully planning their treatment should not be overlooked.  An evaluation plan should be developed before issuance of the AO.  It should cover the recommended staffing for any subcommit­tee or contractor support, review guidelines as well as the procedural flow and schedule of the evaluation.  While not mandatory, such a plan should be considered for each AO.  A fuller discussion of the evaluation and selection process is included in the following sections of this subpart.

 

1872.402  Criteria for evaluation.

   (a) Each AO must indicate those criteria which the evaluators will apply in evaluat­ing a proposal.  The relative impor­tance of each criterion must also be stated.  This information will allow investigators to make informed judgments in formulating proposals that best meet the stated objec­tives.

   (b) Following is a list of general evalua­tion criteria appropriate for inclu­sion in most AOs:

       (1) The scientific, applica­tions, and/or technological merit of the investiga­tion.

       (2) The relevance of the pro­posed inves­tigation to the AO's stated scien­tific, appli­cations, and/or technological objec­tives.

       (3) The competence and expe­rience of the investigator and any investi­gative team.

       (4) Adequacy of whatever appara­tus may be proposed with particular regard to its ability to supply the data needed for the investigation.

       (5) The reputation and interest of the investigator's institution, as mea­sured by the willingness of the institution to provide the support necessary to ensure that the investigation can be completed satisfac­torily.

       (6) Cost and management aspects will be considered in all selections.

       (7) The proposed approach to managing risk (e.g., level of technology maturity being applied or developed, technical complexity, performance specifications and tolerances, delivery schedule, etc.).

       (8) Other or additional criteria may be used, but the evaluation criteria must be germane to the accomplishment of the stated objectives.

   (c) Once the AO is issued, it is essen­tial that the evaluation criteria be applied in a uniform manner.  If it becomes appar­ent, before the date set for receipt of pro­posals, that the criteria or their relative impor­tance should be changed, the AO will be amended, and all known recipients will be informed of the change and given an ade­quate opportunity to consider it in submis­sion of their proposals.  Evaluation criteria and/or their relative importance will not be changed after the date set for receipt of proposals.

 

1872.403  Methods of evaluation.

   Alternative methods are available to initiate the evaluation of proposals received in response to an AO.  These are referred to as the Advisory Subcommittee Evaluation Process, the Contractor Evaluation Process, and the Government Evaluation Process.  In all processes, a subcommittee of the appro­priate Program Office Steering Committee will be formed to categorize the proposals.   Following categorization, those proposals still in consideration will be processed to the selection official.

 

1872.403-1  Advisory subcommittee evaluation process.

   (a) Evaluation of scientific and/or tech­nological merit of proposed investiga­tions is the responsibility of an advisory subcom­mittee of the Steering Committee.  The subcommittee constitutes a peer group qualified to judge the scientific and technological aspects of all investiga­tion proposals.  One or more subcommittees may be established depending on the breadth of the technical or scientific disci­plines inher­ent in the AO's objectives.  Each subcom­mittee represents a discipline or grouping of closely related disciplines.  To maximize the quality of the subcommittee evaluation and categorization, the follow­ing conditions of selection and appointment should be consid­ered.

       (1) The subcommittee normal­ly should be established on an ad hoc basis.

       (2) Qualifications and acknowledgment of the professional abili­ties of the subcom­mittee members are of primary importance.  Institutional affilia­tions are not sufficient qualifications.

       (3) The executive secretary of the sub­committee must be a full-time NASA em­ployee.

       (4) Subcommittee members should nor­mally be appointed as early as possible and prior to receipt of proposals.

       (5) Care must be taken to avoid conflicts of interest.  These include finan­cial inter­ests, institutional affilia­tions, professional biases and associations, as well as familiar relationships.  Conflicts could further occur as a result of imbalance between Govern­ment and non-Government appointees or membership from institutions representing a singular school of thought in discipline areas involving competitive theories in approach to an investigation.

       (6) The subcommittee should con­vene as a group in closed sessions for pro­posal evaluation to protect the proposer's propri­etary ideas and to allow frank discus­sion of the proposer's qualifications and the merit of the proposer's ideas.  Lead review re­sponsibility for each proposal may be as­signed to members most qualified in the involved discipline.  It is important that each proposal be considered by the entire subcommittee.

   (b) It may not be possible to select a subcommittee fully satisfying all of the conditions described in paragraph (a) of this section.  It is the responsibility of the nominating and appointing officials to make trade-offs, where necessary, among the criteria in paragraph (a) of this section. This latitude permits flexibility in making deci­sions in accord with circumstances of each application.  In so doing, however, it is emphasized that recognized expertise in evaluating dissimilar proposals is essential to the continued workability of the investi­gation acquisition process.

   (c) Candidate subcommittee mem­bers should be nominated by the office having responsibility for the evaluation.  Nomina­tions should be approved in accor­dance with NPD 1150.11, "Federal Advisory Committee Act Committees."  The notification of appoint­ment should specify the duration of assign­ment on the subcommittee, provisions con­cerning conflicts of interest, and arrange­ments regarding honoraria, per diem, and travel when actually employed.

   (d) It is important that members of the subcommittee be formally instructed as to their responsibilities with respect to the investigation acquisition process, even where several or all of the members have served previously.  This briefing of subcom­mittee members should include:

       (1) Instruction of subcommittee mem­bers on agency policies and procedures pertinent to acquisition of investigations.

       (2) Review of the program goals, AO objectives, and evaluation crite­ria, includ­ing relative importance, which provide the basis for evaluation.

       (3) Instruction on the use of prelimi­nary proposal evaluation data fur­nished by the Installation Project Office.  The sub­commit­tee should examine these data to gain a better understanding of the proposed inves­tigations, any associated problems, and to consider cost in relation to the value of the investigations' objectives.

       (4) Definition of responsibility of the subcommittee for evaluation and cate­gori­zation with respect to scientific and/or technical merit in accordance with the evaluation criteria.

       (5) Instruction for documenta­tion of deliberations and categorizations of the subcommittee.

       (6) Inform the chairperson of the sub­committee and all members that they should familiarize themselves with the provisions of the Standards of Ethical Conduct for Employees of the Executive Branch, 5 CFR Part 2635, and the Supplemental Standards of Ethical Conduct for employees of the National Aeronautics and Space Administration, 5 CFR Part 6901,  regarding conflicts of interest.  Members should inform the appointing authority if their participation presents a real or apparent conflict of interest situa­tion.  In addition, all partici­pants should inform the selection official in the event they are subjected to pressure or improper contacts.

       (7) Inform members that prior to the selection and announcement of the success­ful investigators and investigations, sub­committee members and NASA person­nel shall not reveal any information con­cerning the evaluation to anyone who is not also participating in the same evaluation pro­ceedings, and then only to the extent that such information is required in connec­tion with such proceedings.  Also, inform mem­bers that subsequent to selection of an investigation and announcement of negotia­tions with the investigator's institution, information concerning the proceedings of the subcommittee and data developed by the subcommittee will be made available to others within NASA only when the request­or demonstrates a need to know for a NASA purpose. Such information will be made available to persons outside NASA includ­ing other Government agencies, only when such disclosure is concurred in by the Of­fice of General Counsel.  In this connection, reference is made to 18 U.S.C. 1905 which provides  criminal sanctions if any officer or employee (including special employees) of the United States discloses or divulges certain kinds of business confidential and trade secret information unless authorized by law.

   (e) The product of an advisory subcom­mittee is the classification of pro­posals into four categories.  The categories are:

       (1) Category I--Well conceived and scien­tifically and technically sound investi­ga­tions pertinent to the goals of the pro­gram and the AO's objectives and of­fered by a competent investigator from an institu­tion capable of supplying the neces­sary support to ensure that any essential flight hardware or other support can be delivered on time and that data can be properly re­duced, analyzed, interpreted, and published in a reasonable time.  Investi­gations in Category I are recommended for acceptance and normally will be displaced only by other Category I investigations.

       (2) Category II--Well conceived and scientifically or technically sound investi­gations which are recommended for accep­tance, but at a lower priority than Category I.

       (3) Category III--Scientifically or tech­nically sound investigations which require further development.  Category III investi­gations may be funded for develop­ment and may be reconsidered at a later time for the same or other opportunities.

       (4) Category IV--Proposed investiga­tions which are recommended for rejection for the particular opportunity under con­sider­ation, whatever the reason.

   (f) A record of the deliberations of the subcommittee shall be prepared by the assigned executive secretary and shall be signed by the Chairperson.  The minutes shall contain the categorizations with basic rationale for such ratings and the significant strengths and weaknesses of the proposals evaluated.

 

1872.403-2  Contractor evaluation process.

   (a) The use of the contractor method for obtaining support for evaluation pur­poses of proposals received in response to an AO requires the approval of the Program AA.  Prior to the use of this method, dis­cussion should be held with the Office of Acquisition.

   (b) It is NASA policy to avoid situa­tions in the acquisition process where, by virtue of the work or services performed for NASA, or as a result of data acquired from NASA or from other entities, a partic­ular company:

       (1) Is given an unfair competi­tive ad­vantage over other companies with respect to future NASA business;

       (2) Is placed in a position to affect Gov­ernment actions under circum­stances in which there is potential that the company's judgment may be biased; or

       (3) Otherwise finds that a conflict exists between the performance of work or servic­es for the Government in an impartial manner and the company's own self-inter­est.

   (c) To reduce the possibility of an orga­nizational conflict of interest problem arising, the following minimum restrictions will be incorporated into the contract:

       (1) No employee of the con­tractor will be permitted to propose in response to the AO;

       (2) The "Limitation on Future Contract­ing" clause contained in 1852.209-71 will be included in all such contracts; and

       (3) Unless authorized by the NASA con­tracting officer, the contractor shall not contact the originator of any proposal concerning its contents.

   (d) The scope of work for the select­ed contractor will provide for an identifica­tion of strengths and weaknesses and a   summary of the proposals.  The contractor will not make selections nor recommend investigations.

   (e) The steps to be taken in estab­lishing evaluation panels and the responsi­bilities of NASA and the contractor in relation to the panels will be as follows:

       (1) The contractor will be required to establish and provide support to panels of experts for review of proposals to evalu­ate their scientific and technical merit;

       (2) These panels will be com­posed of scientists and specialists qualified to evalu­ate the proposals;

       (3) The agency may provide to the con­tractor lists of scientist(s) and spe­cial­ist(s) in the various disciplines it be­lieves are qualified to serve on the panels;

       (4) The contractor will report each panel's membership to NASA for approval; and

       (5) The contractor must make all the necessary arrangements with the panel members.

   (f) The evaluation support by the contractor's panels of experts will be ac­complished as follows:

       (1) The panels will review the scien­tific and technical merit of the propos­als in accordance with the evaluation crite­ria in the AO and will record their strengths and weaknesses.

       (2) The contractor will make records of each panel's deliberations which will form the basis for a report summariz­ing the results of the evaluations.  Upon request, the contractor shall provide all such records to NASA;

       (3) The chairperson of each panel shall certify that the evaluation report correctly represents the findings of the review panel; and

       (4) A final report will be sub­mitted as provided in the contract.

   (g) A subcommittee of the Program Of­fice Steering Committee will be estab­lished on an ad hoc basis.  Utilizing fur­nished data, the subcommittee will classify the proposals into the four categories enu­merat­ed in 1872.403-1(e)(1), Advisory Sub­commit­tee Evaluation Process.  A record of the deliberations of the subcommittee should be prepared by an assigned execu­tive secretary and signed by the chairper­son.  The minutes should contain the catego­rizations with the basic rationale for such ratings and the significant strengths and weaknesses of the proposals evaluated.

 

1872.403-3  Government evaluation process.

   (a) The Program AA may appoint one or more full-time Government employees as subcom­mittee members of the Program Office Steering Committee to evaluate and catego­rize the proposals.

   (b) Each subcommittee member should be qualified and competent to evalu­ate the proposals in accordance with the AO evalu­ation criteria.  It is important that a subcommittee's evaluation not be influ­enced by others either within or outside of NASA.

   (c) The subcommittee members will not contact the proposers for additional infor­mation.

   (d) The subcommittee members will classi­fy the proposals in accordance with the four categories indicated in 1872.403-1(e)(1).  Each categorization will be supported by an appropriate rationale including a narrative of each proposal's strengths and weaknesses.

 

1872.404  Engineering, integration, and management evaluation.

   (a) The subcommittee responsible for categorization of each proposal in terms of its scientific, applications, or technical merit should receive information on proba­ble cost, technical status, developmental risk, integration and safety problems, and management arrangements in time for their deliberations.

   (b) This information should be provided at the discretion of the Headquar­ters Pro­gram Office by the Project Office at the installation.  This information can be in general terms and should reflect what insights the Project Office can provide without requesting additional details from the proposers.  This limited Project Office review will not normally give the subcom­mittees information of significant preci­sion.  The purpose is to give the subcommit­tee sufficient information so it can review the proposals in conjunction with available cost, integration, and management consider­ations to gain an impression of each investi­gator's understanding of the problems of the experiment and to permit gross trade-offs of cost versus value of the investiga­tion objective.

   (c) Following categorization, the Project Office shall evaluate proposals in conten­tion, in depth, including a thorough review of each proposal's engineering, integration, management, and cost aspects.  This review should be accomplished by qualified engi­neering, cost, and business analysts at the project center.

   (d) In assessing proposed costs, the evalu­ation must consider:

       (1) The investigation objec­tive.

       (2) Comparable, similar or related inves­tigations.

       (3) Whether NASA or the inves­tiga­tor should procure the necessary sup­porting instrumentation or services and the relative cost of each mode.

       (4) Total overall or probable costs to the Government including integra­tion and data reduction and analysis.  In the case of investigations proposed by Govern­ment investigators, this includes all associ­ated direct and indirect cost.  With respect to cooperative investigations, integration, and other applicable costs should be consid­ered.

   (e) The Project Office, as part of the in-depth evaluation of proposals that re­quire instrumentation or support equip­ment, will survey all potential sources for Govern­ment-owned instrumentation or support equipment that may be made avail­able, with or without modifications, to the poten­tial investigator.  Such items contrib­uted by foreign cooperating groups which are still available under cooperative project agree­ments will also be considered for use under the terms and conditions specified in the agreements.  As part of the evaluation report to the Program Office, the availabili­ty or nonavailability of instrumentation or support equipment will be indicated.

   (f) Proposals which require instru­men­ta­tion should be evaluated by project per­son­nel.  This evaluation should cover the inter­faces and the assessment of develop­ment risks.  This evaluation should furnish the selection official with sufficient data to contribute to the instrument determinations.  Important among these are:

       (1) Whether the instrument requires fur­ther definition;

       (2) Whether studies and designs are nec­essary to provide a reasonably accu­rate appreciation of the cost;

       (3) Whether the investigation can be carried out without incurring undue cost, schedule, or risk of failure penalties; and

       (4) Whether integration of the in­stru­ment is feasible.

   (g) In reviewing an investigator's man­agement plan, the Project Office should evaluate the investigator's approach for efficiently managing the work, the recogni­tion of essential management functions, and the effective overall integration of these functions.  Evaluation of the proposals under final consideration should include, but not be limited to: workload--present and future related to capacity and capabili­ty; past experience; management approach and organization; e.g.:

       (1) With respect to workload and its rela­tionship to capacity and capabil­ity, it is important to ascertain the extent to which the investigator is capable of provid­ing facilities and personnel skills necessary to perform the required effort on a timely basis.  This review should reveal the need for additional facilities or people, and provide some indication of the Government support the investigator will require.

       (2) A review should be made of the inves­tigator, the investigator's insti­tution, and any supporting contractor's perfor­mance on prior investigations.  This should assist in arriving at an assessment of the investigator and the institution's ability to perform the effort within the proposed cost and time constraints.

       (3) The proposed investigator's manage­ment arrangements should be re­viewed, including make or buy choices, support of any co-investigator, and preselected subcon­tractors or other instrument fabricators to determine whether such arrangements are justified.  The review should determine if the proposed manage­ment arrangements enhance the investigator's ability to devote more time to the proposed experiment ob­jectives and still effectively employ the technical and ad­ministrative support re­quired for a success­ful investigation.  In making these evalua­tions, the Project Of­fice should draw on the installation's engi­neering, business, legal, and other staff resources, as necessary, as well as its scien­tific resources.  If further information is needed from the proposers, it should be obtained through the proper contacts.

 

1872.405  Program Office evaluation.

   (a) A Program Office responsible for the project or program at Headquarters will receive the evaluation of the proposals, and weigh the evaluative data to determine an optimum payload or program of investi­ga­tion.  This determination will involve rec­ommendations concerning individual inves­tigations; but, more importantly, should result in a payload or program which is judged to optimize total mission return within schedule, engineering, and budget­ary constraints.  The recommendations should facilitate sound selection decisions by the Program AA.  Three sets of recom­mendations result from the Program Office evaluation:

       (1) Optimum payload or pro­gram of investigations, or options for alter­native payloads or programs.

       (2) Recommendation for final or tenta­tive selection based on a determina­tion of the degree of uncertainty associated with individual investigations.  A tentative selection may be considered step one of a two-step selection technique.

       (3) Upon consideration of the guide­lines contained in 1872.502(a)(3), recom­mend­ing responsibility for instrument develop­ment.

   (b) The Installation Project Office eval­uation is principally concerned with ensur­ing that the proposed investigation can be managed, developed, integrated, and execut­ed with an appropriate probability of tech­nical success within the estimated probable cost.  The Headquarters program Director, drawing upon these inputs, should be main­ly concerned with determining a payload or program from the point of view of pro­grammatic goals and budgetary con­straints.  Discipline and cost trade-offs are consid­ered at this level.  The Headquarters Pro­gram Office should focus on the poten­tial contribution to program objectives that can be achieved under alternative feasible payload integration options.

   (c) It may be to NASA's advantage to consider certain investigations for tenta­tive selection pending resolution of uncer­tain­ties in their development.  Tentative selec­tions should be reconsidered after a period of time for final selection in a pay­load or program of investigations.  This two-step selection process should be consid­ered when:

       (1) The potential return from the inves­ti­gation is sufficient, relative to that of the other investigations under con­sider­ation, and that its further development appears to be warranted before final selec­tion.

       (2) The investigation potential is of such high priority to the program that the inves­tigation should be developed for flight if at all possible.

       (3) The investigative area is critical to the program and competitive approaches need to be developed further to allow selec­tion of the optimum course.

   (d) Based on evaluation of these consid­er­ations associated with the investi­gations requiring further development of hardware, the following information should be provid­ed to the Steering Committee and the Pro­gram AA responsible for selection:

       (1) The expected gain in po­tential return associated with the eventual incorpo­ration of tentatively recommended investi­gations in the payload(s) or program.

       (2) The expected costs required to devel­op instrumentation to the point of "demon­strated capability."

       (3) The risk involved in added cost, prob­ability of successfully developing the re­quired instrument capability, and the possi­bility of schedule impact.

       (4) Identification of opportu­nities, if any, for inclusion of such investi­gations in later missions.

   (e) In those cases where investiga­tions are tentatively selected, an explicit state­ment should be made of the process to be fol­lowed in determining the final pay­load or program of investigations and the pro­posers so informed.  The two-phase selection ap­proach provides the opportunity for addi­tional assurance of development poten­tial and probable cost prior to a final com­mit­ment to the investigation.

   (f) As instruments used in investi­ga­tions become increasingly complex and costly, the need for greater control of their develop­ment by the responsible Headquar­ters Pro­gram Office also grows. According­ly, as an integral part of the evaluation process, a deliberate decision should be made regard­ing the role of the Principal Investigator with respect to the provision of the major hardware associated with that person's investigation.  The guidelines for the hard­ware acquisition determination are dis­cussed in 1872.502(a)(3).

   (g) The range of options for respon­sibili­ty for the instrumentation consists of:

       (1) Assignment of full responsi­bility to the Principal Investigator.  The responsi­bility includes all in-house or con­tracted activity to provide the instrumenta­tion for integration.

       (2) Retention of developmen­tal respon­sibility by the Government with participa­tion by the Principal Investigator in key events defined for the program.  In all cases the right of the Principal Investi­gator to counsel and recommend is para­mount.  Such involvement of the Principal Investigator may include:

            (i)    Provision of instrument specifica­tions.

            (ii)   Approval of specifica­tions.

            (iii)   Independent monitorship of the development and advice to the Govern­ment on optimization of the instru­mentation for the investigation.

            (iv)  Participation in design re­views and other appropriate reviews.

            (v)   Review and concur­rence in chang­es resulting from design reviews.

            (vi)  Participation in config­ura­tion con­trol board actions.

            (vii)  Advice in definition of test pro­gram.

            (viii) Review and approval of test pro­gram and changes thereto.

            (ix)  Participation in conduct of the test program.

            (x)   Participation in cali­bration of instrument.

            (xi)  Participation in final in­spection and acceptance of the instru­ment.

            (xii)  Participation in sub­sequent test and evaluation processes inci­dent to inte­gration and flight preparation.

            (xiii) Participation in the devel­opment and support of the operations plan.

            (xiv) Analysis and interpre­tation of data.

   (h) The Principal Investigator should as a minimum:

       (1) Approve the instrument specifi­ca­tion.

       (2) Advise the project manager in devel­opment and fabrication.

       (3) Participate in final cali­bration.

       (4) Develop and support the opera­tions plan.

       (5) Analyze and interpret the data.

   (i)  The Project Installation is re­sponsi­ble for implementing the program or project and should make recommendations concern­ing the role for the Principal Inves­tigators.  The Program AA will determine the role, acting upon the advice of the Headquarters Program Office and the Steering Commit­tee.  The Principal Investigator's desires will be respected in the negotiation of the person's role allowing an appeal to the Program AA and the right to withdraw from participation.

   (j) The Program Office should make a presentation to the Steering Com­mittee with supporting documentation on the decisions to be made by the responsible Program AA.

 

1872.406  Steering Committee review.

   (a) The most important role of the Steer­ing Committee is to provide a substan­tive review of a potential payload or pro­gram of investigations and to recommend a selection to the Program AA.  The Steering Commit­tee applies the collective experience of representatives from the program and disci­pline communities and offers a forum for discussing the selection from those points of view.  In addition to this mission-specific evaluation function, the Steering Commit­tee provides guidance to subcommit­tee chairpersons and serves as a clearing­house for problems and complaints regard­ing the process.  The Steering Committee is respon­sible for assuring adherence to re­quired procedures.  Lastly, it is the forum where discipline objectives are weighed against program objectives and constraints.

   (b) The Steering Committee repre­sents the means for exercising three respon­sibili­ties in the process of selecting investi­ga­tions to:

       (1) Review compliance with proce­dures governing application of the AO process.

       (2) Ensure that adequate docu­menta­tion has been made of the steps in the eval­uation process.

       (3) Review the results of the evalua­tion by the subcommittee, Project, and Program Offices and prepare an assess­ment or en­dorsement of a recommended payload or program of investigations to the Program AA.

   (c) The purpose in exercising the first of the responsibilities in paragraph (b) of this section is to ensure equity and consistency in the application of the pro­cess.  The Steering Committee is intend­ed to provide the necessary reviews and coordina­tion inherent in conventional acquisition practices.

   (d) The second and third responsibil­ities of the Steering Committee in paragraph (b) are technical.  They require that the Steering Commit­tee review the evaluations by subcommittee, the Project Office, and the Program Office for completeness and appropriateness be­fore forwarding to the Program AA.  Most im­portant in this review are:

       (1) Degree to which results of evalu­ations and recommendations follow logical­ly from the criteria in the AO.

       (2) Consistency with objectives and poli­cies generally beyond the scope of Project/Program Offices.

       (3) Sufficiency of reasons stated for tentative recommendations of those investi­gations requiring further in­strument re­search and development.

       (4) Sufficiency of reasons stated for determining responsibilities for instrument development.

       (5) Sufficiency of consider­ation of reus­able space flight hardware and support equipment for the recommended investiga­tions.

       (6) Sufficiency of reasons for classi­fying proposed investigations in their re­spective categories.

       (7) Fair treatment of all pro­posals.

   (e) The Steering Committee makes rec­om­mendations to the selection official on the payload or program of investigations and notes caveats or provisions important for consideration of the selection official.

 

1872.407  Principles to apply.

   (a) 1872.406 contains a de­scription of the evaluation function appro­priate for a major payload or very signifi­cant pro­gram of investigation.  The levels of review, evaluation, and refinement described should be applied in those selec­tions where