
04-11
Procurement Notice
December 29, 2005
ADDITIONAL COMPETITIVE PROCEDURES FOR NASA
ANNOUNCEMENTS OF
Background: NASA’s Broad Agency
Announcement (BAA)/Announcement of Opportunity (AO) procurements are conducted as
public/private competitions meeting the spirit and intent of the guidelines
promulgated by the Competitive Sourcing element of the President’s Management
Agenda. The agency seeks to increase the
effectiveness of BAA/AO competitions and raise the efficiency level with which
the BAA/AO competitions are conducted.
ACQUISITIONS AFFECTED BY
CHANGES: All solicitations released
and contracts awarded as a result of the BAA/AO processes.
ACTION REQUIRED BY
CONTRACTING OFFICERS: Compliance with the revised guidance addressing
proposal preparation procedures as contained in the newly added section
1872.308, and adherence to the clarification language regarding requirements
for justifications for other than full and open competition, as contained in
the revisions to section 1872.502.
CLAUSE CHANGES: None
PARTS AFFECTED: Changes are made in Part 1872.
REPLACEMENT PAGES: You may
use the enclosed pages to replace 72:1, 72:2, 72:7 through 72:32 of the NFS.
TYPE OF RULE AND PUBLICATION DATE: Not Applicable. This change does not have a significant effect beyond the internal operating procedures of the Agency or have a significant cost or administrative impact on contractors or offerors and therefore does not require codification in the Code of Federal Regulations (CFR) or publication for public comment.
HEADQUARTERS CONTACT: Jamiel Commodore, Contract Management Division, (202) 358-0302, email: jamiel.c.commodore@nasa.gov.
Tom Luedtke
Assistant Administrator for Procurement
Enclosures
DISTRIBUTION:
PN List
PART 1872
ACQUISITIONS OF INVESTIGATIONS
TABLE OF
CONTENTS
1872.000 Scope
of part.
SUBPART 1872.1 THE INVESTIGATION ACQUISITION
SYSTEM
1872.101 General.
1872.102 Key
features of the system.
1872.103 Management
responsibilities.
SUBPART 1872.2 APPLICABILITY OF THE PROCESS
1872.201 General.
1872.202 Criteria
for determining applicability.
1872.203 Applicable
programs and activities.
1872.204 Approval.
SUBPART 1872.3 THE ANNOUNCEMENT OF
1872.301 General.
1872.302 Preparatory
effort.
1872.303 Responsibilities.
1872.304 Proposal
opportunity period.
1872.305 Guidelines
for announcement of opportunity.
1872.306 Announcement
of opportunity soliciting foreign participation.
1872.307 Guidelines
for proposal preparation.
1872.308
Proposals
submitted by NASA investigators.
SUBPART 1872.4 EVALUATION OF PROPOSALS
1872.401 General.
1872.402 Criteria
for evaluation.
1872.403 Methods
of evaluation.
1872.403-1 Advisory
subcommittee evaluation process.
1872.403-2 Contractor
evaluation process.
1872.403-3 Government
evaluation process.
1872.404 Engineering,
integration, and management evaluation.
1872.405 Program
office evaluation.
1872.406 Steering
committee review.
1872.407 Principles
to apply.
SUBPART 1872.5 THE SELECTION PROCESS
1872.501 General.
1872.502 Decisions
to be made.
1872.503 The
selection statement.
1872.504 Notification of
proposers.
1872.505 Debriefing.
SUBPART 1872.6 PAYLOAD FORMULATION
1872.601 Payload
formulation.
SUBPART 1872.7 ACQUISITION
AND OTHER CONSIDERATIONS
1872.701 Early
involvement essential.
1872.702 Negotiation,
discussions and contract award.
1872.703 Applications of
the Federal Acquisition Regulation (FAR) and the
NASA FAR Supplement.
1872.704 Other
administrative and functional requirements.
1872.705 Format
of announcement of opportunity.
1872.705-1 Appendix
A: General instructions and provisions.
1872.705-2 Appendix
B: Guidelines for proposal preparation.
1872.705-3 Appendix C: Glossary of terms and abbreviations
associated with
investigations.
PART 1872
ACQUISITIONS OF INVESTIGATIONS
1872.000 Scope of part.
This part
prescribes policies and procedures for the acquisition of investigations.
Subpart 1872.1--The
Investigation Acquisition System
1872.101 General.
The investigation
acquisition system encourages the participation of investigators and the selection
of investigations which contribute most effectively to the advancement of
NASA's scientific and technological objectives.
It is a system separate from the acquisition process, but requiring the
same management and discipline to assure compliance with statutory requirements
and considerations of equity.
1872.102 Key Features of the system.
(a)(1) Use of the system commences with the Enterprise Associate Administrator's determination that the investigation acquisition process is appropriate for a program. An Announcement of Opportunity (AO) is disseminated to the interested scientific and technical communities. The AO is a form of broad agency announcement (BAA) (see FAR 35.016 and 1835.016 for general BAA requirements). This solicitation does not specify the investigations to be proposed but solicits investigative ideas which contribute to broad objectives. In order to determine which of the proposals should be selected, a formal competitive evaluation process is utilized. The evaluation for merit is normally made by experts in the fields represented by the proposals. Care should be taken to avoid conflicts of interest. These evaluators may be from NASA, other Government agencies, universities, or the commercial sector. Along with or subsequent to the evaluation for merit, the other factors of the proposals, such as engineering, cost, and integration aspects, are reviewed by specialists in those areas. The evaluation
spread over the number of months most compatible
with the possible flight opportunities and the availability of resources
necessary to evaluate and fund the proposals.
(2)
The AO may be issued establishing a single proposal submission date. However, the AO could provide that NASA amend
the AO to provide for subsequent dates for submission of proposals, if additional
investigations are desired within the AO objectives.
(3)
The AO may provide for an initial submission date with the AO to remain open
for submission of additional proposals up to a final cutoff date. This final date should be related to the
availability of resources necessary to evaluate the continuous flow of
proposals, the time remaining prior to the flight opportunity(s) contemplated
by the AO, and payload funding and availability.
(b)
Generally, a core payload of investigations would be selected from the initial
submission of proposals under the above methods of open-ended AOs. These selections could be final or tentative
recognizing the need for further definition.
Proposals received by subsequent submission
dates would be considered in the scope of the
original AO but would be subject to the opportunities and resources remaining
available or the progress being made by prior selected investigations.
(c) Any
proposal, whether received on the initial submission or subsequent submission,
requires notification to the investigator and the investigator's institution
of the proposal disposition. Some of the proposals will be rejected completely
and the investigators immediately notified.
The remaining unselected proposals may, if agreeable with the proposers,
be held for later consideration and funding and the investigator so
notified. However, if an investigator's
proposal is considered at a later date, the investigator must be given an
opportunity to validate the proposal with the investigator's institution and
for updating the cost and other data contained in the original submission
prior to a final selection. In summary,
NASA may retain proposals, receiving
Category I, II, or III classifications (see 1872.403-1(e)), for possible later
sponsorship until no longer feasible to consider the proposal. When this final stage is reached, the
investigator must be promptly notified. Proposing investigators not desiring
their proposals be held for later consideration should be given the opportunity
to so indicate in their original submissions.
1872.305 Guidelines for announcement of opportunity.
(a) The AO
should be tailored to the particular needs of the contemplated investigations
and be complete in itself. Each AO will
identify the originating program office and be numbered consecutively by
calendar year, e.g., OA‑1-95, OA-2-95; OLMSA-1-95; OSS-1-95; etc. The required format and detailed instructions
regarding the contents of the AO are contained in 1872.705.
(b) The
General Instructions and Provisions (1872.705-1)
are necessary to accommodate the unique aspects of the AO process. Therefore, they must be appended to each AO.
(c) At the
time of issuance, copies of the AO must be furnished to Headquarters, Office of
Procurement (Code HS) and Office of General Counsel (Code GK).
(d)
Proposers should be informed of significant departures from scheduled dates for
activities related in the AO.
1872.306 Announcement of opportunity soliciting
foreign participation.
Foreign
proposals or
1872.307
Guidelines for proposal preparation.
While not all of the guidelines outlined in 1872.705-2 will be applicable in response to every AO, the investigator should be informed of the relevant information required. The proposal may be submitted on a form supplied by the Program Office. However, the proposal should be submitted in at least two sections: (a) Investigation and Technical Plan; and (b) Management and Cost Plan as described in 1872.705-2. Investigators shall be required to identify and discuss risk factors and issues throughout the proposal where they are relevant, and describe their approach to managing these risks.
1872.308
Proposals submitted by NASA investigators.
(a)
NASA solicits, accepts, and evaluates proposals submitted by NASA Centers in
response to an AO. A NASA investigator
may team with one or more non-Government co-investigators. A NASA investigator may also need to acquire
supplies, including instruments and other hardware, and non-research services
in support of the proposed investigation.
If a proposal submitted by a
(b)
In addition to complying with proposal preparation instructions contained in
the AO, proposals submitted by NASA Centers should address the following
matters.
(1)
Non-Government co-investigators.
(i)
The proposal should describe the open and competitive process that was used for
selecting proposed team members. While a
formal solicitation is not required, the process should include the following
competitive aspects: notice of the opportunity to participate to potential
sources, submissions from and/or discussions with potential sources, and
objective criteria for selecting team members among interested sources. If proposed team members are selected without
using an open and competitive process, the proposal should contain a full
justification consistent with the requirements of FAR Subpart 6.3.
(ii)
The proposal should also include a representation that the NASA investigator
has examined his/her financial interests and has determined that no personal
conflict of interest exists.
(2)
Supplies and support services.
(i)
The proposal should indicate that the supplies or services are available under
an existing Government contract; or
(ii)
The proposal should state that the supplies or services will be acquired under
a full and open competition; or
(iii)
The proposal should explain the basis of a justification for acquiring the
supplies or services noncompetitively (see FAR Subpart 6.3).
(c) A selection decision approving the
non-Government team members as selected co-investigators satisfies legal and
regulatory requirements without further competition or justification (see
1872.702).
(d)
For the acquisition of supplies, including hardware, and support services by
non-Government co-investigators, see 1872.502(a)(4).
1872.401 General.
(a) The
evaluation process considers the aspects of each proposal by the following
progressive sorting:
(1) A
review resulting in a categorization is performed by using one of the methods
or combination of the methods outlined in 1872.403. The purpose of this initial review is to
determine the scientific and/or technological merit of the proposals in the
context of the AO objectives.
(2)
Those proposals which are considered to have the greatest scientific or
technological merit are then reviewed in detail for the engineering,
management, and cost aspects, usually by the project office at the installation
responsible for the project.
(3)
Final reviews are performed by the program office and the steering committee
and are aimed at developing a group of investigations which represent an
integrated payload or a well-balanced program of investigation which has the
best possibility for meeting the AO’s objectives within programmatic
constraints.
(b) The
importance of considering the interrelationship of the several aspects of the
proposals to be reviewed in the process and the need for carefully planning
their treatment should not be overlooked.
An evaluation plan should be developed before issuance of the AO. It should cover the recommended staffing for
any subcommittee or contractor support, review guidelines as well as the
procedural flow and schedule of the evaluation.
While not mandatory, such a plan should be considered for each AO. A fuller discussion of the evaluation and
selection process is included in the following sections of this subpart.
1872.402 Criteria for evaluation.
(a) Each
AO must indicate those criteria which the evaluators will apply in evaluating
a proposal. The relative importance of
each criterion must also be stated. This
information will allow investigators to make informed judgments in formulating
proposals that best meet the stated objectives.
(b)
Following is a list of general evaluation criteria appropriate for inclusion
in most AOs:
(1)
The scientific, applications, and/or technological merit of the investigation.
(2)
The relevance of the proposed investigation to the AO's stated scientific,
applications, and/or technological objectives.
(3)
The competence and experience of the investigator and any investigative team.
(4)
Adequacy of whatever apparatus may be proposed with particular regard to its
ability to supply the data needed for the investigation.
(5)
The reputation and interest of the investigator's institution, as measured by
the willingness of the institution to provide the support necessary to ensure
that the investigation can be completed satisfactorily.
(6)
Cost and management aspects will be considered in all selections.
(7) The proposed approach to managing risk (e.g., level of technology
maturity being applied or developed, technical complexity, performance
specifications and tolerances, delivery schedule, etc.).
(8)
Other or additional criteria may be used, but the evaluation criteria must be
germane to the accomplishment of the stated objectives.
(c) Once
the AO is issued, it is essential that the evaluation criteria be applied in a
uniform manner. If it becomes apparent,
before the date set for receipt of proposals, that the criteria or their
relative importance should be changed, the AO will be amended, and all known
recipients will be informed of the change and given an adequate opportunity to
consider it in submission of their proposals.
Evaluation criteria and/or their relative importance will not be changed
after the date set for receipt of proposals.
1872.403
Methods of evaluation.
Alternative
methods are available to initiate the evaluation of proposals received in
response to an AO. These are referred to
as the Advisory Subcommittee Evaluation Process, the Contractor Evaluation
Process, and the Government Evaluation Process.
In all processes, a subcommittee of the appropriate Program Office
Steering Committee will be formed to categorize the proposals. Following categorization, those proposals
still in consideration will be processed to the selection official.
1872.403-1 Advisory subcommittee evaluation process.
(a)
Evaluation of scientific and/or technological merit of proposed investigations
is the responsibility of an advisory subcommittee of the Steering
Committee. The subcommittee constitutes
a peer group qualified to judge the scientific and technological aspects of all
investigation proposals. One or more
subcommittees may be established depending on the breadth of the technical or
scientific disciplines inherent in the AO's objectives. Each subcommittee represents a discipline or
grouping of closely related disciplines.
To maximize the quality of the subcommittee evaluation and
categorization, the following conditions of selection and appointment should
be considered.
(1)
The subcommittee normally should be established on an ad hoc basis.
(2)
Qualifications and acknowledgment of the professional abilities of the subcommittee
members are of primary importance.
Institutional affiliations are not sufficient qualifications.
(3) The
executive secretary of the subcommittee must be a full-time NASA employee.
(4)
Subcommittee members should normally be appointed as early as possible and
prior to receipt of proposals.
(5)
Care must be taken to avoid conflicts of interest. These include financial interests,
institutional affiliations, professional biases and associations, as well as
familiar relationships. Conflicts could
further occur as a result of imbalance between Government and non-Government
appointees or membership from institutions representing a singular school of
thought in discipline areas involving competitive theories in approach to an
investigation.
(6)
The subcommittee should convene as a group in closed sessions for proposal
evaluation to protect the proposer's proprietary ideas and to allow frank
discussion of the proposer's qualifications and the merit of the proposer's
ideas. Lead review responsibility for
each proposal may be assigned to members most qualified in the involved
discipline. It is important that each
proposal be considered by the entire subcommittee.
(b) It may
not be possible to select a subcommittee fully satisfying all of the conditions
described in paragraph (a) of this section.
It is the responsibility of the nominating and appointing officials to
make trade-offs, where necessary, among the criteria in paragraph (a) of this
section. This latitude permits flexibility in making decisions in accord with
circumstances of each application. In so
doing, however, it is emphasized that recognized expertise in evaluating
dissimilar proposals is essential to the continued workability of the investigation
acquisition process.
(c)
Candidate subcommittee members should be nominated by the office having
responsibility for the evaluation. Nominations
should be approved in accordance with NPD 1150.11, "Federal Advisory
Committee Act Committees." The
notification of appointment should specify the duration of assignment on the
subcommittee, provisions concerning conflicts of interest, and arrangements
regarding honoraria, per diem, and travel when actually employed.
(d) It is
important that members of the subcommittee be formally instructed as to their
responsibilities with respect to the investigation acquisition process, even
where several or all of the members have served previously. This briefing of subcommittee members should
include:
(1)
Instruction of subcommittee members on agency policies and procedures
pertinent to acquisition of investigations.
(2)
Review of the program goals, AO objectives, and evaluation criteria, including
relative importance, which provide the basis for evaluation.
(3)
Instruction on the use of preliminary proposal evaluation data furnished by
the Installation Project Office. The subcommittee
should examine these data to gain a better understanding of the proposed investigations,
any associated problems, and to consider cost in relation to the value of the
investigations' objectives.
(4)
Definition of responsibility of the subcommittee for evaluation and categorization
with respect to scientific and/or technical merit in accordance with the
evaluation criteria.
(5)
Instruction for documentation of deliberations and categorizations of the
subcommittee.
(6)
Inform the chairperson of the subcommittee and all members that they should
familiarize themselves with the provisions of the Standards of Ethical Conduct
for Employees of the Executive Branch, 5 CFR Part 2635, and the Supplemental
Standards of Ethical Conduct for employees of the National Aeronautics and
Space Administration, 5 CFR Part 6901,
regarding conflicts of interest.
Members should inform the appointing authority if their participation
presents a real or apparent conflict of interest situation. In addition, all participants should inform
the selection official in the event they are subjected to pressure or improper
contacts.
(7)
Inform members that prior to the selection and announcement of the successful
investigators and investigations, subcommittee members and NASA personnel
shall not reveal any information concerning the evaluation to anyone who is
not also participating in the same evaluation proceedings, and then only to
the extent that such information is required in connection with such
proceedings. Also, inform members that
subsequent to selection of an investigation and announcement of negotiations
with the investigator's institution, information concerning the proceedings of
the subcommittee and data developed by the subcommittee will be made available
to others within NASA only when the requestor demonstrates a need to know for
a NASA purpose. Such information will be made available to persons outside NASA
including other Government agencies, only when such disclosure is concurred in
by the Office of General Counsel. In
this connection, reference is made to 18 U.S.C. 1905 which provides criminal sanctions if any officer or employee
(including special employees) of the
(e) The
product of an advisory subcommittee is the classification of proposals into
four categories. The categories are:
(1) Category I--Well conceived and scientifically and technically sound investigations
pertinent to the goals of the program and the AO's objectives and offered by
a competent investigator from an institution capable of supplying the necessary
support to ensure that any essential flight hardware or other support can be
delivered on time and that data can be properly reduced, analyzed,
interpreted, and published in a reasonable time. Investigations in Category I are recommended
for acceptance and normally will be displaced only by other Category I
investigations.
(2) Category II--Well conceived and
scientifically or technically sound investigations which are recommended for
acceptance, but at a lower priority than Category I.
(3) Category III--Scientifically or
technically sound investigations which require further development. Category III investigations may be funded
for development and may be reconsidered at a later time for the same or other
opportunities.
(4) Category IV--Proposed investigations
which are recommended for rejection for the particular opportunity under consideration,
whatever the reason.
(f) A record of the deliberations of the
subcommittee shall be prepared by the assigned executive secretary and shall be
signed by the Chairperson. The minutes
shall contain the categorizations with basic rationale for such ratings and the
significant strengths and weaknesses of the proposals evaluated.
1872.403-2 Contractor evaluation process.
(a) The
use of the contractor method for obtaining support for evaluation purposes of
proposals received in response to an AO requires the approval of the Program
AA. Prior to the use of this method, discussion
should be held with the Office of Acquisition.
(b) It is
NASA policy to avoid situations in the acquisition process where, by virtue of
the work or services performed for NASA, or as a result of data acquired from
NASA or from other entities, a particular company:
(1) Is
given an unfair competitive advantage over other companies with respect to
future NASA business;
(2) Is
placed in a position to affect Government actions under circumstances in
which there is potential that the company's judgment may be biased; or
(3)
Otherwise finds that a conflict exists between the performance of work or
services for the Government in an impartial manner and the company's own
self-interest.
(c) To
reduce the possibility of an organizational conflict of interest problem
arising, the following minimum restrictions will be incorporated into the
contract:
(1) No
employee of the contractor will be permitted to propose in response to the AO;
(2)
The "Limitation on Future Contracting" clause contained in 1852.209-71 will be included in all such contracts; and
(3)
Unless authorized by the NASA contracting officer, the contractor shall not
contact the originator of any proposal concerning its contents.
(d) The
scope of work for the selected contractor will provide for an identification
of strengths and weaknesses and a
summary of the proposals. The
contractor will not make selections nor recommend investigations.
(e) The
steps to be taken in establishing evaluation panels and the responsibilities
of NASA and the contractor in relation to the panels will be as follows:
(1)
The contractor will be required to establish and provide support to panels of experts
for review of proposals to evaluate their scientific and technical merit;
(2)
These panels will be composed of scientists and specialists qualified to evaluate
the proposals;
(3)
The agency may provide to the contractor lists of scientist(s) and specialist(s)
in the various disciplines it believes are qualified to serve on the panels;
(4)
The contractor will report each panel's membership to NASA for approval; and
(5)
The contractor must make all the necessary arrangements with the panel members.
(f) The evaluation support by the contractor's
panels of experts will be accomplished as follows:
(1)
The panels will review the scientific and technical merit of the proposals in
accordance with the evaluation criteria in the AO and will record their
strengths and weaknesses.
(2)
The contractor will make records of each panel's deliberations which will form
the basis for a report summarizing the results of the evaluations. Upon request, the contractor shall provide
all such records to NASA;
(3)
The chairperson of each panel shall certify that the evaluation report
correctly represents the findings of the review panel; and
(4) A
final report will be submitted as provided in the contract.
(g) A
subcommittee of the Program Office Steering Committee will be established on
an ad hoc basis. Utilizing furnished
data, the subcommittee will classify the proposals into the four categories enumerated
in 1872.403-1(e)(1), Advisory Subcommittee Evaluation Process. A record of the deliberations of the
subcommittee should be prepared by an assigned executive secretary and signed
by the chairperson. The minutes should
contain the categorizations with the basic rationale for such ratings and the
significant strengths and weaknesses of the proposals evaluated.
1872.403-3 Government evaluation process.
(a) The Program AA
may appoint one or more full-time Government employees as subcommittee members
of the Program Office Steering Committee to evaluate and categorize the
proposals.
(b) Each
subcommittee member should be qualified and competent to evaluate the
proposals in accordance with the AO evaluation criteria. It is important that a subcommittee's
evaluation not be influenced by others either within or outside of NASA.
(c) The
subcommittee members will not contact the proposers for additional information.
(d) The
subcommittee members will classify the proposals in accordance with the four
categories indicated in 1872.403-1(e)(1).
Each categorization will be supported by an appropriate rationale
including a narrative of each proposal's strengths and weaknesses.
1872.404 Engineering, integration,
and management evaluation.
(a) The
subcommittee responsible for categorization of each proposal in terms of its scientific,
applications, or technical merit should receive information on probable cost,
technical status, developmental risk, integration and safety problems, and
management arrangements in time for their deliberations.
(b) This
information should be provided at the discretion of the Headquarters Program
Office by the Project Office at the installation. This information can be in general terms and
should reflect what insights the Project Office can provide without requesting
additional details from the proposers.
This limited Project Office review will not normally give the subcommittees
information of significant precision.
The purpose is to give the subcommittee sufficient information so it
can review the proposals in conjunction with available cost, integration, and
management considerations to gain an impression of each investigator's
understanding of the problems of the experiment and to permit gross trade-offs
of cost versus value of the investigation objective.
(c)
Following categorization, the Project Office shall evaluate proposals in contention,
in depth, including a thorough review of each proposal's engineering,
integration, management, and cost aspects.
This review should be accomplished by qualified engineering, cost, and
business analysts at the project center.
(d) In
assessing proposed costs, the evaluation must consider:
(1)
The investigation objective.
(2)
Comparable, similar or related investigations.
(3)
Whether NASA or the investigator should procure the necessary supporting
instrumentation or services and the relative cost of each mode.
(4)
Total overall or probable costs to the Government including integration and
data reduction and analysis. In the case
of investigations proposed by Government investigators, this includes all
associated direct and indirect cost.
With respect to cooperative investigations, integration, and other
applicable costs should be considered.
(e) The
Project Office, as part of the in-depth evaluation of proposals that require
instrumentation or support equipment, will survey all potential sources for
Government-owned instrumentation or support equipment that may be made available,
with or without modifications, to the potential investigator. Such items contributed by foreign
cooperating groups which are still available under cooperative project agreements
will also be considered for use under the terms and conditions specified in the
agreements. As part of the evaluation
report to the Program Office, the availability or nonavailability of
instrumentation or support equipment will be indicated.
(f) Proposals which require instrumentation
should be evaluated by project personnel.
This evaluation should cover the interfaces and the assessment of development
risks. This evaluation should furnish
the selection official with sufficient data to contribute to the instrument
determinations. Important among these
are:
(1)
Whether the instrument requires further definition;
(2)
Whether studies and designs are necessary to provide a reasonably accurate
appreciation of the cost;
(3)
Whether the investigation can be carried out without incurring undue cost,
schedule, or risk of failure penalties; and
(4)
Whether integration of the instrument is feasible.
(g) In
reviewing an investigator's management plan, the Project Office should
evaluate the investigator's approach for efficiently managing the work, the
recognition of essential management functions, and the effective overall
integration of these functions.
Evaluation of the proposals under final consideration should include,
but not be limited to: workload--present and future related to capacity and
capability; past experience; management approach and organization; e.g.:
(1)
With respect to workload and its relationship to capacity and capability, it
is important to ascertain the extent to which the investigator is capable of
providing facilities and personnel skills necessary to perform the required
effort on a timely basis. This review
should reveal the need for additional facilities or people, and provide some
indication of the Government support the investigator will require.
(2) A
review should be made of the investigator, the investigator's institution,
and any supporting contractor's performance on prior investigations. This should assist in arriving at an
assessment of the investigator and the institution's ability to perform the
effort within the proposed cost and time constraints.
(3)
The proposed investigator's management arrangements should be reviewed,
including make or buy choices, support of any co-investigator, and preselected
subcontractors or other instrument fabricators to determine whether such
arrangements are justified. The review
should determine if the proposed management arrangements enhance the
investigator's ability to devote more time to the proposed experiment objectives
and still effectively employ the technical and administrative support required
for a successful investigation. In
making these evaluations, the Project Office should draw on the
installation's engineering, business, legal, and other staff resources, as
necessary, as well as its scientific resources. If further information is needed from the
proposers, it should be obtained through the proper contacts.
1872.405 Program Office evaluation.
(a) A
Program Office responsible for the project or program at Headquarters will
receive the evaluation of the proposals, and weigh the evaluative data to
determine an optimum payload or program of investigation. This determination will involve recommendations
concerning individual investigations; but, more importantly, should result in
a payload or program which is judged to optimize total mission return within
schedule, engineering, and budgetary constraints. The recommendations should facilitate sound
selection decisions by the Program AA.
Three sets of recommendations result from the Program Office
evaluation:
(1)
Optimum payload or program of investigations, or options for alternative
payloads or programs.
(2)
Recommendation for final or tentative selection based on a determination of
the degree of uncertainty associated with individual investigations. A tentative selection may be considered step
one of a two-step selection technique.
(3)
Upon consideration of the guidelines contained in 1872.502(a)(3), recommending
responsibility for instrument development.
(b) The
Installation Project Office evaluation is principally concerned with ensuring
that the proposed investigation can be managed, developed, integrated, and
executed with an appropriate probability of technical success within the
estimated probable cost. The
Headquarters program Director, drawing upon these inputs, should be mainly
concerned with determining a payload or program from the point of view of programmatic
goals and budgetary constraints.
Discipline and cost trade-offs are considered at this level. The Headquarters Program Office should focus
on the potential contribution to program objectives that can be achieved under
alternative feasible payload integration options.
(c) It may
be to NASA's advantage to consider certain investigations for tentative
selection pending resolution of uncertainties in their development. Tentative selections should be reconsidered
after a period of time for final selection in a payload or program of
investigations. This two-step selection
process should be considered when:
(1)
The potential return from the investigation is sufficient, relative to that
of the other investigations under consideration, and that its further
development appears to be warranted before final selection.
(2)
The investigation potential is of such high priority to the program that the
investigation should be developed for flight if at all possible.
(3)
The investigative area is critical to the program and competitive approaches
need to be developed further to allow selection of the optimum course.
(d) Based
on evaluation of these considerations associated with the investigations
requiring further development of hardware, the following information should be
provided to the Steering Committee and the Program AA responsible for
selection:
(1)
The expected gain in potential return associated with the eventual incorporation
of tentatively recommended investigations in the payload(s) or program.
(2)
The expected costs required to develop instrumentation to the point of
"demonstrated capability."
(3)
The risk involved in added cost, probability of successfully developing the required
instrument capability, and the possibility of schedule impact.
(4)
Identification of opportunities, if any, for inclusion of such investigations
in later missions.
(e) In
those cases where investigations are tentatively selected, an explicit statement
should be made of the process to be followed in determining the final payload
or program of investigations and the proposers so informed. The two-phase selection approach provides
the opportunity for additional assurance of development potential and
probable cost prior to a final commitment to the investigation.
(f) As instruments used in investigations become
increasingly complex and costly, the need for greater control of their development
by the responsible Headquarters Program Office also grows. Accordingly, as
an integral part of the evaluation process, a deliberate decision should be
made regarding the role of the Principal Investigator with respect to the
provision of the major hardware associated with that person's
investigation. The guidelines for the
hardware acquisition determination are discussed in 1872.502(a)(3).
(g) The
range of options for responsibility for the instrumentation consists of:
(1) Assignment
of full responsibility to the Principal Investigator. The responsibility includes all in-house or
contracted activity to provide the instrumentation for integration.
(2)
Retention of developmental responsibility by the Government with participation
by the Principal Investigator in key events defined for the program. In all cases the right of the Principal
Investigator to counsel and recommend is paramount. Such involvement of the Principal
Investigator may include:
(i) Provision of instrument specifications.
(ii) Approval of specifications.
(iii) Independent monitorship of the development
and advice to the Government on optimization of the instrumentation for the
investigation.
(iv) Participation in design reviews and other
appropriate reviews.
(v) Review and concurrence in changes resulting
from design reviews.
(vi) Participation in configuration control
board actions.
(vii) Advice in definition of test program.
(viii)
Review and approval of test program and changes thereto.
(ix) Participation in conduct of the test program.
(x) Participation in calibration of instrument.
(xi) Participation in final inspection and
acceptance of the instrument.
(xii) Participation in subsequent test and
evaluation processes incident to integration and flight preparation.
(xiii)
Participation in the development and support of the operations plan.
(xiv)
Analysis and interpretation of data.
(h) The
Principal Investigator should as a minimum:
(1)
Approve the instrument specification.
(2)
Advise the project manager in development and fabrication.
(3)
Participate in final calibration.
(4)
Develop and support the operations plan.
(5)
Analyze and interpret the data.
(i) The Project Installation is responsible for
implementing the program or project and should make recommendations concerning
the role for the Principal Investigators.
The Program AA will determine the role, acting upon the advice of the
Headquarters Program Office and the Steering Committee. The Principal Investigator's desires will be
respected in the negotiation of the person's role allowing an appeal to the
Program AA and the right to withdraw from participation.
(j) The Program Office should make a presentation
to the Steering Committee with supporting documentation on the decisions to be
made by the responsible Program AA.
1872.406 Steering Committee review.
(a) The
most important role of the Steering Committee is to provide a substantive
review of a potential payload or program of investigations and to recommend a
selection to the Program AA. The
Steering Committee applies the collective experience of representatives from
the program and discipline communities and offers a forum for discussing the
selection from those points of view. In
addition to this mission-specific evaluation function, the Steering Committee
provides guidance to subcommittee chairpersons and serves as a clearinghouse
for problems and complaints regarding the process. The Steering Committee is responsible for
assuring adherence to required procedures.
Lastly, it is the forum where discipline objectives are weighed against
program objectives and constraints.
(b) The
Steering Committee represents the means for exercising three responsibilities
in the process of selecting investigations to:
(1)
Review compliance with procedures governing application of the AO process.
(2)
Ensure that adequate documentation has been made of the steps in the evaluation
process.
(3)
Review the results of the evaluation by the subcommittee, Project, and Program
Offices and prepare an assessment or endorsement of a recommended payload or
program of investigations to the Program AA.
(c) The
purpose in exercising the first of the responsibilities in paragraph (b) of
this section is to ensure equity and consistency in the application of the process. The Steering Committee is intended to
provide the necessary reviews and coordination inherent in conventional
acquisition practices.
(d) The
second and third responsibilities of the Steering Committee in paragraph (b)
are technical. They require that the
Steering Committee review the evaluations by subcommittee, the Project Office,
and the Program Office for completeness and appropriateness before forwarding
to the Program AA. Most important in
this review are:
(1)
Degree to which results of evaluations and recommendations follow logically
from the criteria in the AO.
(2)
Consistency with objectives and policies generally beyond the scope of
Project/Program Offices.
(3)
Sufficiency of reasons stated for tentative recommendations of those investigations
requiring further instrument research and development.
(4)
Sufficiency of reasons stated for determining responsibilities for instrument
development.
(5)
Sufficiency of consideration of reusable space flight hardware and support
equipment for the recommended investigations.
(6)
Sufficiency of reasons for classifying proposed investigations in their respective
categories.
(7)
Fair treatment of all proposals.
(e) The
Steering Committee makes recommendations to the selection official on the
payload or program of investigations and notes caveats or provisions important
for consideration of the selection official.
1872.407 Principles to apply.
(a) 1872.406 contains a description of the evaluation function appropriate for a major payload or very significant program of investigation. The levels of review, evaluation, and refinement described should be applied in those selections where