1. PURPOSE OF THE GUIDE
This guide
is intended to promote NASA-wide implementation of best practices for
debriefing offerors. It is also designed
to offer a set of uniform reference materials that include perspectives and
guiding principles in addition to debriefing practices and policies. The guide is intended to promote consistent
debriefing approaches within NASA while recognizing that the Federal
Acquisition Regulations (FAR) provide contracting officers with extensive
discretion in conducting debriefings.
Accordingly, this guide reflects the type of information that NASA generally
provides to offerors during debriefings but it does not mandate the manner in
which debriefings must be conducted. FAR
Part 15 reflects that debriefings may be done in any manner acceptable to the
contracting officer.
While
recognizing that contracting officers have extensive flexibility in conducting
debriefings, it is NASA’s operational philosophy that debriefings provide
offerors the maximum practical extent of information permitted by the FAR.
Offerors expend substantial sums of money and time to participate in the
acquisition process and deserve to receive a thorough and meaningful
debriefing.
This
Debriefing Guide is provided as a practical guide for conducting NASA
debriefings in accordance with FAR Part 15.5 and must be read in conjunction
with the FAR and the corresponding NASA FAR Supplement (NFS) 1815.5. This document does not expand, limit, modify,
or affect in any way the substantive or procedural requirements or rights
provided by statute, the FAR, or NFS.
2. DEBRIEFING DEFINITION
A
debriefing is a meeting with NASA personnel requested by an offeror, who
typically has been eliminated from the competition, either prior to or after
contract award. FAR Parts 15.505 and
15.506 provide debriefing guidance in accordance with the statutory
requirements at 10 U.S.C. 2305(b)(5) and (6).
The statutes and regulations prescribe the boundaries for information
that may be provided during pre and post award debriefings. This guide is designed to help NASA
contracting officers provide debriefings consistent with these
requirements. Additional guidance is
provided in NFS 1815.506-70 for Major Systems Acquisition Debriefings.
The FAR
envisions two types of debriefings, pre-award and post-award. NASA also routinely provides debriefings to
successful offerors upon their request.
Whichever the circumstance, debriefings shall facilitate:
Open,
appropriate and meaningful information exchanges that reduce misunderstandings
and protests;
A clear
understanding of NASA’s evaluation process and the basis for the selection
decision;
An
opportunity to show that NASA followed the rules and conducted the acquisition
in an objective and fair manner;
An
opportunity to positively affect the quality of offerors’ future proposals by
providing meaningful feedback for improvement in future acquisitions;
An
opportunity to obtain feedback from the debriefed contractors of their views of
the acquisition process for that particular procurement.
3. PRE-AWARD DEBRIEFING
A
pre-award debriefing is a meeting, prior to contract award, with NASA personnel
requested by an offeror who has been eliminated from the competition. Pre-award debriefing requirements are discussed
in FAR Part 15.505. Such a debriefing
may be conducted orally, in writing, or in any other method acceptable to the
contracting officer. The purposes of a pre-award debriefing are to:
Explain
NASA’s evaluation of significant elements in the offeror’s proposal.
Summarize
NASA’s rationale for eliminating the offeror from the competition; and
Reasonably
respond to offeror’s relevant questions about whether NASA personnel followed
the source selection procedures contained in the solicitation, applicable
regulations and other applicable authorities in the process of eliminating the
offer from the competition.
4. POST-AWARD DEBRIEFING
A post-award debriefing is a meeting,
after contract award, with NASA personnel requested by an offeror who typically
has been eliminated from the competition.
Post-award debriefing requirements are discussed in FAR Part 15.506. The purpose of a post-award debriefing is to
furnish the basis for the selection decision.
Post-award debriefings must include the information listed in FAR
15.506(d). Successful offerors may also
request post award debriefings.
5.
INFORMATION PROVIDED
A.
The information listed in paragraph C, below, is typically shared with
unsuccessful offerors in keeping with NASA’s policy of providing the maximum
practical information to debriefed offerors.
Prior to the debriefing, the debriefed offeror is provided with a meeting
agenda and a copy of the source selection statement that includes much of the
listed information. The agenda may be in
“bullet” format and should include the topics addressed in the debriefing (for
topics, see section 17 of this guide).
B.
One possible way to provide information at a debriefing is to provide a
copy of the relevant portions (properly redacted to preclude release of
proprietary information) of charts or materials that the Source Evaluation
Board (SEB) presented to the contracting officer (for pre-award debriefings) or
the Source Selection Authority (SSA)
(for post-award debriefings). The
information must be completely consistent with the information presented to the
contracting officer or the SSA regardless of the form or document used to
convey this information to the debriefed offeror.
C. INFORMATION PROVIDED DURING DEBRIEFINGS:
PRE-AWARD DEBRIEFING—
In addition to the information
listed for disclosure in FAR 15.505(c), NASA will provide the following, if
applicable:
All of the debriefed offeror’s
strengths, weaknesses and deficiencies
The debriefed offeror’s adjectival ratings for each
Mission Suitability subfactor
The debriefed offeror’s overall
Mission Suitability point
score
The debriefed offeror’s Past Performance
level of confidence rating
A summary of the debriefed offeror’s
performance that resulted in the assigned Past Performance rating
Significant probable cost
adjustments made for the debriefed offeror (explain reasons for adjustments and
quantify in approximate percentage increase or decrease).
The information listed in FAR
15.505(f) shall not be disclosed.
POST-AWARD
DEBRIEFING—
(1) In addition to the information listed for disclosure in
FAR 15.505(d) and the information listed above for disclosure in NASA Pre-award
Debriefings, NASA will provide the following:
Number of offerors
Identity of offerors
Identity of offerors in the
competitive range
The awardee’s adjectival ratings for each
Mission Suitability subfactor
The awardee’s overall Mission
Suitability point score
The awardee’s Past Performance level of confidence
rating
Probable cost comparison between the
awardee and debriefed offeror (expressed in approximate percentage difference)
Summary of
the impact of discussions and proposal revisions on the original evaluation
findings for the proposal, so that the offeror can gauge the success of its efforts
regarding any discussions that were held and proposal revisions that were
submitted
The information listed in FAR
15.506(e) and, for major systems acquisitions conducted in accordance with NFS
Part 1834, the information set forth in NFS1815.506-70(a), shall not be
disclosed.
(2) Point scores at the mission suitability
sub-factor level shall not be disclosed.
The release of point scores at the
sub-factor level may lead offerors to inappropriately focus on point scores as
discriminators, whereas the proper focus should be on the real discriminators
in the evaluation and selection decision, i.e., the government’s evaluation of
the strengths and weaknesses in the offeror’s proposal.
6. PRIMARY DEBRIEFING AUDIENCE
Debriefings
must be conducted when they are “required” by the FAR. The FAR prescribes required debriefings as
those properly requested within the timeframes specified in Parts 15.505 and
15.506. An offeror who complies with
those requirements is entitled to receive a debriefing. Accordingly, NASA must conduct a debriefing
for an offeror if: (1) the offeror makes
a written request for a debriefing, and (2) the request is received by the contracting
activity within 3 calendar days after the offeror received notice of exclusion
from the competition or contract award.
Time periods are computed in accordance with FAR 15.501.
7. SECONDARY DEBRIEFING AUDIENCE
A. Offerors Who Submit Late Debriefing
Requests. NASA generally provides
debriefings to offerors who submit untimely requests. However, such debriefings are not “required”
and considerations related to such non-required debriefings are discussed under
the heading “Timeliness Considerations,” below.
B. Successful Offerors. Awardees may request and receive a
debriefing.
C. Subcontractors. NASA does not provide subcontractors with
separate debriefings. However, with the
prime contractor’s consent, subcontractors may attend debriefings provided to
their prime contractors.
8. REQUIREMENTS
A. REQUIRED PRE-AWARD DEBRIEFINGS: See FAR 15.505(a) and (b).
B. REQUIRED
POST-AWARD DEBRIEFINGS: See FAR
15.506(a).
9. TIMELINESS CONSIDERATIONS
A. The scheduling of a debriefing may have
significant ramifications if a protest is later filed. It is important that the
contracting officer schedule the debriefing at the earliest possible date. The
contracting officer should inform the offeror of the scheduled date in writing
via electronic means with immediate acknowledgment requested or by other immediate
means, which provides positive indication of the time of actual receipt. If the
offeror is unable to attend the scheduled date and requests a later date, it is
important to maintain a copy of the written notice to the offeror of the date
that was initially offered as well as a copy of the offeror’s written request
for a later date.
B. If an offeror submits an untimely request for
debriefing (i.e., received more than 3 days after notice of exclusion), the contracting
officer should nonetheless conduct a debriefing if it is appropriate and
feasible. However, the contracting officer should inform the offeror that the
request is untimely, that the debriefing is not a “required” debriefing under
FAR 15.506(a), and that the debriefing is voluntarily provided by the Agency
solely as an accommodation to assist the offeror in improving future
proposals. See also FAR Part
33.104(c)(1) (Protests after award).
10. LOCATION
A. The Contracting officer is responsible for
selecting a suitable location for the debriefing that ensures a professional
presentation in a non-distracting environment.
A debriefing may be conducted orally or in writing. It may be
accomplished via telephone, video, fax, or a face-to-face meeting. However,
within NASA, SEB procedure debriefings are generally conducted in face-to-face
meetings. In some instances, especially
where the contracting officer is the SSA, it may be appropriate for NASA to
respond with a written debrief. When a
written debriefing is used, the contracting officer must ensure that all
parties are clearly aware of the conclusion of the debriefing (in the event the
contractor responds with written questions). The conclusion of the debriefing
process triggers protest time-lines.
B.
A conference room of appropriate size and appointment will normally
suffice. The contracting officer should consult with the offeror and arrange
for adequate seating. The contracting officer should verify whether any
classified material will be discussed and make appropriate arrangements as well
as determine the availability of a nearby room for the NASA personnel to use in
case a discussion or caucus is necessary.
C. Although face-to-face debriefings are usually
preferable, it may be appropriate to conduct debriefings by telephone or other
electronic means. Sometimes, it will be financially prohibitive for the offeror
to attend in person. The offeror’s needs should be afforded due consideration
when determining the method of conducting the debriefing.
D.
If some of the NASA personnel cannot be present, the contracting officer
may decide that those individuals' attendance is best accomplished by telephone
or video-conference. The possibilities are endless - make use of available
technology and what makes sense.
11. NASA PERSONNEL
The
NASA personnel at the debriefing should include the contracting officer and
other management, procurement, technical, and legal personnel as required to
appropriately debrief the particular offeror.
However, NASA personnel attendance should be kept to the minimum
necessary to provide a comprehensive debriefing.
Debriefing officials and other NASA personnel should employ
tact and diplomacy in conveying the findings. Caution should be taken to ensure
that the manner in which the information is conveyed facilitates open,
appropriate and meaningful information exchanges.
A.
Facilitator/Contracting Lead. The
FAR specifies that the contracting officer normally conduct the
debriefing. However, nothing requires
the contracting officer to personally conduct the debriefing. Best practices dictate that the
best-qualified individuals to conduct the debriefing are those adequately
involved with and knowledgeable of the procurement.
B. Contracting
Officer. Typically within NASA, the
contracting officer convenes the debriefing, facilitates introductions,
discusses the agenda for the debriefing, lays the ground rules and discusses
the solicitation and acquisition process.
Later portions of this guide suggest a format. However, the contracting officer may delegate
such functions to an adequately prepared and knowledgeable contract specialist
or other SEB member involved with most aspects of the procurement. As chair of the debriefing, the contracting
officer should ensure that such administrative portions of the debriefing are
competently and professionally presented.
C. Technical
Leads. Typically, the SEB chairperson or
other well-qualified members of the SEB present the discussion of mission
suitability or technical findings. The
designated individual should have a thorough understanding of the technical
evaluation and findings, the solicitation, the acquisition process and the
SSA’s considerations.
D. Legal
Counsel. Legal counsel should be
involved in debriefing preparation and typically should attend the
debriefing. If the acquisition is in
litigation or under protest, legal counsel must attend the debriefing. Absent extraordinary circumstances, legal
counsel should also attend if the offeror brings or involves legal
counsel. In other situations, it remains
a sound practice to have legal counsel attend the debriefing. Counsel may assist if difficult questions
arise regarding such issues as disclosure of sensitive information. However, be prepared for legal counsel to
take a passive role in the debriefing unless called upon to do otherwise.
E. Cost/Price
Analysts. Experience within NASA has
demonstrated that it may be best for a well-prepared contracting lead to cover cost
and pricing issues during the debriefing.
However, on procurements with complex pricing issues, it is advisable to
have the price analyst lead the Cost portion of discussions or be “on call” for
the debriefing team to caucus with regarding difficult pricing issues.
F. Other
“Specialists”. On rare occasions, when a
key item within the selection decision hinged on a particular specialist’s
input, the specialist may be invited to attend the debriefing or be “on call”
and readily available for a caucus.
G. Source Selection Authority. Best practices dictate that, unless the SSA
is the contracting officer, the SSA does not attend or participate in a
debriefing. However, NASA debriefing
participants shall be intimately familiar with the SSA decision rationale. In the rare occasions when a high-level SSA
desires to attend the debriefing, both procurement and legal personnel should
carefully prepare the SSA.
12. OFFEROR PERSONNEL
A.
Normally, no limitation is placed on the number of offeror personnel in
attendance at the debriefing. Nonetheless, space limitations of NASA facilities
may require restrictions on the number of offeror personnel invited to attend.
The contracting officer, however, should not impose restrictions unless suitable
alternate facilities are unavailable.
B.
The contracting officer should ensure that the offeror personnel consist
of corporate level individuals familiar with the offeror’s efforts in relation
to the procurement. The spectrum may
range from corporate Chief Executive Officers to Program and Project Managers,
Cost and Proposal Writing Individuals and key subcontractor personnel. It is imperative to request the list of
attendees, company affiliation and title prior to the debriefing so that
appropriate accommodations may be made.
13. SUCCESSFUL DEBRIEFING
A. Often, unsuccessful offerors are able to
accept negative findings in a debriefing if they perceive that NASA has acted with
fairness, consistency, objectivity and in accordance with the evaluation
criteria described in the solicitation.
However, the debriefing is unlikely to alter opinions and perceptions
that arise earlier in the process. Thus,
the Agency’s credibility and rapport with a debriefed offeror begins upon the
first contact with the offeror.
B. Items that serve as a foundation for
successful debriefings include:
·
Cordial
treatment of the offerors by procurement personnel
·
Procurement
personnel availability during the “pre-proposal” phase to allow industry to
become familiar with agency requirements
·
Development
of evaluation factors that help identify “key swingers” and that favor no
particular offeror
·
Development
of professional work products (e.g., the draft solicitation, the solicitation,
questions to offerors)
·
Meaningful
consideration of issues received during any pre-proposal conference
·
Establishment
of an “open door” policy prior to the communications blackout used to help
potential offerors become familiar with NASA’s requirements
·
Sincere
and apparent effort to obtain good competition
·
Fair
and unbiased approach employed and conveyed by procurement personnel
If
these steps are taken during the course of the procurement, the debriefing has
a good prospect of opening with a degree of rapport that can foster a
productive and successful meeting.
14. PREPARATION
A.
A successful debriefing
requires preparation. Experience has shown that going into a debriefing
unprepared is the surest way to lose the confidence of the offeror and lose the
opportunity to effectively communicate the agency position.
B. Preparation must begin prior to
notification to offerors of exclusion from the competitive range and/or before
announcement of contract award, because debriefings may occur promptly
following the establishment of the competitive range and/or after a contract is
awarded. The keys to a smooth debriefing
process include determining the timing of the announcements, the establishment
of the competitive range or the source selection and availability of personnel
who will attend. The SEB assists the contracting officer in preparing for and
conducting the debriefing. Accordingly,
the contracting officer must inform the evaluators that their duties include
assisting with debriefings at the time the evaluation board is formed.
C. The extent of preparation necessary varies
considerably with the complexity of the procurement. Sometimes, merely
preparing debriefing talking points is sufficient. Other times, dry run
rehearsals are necessary. Within NASA,
dry runs are routinely performed prior to complex or high-dollar value
procurement debriefings.
D. All NASA personnel attending the debriefing
must be briefed on their roles and expected demeanor during the debriefing.
Argumentative or defensive conduct should be forbidden, and NASA personnel
should be instructed to make their presentations in a positive manner.
15. LIMITATIONS
Debriefed
offerors often display persistent curiosity about the content of their
competitor’s proposals. Resist these
efforts. Statutory and regulatory rules strictly
limit the information that NASA may disclose regarding other offeror’s
proposals.
A. Among other things, FAR Part 15.505(f)(3)
precludes NASA from disclosing, directly or indirectly, the content of any
other proposal in a pre-award debriefing.
Likewise, the Procurement Integrity Act, 41 U.S.C. 423, implemented at
FAR Part 3.104-4, precludes NASA from releasing contractor bid or proposal
information, other than as permitted by law.
It also precludes release of source selection information including
information regarding the evaluation of offeror’s proposals. Similarly, the Trade Secrets Act, 18 U.S.C.
1905, generally precludes NASA personnel from disclosing proprietary
information received from an offeror.
B. In a post-award debriefing, the FAR permits
disclosure of the successful offeror’s top-level prices and the rationale for
award. However, FAR Part 15.506(e)
precludes NASA from disclosing information that is exempt from release under
the Freedom of Information Act including trade secrets, privileged or
confidential manufacturing information, confidential commercial and financial
information (including cost breakdowns and profit) and the names of individuals
providing past performance information.
The Trade Secrets Act also precludes NASA from releasing proprietary
information received from an offeror.
This guide permits maximum practical disclosure, during post-award
debriefings, of source selection information consistent with applicable legal limitations.
C. When offerors
inquire regarding the specifics of other proposals, NASA contracting officers
should remind them of the purposes of debriefings. Debriefings are not a forum in which to
compare and contrast successful and unsuccessful offers or to debate NASA
decisions. The General Accounting Office
has recognized that debriefing purposes are two fold: (1) to inform the offeror of its significant
weaknesses and deficiencies, and (2) to provide essential information in a
post-award debriefing on the rationale for the source selection decision.
16. DEBRIEFING MATERIALS
A. Debriefing material may include
briefing charts, if any were prepared in accordance with Center procedures, and
notes prepared for use during the debriefing.
NASA contracting officers should bring a copy of the source selection
statement and the solicitation, as well as a sign-in sheet to document
attendance.
B. NASA personnel shall NOT bring the
following materials into the debriefing room.
The presence of these documents can lead to the inadvertent disclosure
of proprietary information and prohibited point-by-point comparisons of
proposals.
·
Proposals
of other offerors, and
·
Proposal
evaluation reports of other offerors' proposals.
C.
Faulty memory or misstatements by SEB or acquisition personnel are
detrimental to a successful debriefing. Thus, contracting officers must ensure
that necessary notes or other documents are accessible during the debriefing.
17. A SUGGESTED DEBRIEFING FORMAT
A. Debriefing Official
The
contracting officer is responsible for the debriefing and must never lose
control of the debriefing. (Procurement
representatives at a level higher than the contracting officer with thorough
knowledge of the procurement may serve as the debriefing official in more
complex, high dollar value actions.)
Speak
with One Voice
It is
important for agency debriefing participants to have a unified
presentation. Disunity among the NASA
representatives may impair teamwork, consistency and, ultimately, credibility. Likewise, inconsistency with statements or
conclusions within the source selection document or other information that may
be available to the offeror can lower the offeror’s confidence that the agency
acted appropriately in making its decision.
Thus, the debriefing team must speak with one voice; a voice that is
consistent with the acquisition documents and decision.
B. Introduction and Objective
The debriefing should begin with a
brief introduction of all attendees. The
contracting officer should advise the offeror of the objective of the
debriefing (i.e., to advise it of the basis for the selection and to share the
strengths, weaknesses, and deficiencies of its proposal in furtherance of
helping it with future competitive endeavors.)
C. Ground Rules
Next,
the contracting officer should inform the offeror of the ground rules for the
debriefing, any time constraints, and the debriefing agenda.
Some
suggested ground rules include:
·
Establish
the appropriate time for raising questions
- Entertained at any time or
- Held until the end of the debriefing
·
State
NASA’s intent to share as much information as practical with the debriefed
offeror; however, the debriefing is not a forum for a debate.
·
NASA
will not engage in a point-by-point comparison between the debriefed offeror’s
proposal and the winning proposal.
·
NASA
will not divulge information from the selected offeror’s proposal.
·
NASA
will not indulge in hypothetical questions ( e.g., “if we had proposed this,
how would you have rated it?”)
D. Source Selection Process
The
contracting officer should briefly explain the process used in evaluating
proposals and, if a post-award debriefing, selecting the awardee. NASA normally
identifies the SSA by name. The identities of the evaluators, other than those
present at the debriefing, may also be disclosed at the discretion of the local
activity. NASA should convey to the
offeror that qualified personnel evaluated the proposals.
E. Evaluation Factors
The
evaluation factors, subfactors, weights of scored factors, and relative
importance of evaluation factors, as disclosed in the solicitation, should be
restated as the foundation for discussing the evaluation results. Share the adjective ratings and definitions
used in the evaluation.
F. Evaluation Results
Refer
to Section 5 of this guide, and FAR 15.505 for pre-award and 15.506 for
post-award, for the information that NASA generally provides during debriefings
and the information that may not be disclosed at pre-award and post-award
debriefings.
G.
Overall Ranking Of All Proposals (POST-AWARD)
Provide
the overall ranking of all proposals, if the proposals were ranked (i.e., the
best overall proposal, 2nd best, etc.).
H.
Make and Model of Commercial End Items (POST-AWARD)
Provide
the make and model of the item, if the awardee's proposal includes a commercial
item that is an end item under the contract.
I.
Rationale for Award Decision (POST-AWARD)
NASA
must disclose a summary of the rationale for the contract award decision. The rationale is contained in the source
selection statement. Contracting
officers should furnish the debriefed offeror a copy of the source selection
statement prior to the debriefing and be prepared to discuss it during the
debriefing.
J. Questions
The
debriefed offeror is permitted to ask relevant questions pertaining to whether
NASA personnel followed the source selection procedures set forth in the
solicitation, applicable regulations, and other applicable authorities. The
contracting officer must make every effort to provide reasonable responses to
those questions. However, caution must be exercised not to inadvertently
disclose proprietary information of other offerors or get into debating
hypothetical questions that may undermine the credibility of the selection.
Normally, responses should stay within the confines of the guidance stated
above. If the contracting officer is unprepared to answer a specific question
at the debriefing, the contracting officer should obtain the answer immediately
following the debriefing and promptly furnish it to the offeror.
The
debriefing is the forum for answering reasonable questions; offerors should not
be invited to submit questions after the debriefing.
K. Feedback
Prior
to concluding the debriefing, NASA contracting officers often solicit the
debriefed offerors comments regarding the solicitation and the procurement
process. This is an opportunity for NASA
to obtain valuable input regarding the offeror’s perspective.
L.
Conclusion.
The
contracting officer should ensure that the offeror understands that the
debriefing is officially concluded unless outstanding questions remain. If follow-up is necessary to answer
questions, the contracting officer should ensure that the offeror understands
that the debriefing is concluded as soon as the answers are provided.
19. THE DEBRIEFING MEMORANDUM
A. By statute and regulation, the contracting
officer must include a summary of each debriefing in the contract file. Good debriefing memorandums are essential
especially if the procurement is re-opened or re-solicited as a result of a
protest or otherwise within one year of the contract award date.
B. The debriefing memorandum should
include at a minimum:
·
A list of all attendees.
·
A
summary of the information disclosed during the debriefing. If charts were used
at the debriefing they may be attached as an efficient means to summarize
disclosed information a copy of them to the memorandum.
·
The
substance of all questions and answers discussed at the debriefing. Include answers provided after the
debriefing.
