CANCELLED BY PN 04-77
Procurement Information Circular
PURPOSE: To promote NASA wide implementation of best practices for debriefing offerors and provides consistent debriefing approaches within NASA.
BACKGROUND: The offeror debriefing process within NASA varies from Center to Center, not only in procedure, but also in the type and amount of information provided. In an attempt to develop an Agency-wide approach, a team of Headquarters and Center procurement and legal representatives reviewed and developed suggested improvements for the debriefing process. The result was the NASA Procurement Debriefing Guide.
GUIDANCE: The debriefing guide is designed to facilitate:
Ø Open, appropriate and meaningful information exchanges that reduce misunderstandings and protests
Ø An opportunity to show that NASA personnel conducted the acquisition in an objective and fair manner
Ø An opportunity to positively affect the quality of offerors’ future proposals by providing meaningful feedback for improvement
Ø An opportunity to obtain feedback from the debriefed contractors of their views of the acquisition process for that particular procurement
EFFECTIVE DATE: This PIC supersedes PIC 04-11 and shall remain in effect until cancelled or superseded.
HEADQUARTERS CONTACT: Marilyn J. Seppi, Contract Management Division, (202) 358-0447, Marilyn.firstname.lastname@example.org
James A. Balinskas
Director, Contract Management Division
A Practical Guide
Conducting NASA Debriefings
NASA PROCUREMENT DEBRIEFINGS GUIDE
TABLE OF CONTENTS
Purpose of the Guide……………………………………………………..….1
Primary Debriefing Audience……………………………………………….4
Secondary Debriefing Audience………………………………………….....4
A Suggested Debriefing Format………………………………………….....9
The Debriefing Memorandum………………………………………….…...12
1. PURPOSE OF THE GUIDE
This guide is intended to promote NASA-wide implementation of best practices for debriefing offerors. It is also designed to offer a set of uniform reference materials that include perspectives and guiding principles in addition to debriefing practices and policies. The guide is intended to promote consistent debriefing approaches within NASA while recognizing that the Federal Acquisition Regulations (FAR) provide contracting officers with extensive discretion in conducting debriefings. Accordingly, this guide reflects the type of information that NASA generally provides to offerors during debriefings but it does not mandate the manner in which debriefings must be conducted. FAR Part 15 reflects that debriefings may be done in any manner acceptable to the contracting officer.
While recognizing that contracting officers have extensive flexibility in conducting debriefings, it is NASA’s operational philosophy that debriefings provide offerors the maximum practical extent of information permitted by the FAR. Offerors expend substantial sums of money and time to participate in the acquisition process and deserve to receive a thorough and meaningful debriefing.
This Debriefing Guide is provided as a practical guide for conducting NASA debriefings in accordance with FAR Part 15.5 and must be read in conjunction with the FAR and the corresponding NASA FAR Supplement (NFS) 1815.5. This document does not expand, limit, modify, or affect in any way the substantive or procedural requirements or rights provided by statute, the FAR, or NFS.
2. DEBRIEFING DEFINITION
A debriefing is a meeting with NASA personnel requested by an offeror, who typically has been eliminated from the competition, either prior to or after contract award. FAR Parts 15.505 and 15.506 provide debriefing guidance in accordance with the statutory requirements at 10 U.S.C. 2305(b)(5) and (6). The statutes and regulations prescribe the boundaries for information that may be provided during pre and post award debriefings. This guide is designed to help NASA contracting officers provide debriefings consistent with these requirements. Additional guidance is provided in NFS 1815.506-70 for Major Systems Acquisition Debriefings.
The FAR envisions two types of debriefings, pre-award and post-award. NASA also routinely provides debriefings to successful offerors upon their request. Whichever the circumstance, debriefings shall facilitate:
Open, appropriate and meaningful information exchanges that reduce misunderstandings and protests;
A clear understanding of NASA’s evaluation process and the basis for the selection decision;
An opportunity to show that NASA followed the rules and conducted the acquisition in an objective and fair manner;
An opportunity to positively affect the quality of offerors’ future proposals by providing meaningful feedback for improvement in future acquisitions;
An opportunity to obtain feedback from the debriefed contractors of their views of the acquisition process for that particular procurement.
3. PRE-AWARD DEBRIEFING
A pre-award debriefing is a meeting, prior to contract award, with NASA personnel requested by an offeror who has been eliminated from the competition. Pre-award debriefing requirements are discussed in FAR Part 15.505. Such a debriefing may be conducted orally, in writing, or in any other method acceptable to the contracting officer. The purposes of a pre-award debriefing are to:
Explain NASA’s evaluation of significant elements in the offeror’s proposal.
Summarize NASA’s rationale for eliminating the offeror from the competition; and
Reasonably respond to offeror’s relevant questions about whether NASA personnel followed the source selection procedures contained in the solicitation, applicable regulations and other applicable authorities in the process of eliminating the offer from the competition.
4. POST-AWARD DEBRIEFING
A post-award debriefing is a meeting, after contract award, with NASA personnel requested by an offeror who typically has been eliminated from the competition. Post-award debriefing requirements are discussed in FAR Part 15.506. The purpose of a post-award debriefing is to furnish the basis for the selection decision. Post-award debriefings must include the information listed in FAR 15.506(d). Successful offerors may also request post award debriefings.
5. INFORMATION PROVIDED
A. The information listed in paragraph C, below, is typically shared with unsuccessful offerors in keeping with NASA’s policy of providing the maximum practical information to debriefed offerors. Prior to the debriefing, the debriefed offeror is provided with a meeting agenda and a copy of the source selection statement that includes much of the listed information. The agenda may be in “bullet” format and should include the topics addressed in the debriefing (for topics, see section 17 of this guide).
B. One possible way to provide information at a debriefing is to provide a copy of the relevant portions (properly redacted to preclude release of proprietary information) of charts or materials that the Source Evaluation Board (SEB) presented to the contracting officer (for pre-award debriefings) or the Source Selection Authority (SSA) (for post-award debriefings). The information must be completely consistent with the information presented to the contracting officer or the SSA regardless of the form or document used to convey this information to the debriefed offeror.
C. INFORMATION PROVIDED DURING DEBRIEFINGS:
In addition to the information listed for disclosure in FAR 15.505(c), NASA will provide the following, if applicable:
All of the debriefed offeror’s strengths, weaknesses and deficiencies
The debriefed offeror’s adjectival ratings for each Mission Suitability subfactor
The debriefed offeror’s overall Mission Suitability point score
The debriefed offeror’s Past Performance level of confidence rating
A summary of the debriefed offeror’s performance that resulted in the assigned Past Performance rating
Significant probable cost adjustments made for the debriefed offeror (explain reasons for adjustments and quantify in approximate percentage increase or decrease).
The information listed in FAR 15.505(f) shall not be disclosed.
(1) In addition to the information listed for disclosure in FAR 15.505(d) and the information listed above for disclosure in NASA Pre-award Debriefings, NASA will provide the following:
Number of offerors
Identity of offerors
Identity of offerors in the competitive range
The awardee’s adjectival ratings for each Mission Suitability subfactor
The awardee’s overall Mission Suitability point score
The awardee’s Past Performance level of confidence rating
Probable cost comparison between the awardee and debriefed offeror (expressed in approximate percentage difference)
Summary of the impact of discussions and proposal revisions on the original evaluation findings for the proposal, so that the offeror can gauge the success of its efforts regarding any discussions that were held and proposal revisions that were submitted
The information listed in FAR 15.506(e) and, for major systems acquisitions conducted in accordance with NFS Part 1834, the information set forth in NFS1815.506-70(a), shall not be disclosed.
(2) Point scores at the mission suitability sub-factor level shall not be disclosed. The release of point scores at the sub-factor level may lead offerors to inappropriately focus on point scores as discriminators, whereas the proper focus should be on the real discriminators in the evaluation and selection decision, i.e., the government’s evaluation of the strengths and weaknesses in the offeror’s proposal.
6. PRIMARY DEBRIEFING AUDIENCE
Debriefings must be conducted when they are “required” by the FAR. The FAR prescribes required debriefings as those properly requested within the timeframes specified in Parts 15.505 and 15.506. An offeror who complies with those requirements is entitled to receive a debriefing. Accordingly, NASA must conduct a debriefing for an offeror if: (1) the offeror makes a written request for a debriefing, and (2) the request is received by the contracting activity within 3 calendar days after the offeror received notice of exclusion from the competition or contract award. Time periods are computed in accordance with FAR 15.501.
7. SECONDARY DEBRIEFING AUDIENCE
A. Offerors Who Submit Late Debriefing Requests. NASA generally provides debriefings to offerors who submit untimely requests. However, such debriefings are not “required” and considerations related to such non-required debriefings are discussed under the heading “Timeliness Considerations,” below.
B. Successful Offerors. Awardees may request and receive a debriefing.
C. Subcontractors. NASA does not provide subcontractors with separate debriefings. However, with the prime contractor’s consent, subcontractors may attend debriefings provided to their prime contractors.
A. REQUIRED PRE-AWARD DEBRIEFINGS: See FAR 15.505(a) and (b).
B. REQUIRED POST-AWARD DEBRIEFINGS: See FAR 15.506(a).
9. TIMELINESS CONSIDERATIONS
A. The scheduling of a debriefing may have significant ramifications if a protest is later filed. It is important that the contracting officer schedule the debriefing at the earliest possible date. The contracting officer should inform the offeror of the scheduled date in writing via electronic means with immediate acknowledgment requested or by other immediate means, which provides positive indication of the time of actual receipt. If the offeror is unable to attend the scheduled date and requests a later date, it is important to maintain a copy of the written notice to the offeror of the date that was initially offered as well as a copy of the offeror’s written request for a later date.
B. If an offeror submits an untimely request for debriefing (i.e., received more than 3 days after notice of exclusion), the contracting officer should nonetheless conduct a debriefing if it is appropriate and feasible. However, the contracting officer should inform the offeror that the request is untimely, that the debriefing is not a “required” debriefing under FAR 15.506(a), and that the debriefing is voluntarily provided by the Agency solely as an accommodation to assist the offeror in improving future proposals. See also FAR Part 33.104(c)(1) (Protests after award).
A. The Contracting officer is responsible for selecting a suitable location for the debriefing that ensures a professional presentation in a non-distracting environment. A debriefing may be conducted orally or in writing. It may be accomplished via telephone, video, fax, or a face-to-face meeting. However, within NASA, SEB procedure debriefings are generally conducted in face-to-face meetings. In some instances, especially where the contracting officer is the SSA, it may be appropriate for NASA to respond with a written debrief. When a written debriefing is used, the contracting officer must ensure that all parties are clearly aware of the conclusion of the debriefing (in the event the contractor responds with written questions). The conclusion of the debriefing process triggers protest time-lines.
B. A conference room of appropriate size and appointment will normally suffice. The contracting officer should consult with the offeror and arrange for adequate seating. The contracting officer should verify whether any classified material will be discussed and make appropriate arrangements as well as determine the availability of a nearby room for the NASA personnel to use in case a discussion or caucus is necessary.
C. Although face-to-face debriefings are usually preferable, it may be appropriate to conduct debriefings by telephone or other electronic means. Sometimes, it will be financially prohibitive for the offeror to attend in person. The offeror’s needs should be afforded due consideration when determining the method of conducting the debriefing.
D. If some of the NASA personnel cannot be present, the contracting officer may decide that those individuals' attendance is best accomplished by telephone or video-conference. The possibilities are endless - make use of available technology and what makes sense.
11. NASA PERSONNEL
The NASA personnel at the debriefing should include the contracting officer and other management, procurement, technical, and legal personnel as required to appropriately debrief the particular offeror. However, NASA personnel attendance should be kept to the minimum necessary to provide a comprehensive debriefing.
Debriefing officials and other NASA personnel should employ tact and diplomacy in conveying the findings. Caution should be taken to ensure that the manner in which the information is conveyed facilitates open, appropriate and meaningful information exchanges.
A. Facilitator/Contracting Lead. The FAR specifies that the contracting officer normally conduct the debriefing. However, nothing requires the contracting officer to personally conduct the debriefing. Best practices dictate that the best-qualified individuals to conduct the debriefing are those adequately involved with and knowledgeable of the procurement.
B. Contracting Officer. Typically within NASA, the contracting officer convenes the debriefing, facilitates introductions, discusses the agenda for the debriefing, lays the ground rules and discusses the solicitation and acquisition process. Later portions of this guide suggest a format. However, the contracting officer may delegate such functions to an adequately prepared and knowledgeable contract specialist or other SEB member involved with most aspects of the procurement. As chair of the debriefing, the contracting officer should ensure that such administrative portions of the debriefing are competently and professionally presented.
C. Technical Leads. Typically, the SEB chairperson or other well-qualified members of the SEB present the discussion of mission suitability or technical findings. The designated individual should have a thorough understanding of the technical evaluation and findings, the solicitation, the acquisition process and the SSA’s considerations.
D. Legal Counsel. Legal counsel should be involved in debriefing preparation and typically should attend the debriefing. If the acquisition is in litigation or under protest, legal counsel must attend the debriefing. Absent extraordinary circumstances, legal counsel should also attend if the offeror brings or involves legal counsel. In other situations, it remains a sound practice to have legal counsel attend the debriefing. Counsel may assist if difficult questions arise regarding such issues as disclosure of sensitive information. However, be prepared for legal counsel to take a passive role in the debriefing unless called upon to do otherwise.
E. Cost/Price Analysts. Experience within NASA has demonstrated that it may be best for a well-prepared contracting lead to cover cost and pricing issues during the debriefing. However, on procurements with complex pricing issues, it is advisable to have the price analyst lead the Cost portion of discussions or be “on call” for the debriefing team to caucus with regarding difficult pricing issues.
F. Other “Specialists”. On rare occasions, when a key item within the selection decision hinged on a particular specialist’s input, the specialist may be invited to attend the debriefing or be “on call” and readily available for a caucus.
G. Source Selection Authority. Best practices dictate that, unless the SSA is the contracting officer, the SSA does not attend or participate in a debriefing. However, NASA debriefing participants shall be intimately familiar with the SSA decision rationale. In the rare occasions when a high-level SSA desires to attend the debriefing, both procurement and legal personnel should carefully prepare the SSA.
12. OFFEROR PERSONNEL
A. Normally, no limitation is placed on the number of offeror personnel in attendance at the debriefing. Nonetheless, space limitations of NASA facilities may require restrictions on the number of offeror personnel invited to attend. The contracting officer, however, should not impose restrictions unless suitable alternate facilities are unavailable.
B. The contracting officer should ensure that the offeror personnel consist of corporate level individuals familiar with the offeror’s efforts in relation to the procurement. The spectrum may range from corporate Chief Executive Officers to Program and Project Managers, Cost and Proposal Writing Individuals and key subcontractor personnel. It is imperative to request the list of attendees, company affiliation and title prior to the debriefing so that appropriate accommodations may be made.
13. SUCCESSFUL DEBRIEFING
A. Often, unsuccessful offerors are able to accept negative findings in a debriefing if they perceive that NASA has acted with fairness, consistency, objectivity and in accordance with the evaluation criteria described in the solicitation. However, the debriefing is unlikely to alter opinions and perceptions that arise earlier in the process. Thus, the Agency’s credibility and rapport with a debriefed offeror begins upon the first contact with the offeror.
B. Items that serve as a foundation for successful debriefings include:
· Cordial treatment of the offerors by procurement personnel
· Procurement personnel availability during the “pre-proposal” phase to allow industry to become familiar with agency requirements
· Development of evaluation factors that help identify “key swingers” and that favor no particular offeror
· Development of professional work products (e.g., the draft solicitation, the solicitation, questions to offerors)
· Meaningful consideration of issues received during any pre-proposal conference
· Establishment of an “open door” policy prior to the communications blackout used to help potential offerors become familiar with NASA’s requirements
· Sincere and apparent effort to obtain good competition
· Fair and unbiased approach employed and conveyed by procurement personnel
If these steps are taken during the course of the procurement, the debriefing has a good prospect of opening with a degree of rapport that can foster a productive and successful meeting.
A. A successful debriefing requires preparation. Experience has shown that going into a debriefing unprepared is the surest way to lose the confidence of the offeror and lose the opportunity to effectively communicate the agency position.
B. Preparation must begin prior to notification to offerors of exclusion from the competitive range and/or before announcement of contract award, because debriefings may occur promptly following the establishment of the competitive range and/or after a contract is awarded. The keys to a smooth debriefing process include determining the timing of the announcements, the establishment of the competitive range or the source selection and availability of personnel who will attend. The SEB assists the contracting officer in preparing for and conducting the debriefing. Accordingly, the contracting officer must inform the evaluators that their duties include assisting with debriefings at the time the evaluation board is formed.
C. The extent of preparation necessary varies considerably with the complexity of the procurement. Sometimes, merely preparing debriefing talking points is sufficient. Other times, dry run rehearsals are necessary. Within NASA, dry runs are routinely performed prior to complex or high-dollar value procurement debriefings.
D. All NASA personnel attending the debriefing must be briefed on their roles and expected demeanor during the debriefing. Argumentative or defensive conduct should be forbidden, and NASA personnel should be instructed to make their presentations in a positive manner.
Debriefed offerors often display persistent curiosity about the content of their competitor’s proposals. Resist these efforts. Statutory and regulatory rules strictly limit the information that NASA may disclose regarding other offeror’s proposals.
A. Among other things, FAR Part 15.505(f)(3) precludes NASA from disclosing, directly or indirectly, the content of any other proposal in a pre-award debriefing. Likewise, the Procurement Integrity Act, 41 U.S.C. 423, implemented at FAR Part 3.104-4, precludes NASA from releasing contractor bid or proposal information, other than as permitted by law. It also precludes release of source selection information including information regarding the evaluation of offeror’s proposals. Similarly, the Trade Secrets Act, 18 U.S.C. 1905, generally precludes NASA personnel from disclosing proprietary information received from an offeror.
B. In a post-award debriefing, the FAR permits disclosure of the successful offeror’s top-level prices and the rationale for award. However, FAR Part 15.506(e) precludes NASA from disclosing information that is exempt from release under the Freedom of Information Act including trade secrets, privileged or confidential manufacturing information, confidential commercial and financial information (including cost breakdowns and profit) and the names of individuals providing past performance information. The Trade Secrets Act also precludes NASA from releasing proprietary information received from an offeror. This guide permits maximum practical disclosure, during post-award debriefings, of source selection information consistent with applicable legal limitations.
C. When offerors inquire regarding the specifics of other proposals, NASA contracting officers should remind them of the purposes of debriefings. Debriefings are not a forum in which to compare and contrast successful and unsuccessful offers or to debate NASA decisions. The General Accounting Office has recognized that debriefing purposes are two fold: (1) to inform the offeror of its significant weaknesses and deficiencies, and (2) to provide essential information in a post-award debriefing on the rationale for the source selection decision.
16. DEBRIEFING MATERIALS
A. Debriefing material may include briefing charts, if any were prepared in accordance with Center procedures, and notes prepared for use during the debriefing. NASA contracting officers should bring a copy of the source selection statement and the solicitation, as well as a sign-in sheet to document attendance.
B. NASA personnel shall NOT bring the following materials into the debriefing room. The presence of these documents can lead to the inadvertent disclosure of proprietary information and prohibited point-by-point comparisons of proposals.
· Proposals of other offerors, and
· Proposal evaluation reports of other offerors' proposals.
C. Faulty memory or misstatements by SEB or acquisition personnel are detrimental to a successful debriefing. Thus, contracting officers must ensure that necessary notes or other documents are accessible during the debriefing.
17. A SUGGESTED DEBRIEFING FORMAT
A. Debriefing Official
The contracting officer is responsible for the debriefing and must never lose control of the debriefing. (Procurement representatives at a level higher than the contracting officer with thorough knowledge of the procurement may serve as the debriefing official in more complex, high dollar value actions.)
Speak with One Voice
It is important for agency debriefing participants to have a unified presentation. Disunity among the NASA representatives may impair teamwork, consistency and, ultimately, credibility. Likewise, inconsistency with statements or conclusions within the source selection document or other information that may be available to the offeror can lower the offeror’s confidence that the agency acted appropriately in making its decision. Thus, the debriefing team must speak with one voice; a voice that is consistent with the acquisition documents and decision.
B. Introduction and Objective
The debriefing should begin with a brief introduction of all attendees. The contracting officer should advise the offeror of the objective of the debriefing (i.e., to advise it of the basis for the selection and to share the strengths, weaknesses, and deficiencies of its proposal in furtherance of helping it with future competitive endeavors.)
C. Ground Rules
Next, the contracting officer should inform the offeror of the ground rules for the debriefing, any time constraints, and the debriefing agenda.
Some suggested ground rules include:
· Establish the appropriate time for raising questions
· State NASA’s intent to share as much information as practical with the debriefed offeror; however, the debriefing is not a forum for a debate.
· NASA will not engage in a point-by-point comparison between the debriefed offeror’s proposal and the winning proposal.
· NASA will not divulge information from the selected offeror’s proposal.
· NASA will not indulge in hypothetical questions ( e.g., “if we had proposed this, how would you have rated it?”)
D. Source Selection Process
The contracting officer should briefly explain the process used in evaluating proposals and, if a post-award debriefing, selecting the awardee. NASA normally identifies the SSA by name. The identities of the evaluators, other than those present at the debriefing, may also be disclosed at the discretion of the local activity. NASA should convey to the offeror that qualified personnel evaluated the proposals.
E. Evaluation Factors
The evaluation factors, subfactors, weights of scored factors, and relative importance of evaluation factors, as disclosed in the solicitation, should be restated as the foundation for discussing the evaluation results. Share the adjective ratings and definitions used in the evaluation.
F. Evaluation Results
Refer to Section 5 of this guide, and FAR 15.505 for pre-award and 15.506 for post-award, for the information that NASA generally provides during debriefings and the information that may not be disclosed at pre-award and post-award debriefings.
G. Overall Ranking Of All Proposals (POST-AWARD)
Provide the overall ranking of all proposals, if the proposals were ranked (i.e., the best overall proposal, 2nd best, etc.).
H. Make and Model of Commercial End Items (POST-AWARD)
Provide the make and model of the item, if the awardee's proposal includes a commercial item that is an end item under the contract.
I. Rationale for Award Decision (POST-AWARD)
NASA must disclose a summary of the rationale for the contract award decision. The rationale is contained in the source selection statement. Contracting officers should furnish the debriefed offeror a copy of the source selection statement prior to the debriefing and be prepared to discuss it during the debriefing.
The debriefed offeror is permitted to ask relevant questions pertaining to whether NASA personnel followed the source selection procedures set forth in the solicitation, applicable regulations, and other applicable authorities. The contracting officer must make every effort to provide reasonable responses to those questions. However, caution must be exercised not to inadvertently disclose proprietary information of other offerors or get into debating hypothetical questions that may undermine the credibility of the selection. Normally, responses should stay within the confines of the guidance stated above. If the contracting officer is unprepared to answer a specific question at the debriefing, the contracting officer should obtain the answer immediately following the debriefing and promptly furnish it to the offeror.
The debriefing is the forum for answering reasonable questions; offerors should not be invited to submit questions after the debriefing.
Prior to concluding the debriefing, NASA contracting officers often solicit the debriefed offerors comments regarding the solicitation and the procurement process. This is an opportunity for NASA to obtain valuable input regarding the offeror’s perspective.
The contracting officer should ensure that the offeror understands that the debriefing is officially concluded unless outstanding questions remain. If follow-up is necessary to answer questions, the contracting officer should ensure that the offeror understands that the debriefing is concluded as soon as the answers are provided.
19. THE DEBRIEFING MEMORANDUM
A. By statute and regulation, the contracting officer must include a summary of each debriefing in the contract file. Good debriefing memorandums are essential especially if the procurement is re-opened or re-solicited as a result of a protest or otherwise within one year of the contract award date.
B. The debriefing memorandum should include at a minimum:
· A list of all attendees.
· A summary of the information disclosed during the debriefing. If charts were used at the debriefing they may be attached as an efficient means to summarize disclosed information a copy of them to the memorandum.
· The substance of all questions and answers discussed at the debriefing. Include answers provided after the debriefing.