Assurance Activities for Complex Electronics Preliminary Design

As the engineers develop the high-level design for complex electronics, the assurance engineer(s) works alongside to assess the processes and the product (preliminary design). These two activity streams are opposite sides of the same coin, and result in a verified design.

Use the Tailoring chart to determine which activities or analyses are required for a particular criticality classification. Activities that are not required may still be performed, if desired. The assurance activities for complex electronic preliminary design include:

• Preliminary Design Evaluation
• Preliminary Design Review (formal or informal)
• Simulation
• Process Verification
• Updating:
• Risk analysis
• Interface Analysis
• Traceability Analysis
• Fault Tree Analysis
• Failure Modes and Effects Analysis

Every method listed above does not have to be applied to every project. The table below uses the Complex Electronics Classification to map the activities, and depth of each activity, against the classification. This table allows for easy tailoring of the assurance activities to the device complexity and criticality.

Preliminary Design Evaluation

A key assurance activity for any project is expert evaluation and assessment of the design. For complex electronics, some portions of the design can be evaluated by an assurance engineer with sufficient understanding of complex electronics. Most assurance engineers will not be able to provide expert assessment, however. If a design is safety or mission critical, an independent engineer with experience in complex electronics should provide a detailed evaluation of the design.

For complex electronics, the questions to ask when evaluating the preliminary design are:

• Is the design complete (i.e., are all the requirements implemented)? Do all requirements trace to architectural blocks or functions?
• Are the requirements correctly implemented ? If the requirements were ambiguous, various people will have different interpretations of what the device should do.
• Does the design contain functions that are not required? If so, this may indicate missed requirements.
• Does the design contain any internal inconsistencies or conflicts?
• Does the design conflict with any requirements or with another system element?
• Does the design clearly identify constraints on other system elements, and on the installation and operation processes?
• Is the quality of the design (documentation and HDL model) adequate for its usage? Consider if the design is readable, understandable, and maintainable.
• Will the design result in a testable system? Consider both physical access (e.g., test pins) and the ability to test modules within the device.
• Are special pins (e.g., mode pin on FPGA, JTAG pins, no-connect pins) used correctly?
• If some requirements conflict (such as power consumption and performance speed), was a trade-off performed to determine the optimal levels for each requirement?
• Is the design flexible enough to accommodate anticipated requirements changes?
• Did the design follow the coding standard and design guidelines?

If COTS or re-used IP cores or modules are used within the complex electronics, they need to be evaluated for how they will operate within and interface to elements of the complex electronics. Areas to consider are:

• Functions within the IP module that will not be used.
• Interfaces (voltages, timing, signal characteristics, etc.)
• Verification performed by the vendor. What documentation exists for that verification? What additional testing needs to be performed?
• IP module upgrades. If the vendor indicates that there is a problem with the IP module, how will that problem (and fix) be evaluated for the complex electronics?

A preliminary design evaluation checklist can be found in the Checklist section.

Preliminary (Architectural) design review

A Design Review is a more formalized process that brings together individuals with various understandings of the system and the complex electronics. The goal is to have each person review the design (preliminary in this case) against their area of knowledge. By performing a wide review, problems related to interfaces, system operations, etc. are more likely to be found.

During a design review, the reviewers need to verify that:

• All requirements are addressed, including environmental, safety, and reliability
• Safety aspects of the design are explicitly identified
• Design can be implemented, tested, and maintained
• Any new manufacturing or other techniques are evaluated
• Design is traceable to requirements

Details of design reviews, including a discussion of reviewing HDL code, can be found in the Techniques section.

Simulation

The assurance engineer has two simulation tasks at the preliminary design phase: 1) evaluate the designer's simulations and 2) perform independent simulations. For complex electronics classified as low criticality, independent simulations are not necessary. The amount of independent simulation will depend on the CE classification and the functions the CE will perform.

The designer's simulations, including test benches, needs to be evaluated for correctness. Test benches are created by human beings, often by the designer, and are subject to faults and failings like any human endeavor. If the logic of the test bench is incorrect, or if a particular stimulus is not defined, then the end result of the tests may not show an actual error. You can't assume that the test bench accurately and completely tested the device - especially if the device will be used in a safety-critical application. For the simulations (tests) that the designer performed, consider:

• Does the simulation show that the architecture will meet the requirements?
• Does the simulation address all expected inputs and combinations of inputs?
• Does the simulation consider inputs that are out-of-range or otherwise invalid?
• Does the simulation include the effects of faults or failures in other parts of the system?

If an independent simulation is performed, the first question is: can I use the designer's test bench? For CE classified as moderate criticality, that is probably alright. Using an already developed test bench will save a lot of time and effort. Just be aware that there may be errors in the test bench, as well as in the complex electronics model under test. For high criticality devices, the test bench should be developed by the assurance engineer or the person who will perform independent simulation.

The questions asked above are valid for an independent simulation. You want to design tests that will show if the design does, or does not, function properly in all anticipated situations. Designers are success oriented, so you should focus on unexpected inputs, sequence of inputs, or simultaneous combinations of inputs. Simulations can focus on small modules or parts of modules, as well as on the complex electronics model as a whole.

Creating a simulation involves a thorough understanding of the HDL used in developing the test bench and the methods of testing complex electronics. It is usually much easier to learn to review a simulation than to create one from scratch. The level of expertise of the assurance engineer should not preclude independent simulation, if that is called for. An engineer familiar with simulations, but not part of the project, can be used to develop the test bench and test cases, and possibly to even execute the tests.

Process Verification

Process verification is an activity performed at each life cycle phase. The plans that were generated at the beginning of the project should be followed as the complex electronics is developed. Assuring that the processes are followed is one way to help the project stay on track. Processes are not written in stone, but can be changed to reflect project reality. One result of process verification is to advise the project when updating the processes is a necessary activity.

For the preliminary design phase, the assurance engineer will:

• Verify that the entrance criteria was met before the project moved into this phase. If not, document the increased risk due to non-approved requirements or other issues. Provide risk mitigation suggestions to the project.
• Verify that the exit criteria is met before the project moves to the Detailed Design phase. If not, document the increased risk due to non-approved requirements or other issues. Provide risk mitigation suggestions to the project.
• Check that the Design Description, together with the documentation of previous development phases, is complete and documented in a level of detail sufficient to proceed with the Detailed Design.
• Review selected tools for applicability to the design process. Check the tool vendor web-site and other sources for known tool defects or operational workarounds.
• Make sure a disciplined design process is in place, and the design engineer is willing to follow it. Negotiate as necessary.
• Check that the planned measures, tools, methods and procedures have been applied.
• Ensure that the complex electronics preliminary design is under configuration control. At the end of the phase, ensure that the preliminary design is approved and baselined.
• Ensure that the design team is using a design and coding standard. This standard will define the basic design philosophy and specify aspects of the HDL program structure. Even if only one engineer is designing the device, a standard 1) helps ensure that the HDL program is understandable by others (and the design engineer, six months down the road) and 2) provides a way to capture and incorporate best practices in the design process.
• Verify that all planned activities for complex electronics were performed.

Another process verification activity at the beginning of this phase is to review the design guidelines and coding standard (after ensuring that the project has a coding standard). A design and coding standard should include:

• Specific HDL coding features and methods that either should be used, or should not be used.
• Design “best practices”, either as a guideline or as requirements.
• Naming conventions for modules, inputs, outputs, etc.
• Commenting rules that define what types of information to include in comments. One example would be to define a module header that includes comments on the module's purpose and structure.
• Readability rules may be covered under naming and commenting conventions. But the standard should help guide the designer into creating HDL code that is readable by others.
• Modularization guidelines provide information on how to decompose the high level design into individual modules.

Update Analyses

Analyses performed during the requirements phase should be updated at this time.

Risk Analysis

Evaluate previous risks to identify those that no longer apply or that have changed their priority based on changes in probability or impact. Identify any new risks relevant to this phase of development and determine which require mitigation plans. Check that preventive measures and/or contingency plans exist for all identified risk items and that the risk, with mitigations in place, is acceptable for moving to the Detailed Design phase.

Design Interface Analysis

Are all interfaces identified, both internally and externally? Are they correct, consistent, complete, and accurate?

Traceability Analysis

Trace the requirements into the design elements and functions. Are all requirements implemented? Is there any additional functionality in the design that is not included in a requirement? Also, evaluate the tracing for accuracy, completeness, consistency, and correctness

Fault Tree Analysis

If a Fault Tree Analysis was developed, and it traced to the complex electronics, expand the fault tree to include the appropriate architectural blocks. Also review the FTA against the preliminary design and ensure that the design incorporates sufficient fault prevention.

Failure Modes and Effects Analysis

If a Failure Modes and Effects Analysis (FMEA) was developed, and it included complex electronics, look at the failure modes defined for the CE device. Does the preliminary design provide any additional possible failure modes? If so, add them to the analysis. Consider failures in interactions between the complex electronics and system elements it interfaces with, including noisy signals and invalid outputs.