NASA Logo

Complex Electronics Metrics

Metrics are defined as: " A system of parameters or ways of quantitative and periodic assessment of a process that is to be measured and are usually specialized by the subject area". Metrics can be used to track trends, problems, productivity and much more. With CE, metrics are difficult as they must cover both the hardware and software portions of the development cycle. There are two types of metrics available for use. They are:

Primitive (elemental)
Derived (complex)

Metrics are simply measurements. We use them all the time to make decisions and determine the "best deal". Primitives are the base item we use to make a decision. These can be items such as time, number of problems, or lines of code. A derived metrics takes multiple primitives to determine a unit. A good example from the software world is errors per lines of code (errors per KLoc). The two primitives used in this measure is the number of errors and the number of lines of code.

Complex Electronics Primitives (Elementals)

Type of defects
- Interface
- Data type
- Incorrect logic
- Requirements
Where found
- Requirements Review
- Code Review
- Test
- Inspection
Number of design changes
Before or after burn to hardware
Test coverage
Number of gates
Number of functions
Number of modules
Number of IPs (Intellectual Properties)
Lines of HDL (Hardware Description Language)
Number of Decision Points

Complex Electronics Derived metrics (Complex)

The process of measuring the quality of complex electronics involves more then just gathering primitives. Most times this information is combined to produce specific information needed by the CE project. This combination of combining primitives produces the derived metrics used by most development groups to get the feel for where the project is heading. What follows is a list of derived metrics you should find useful in measuring of quality of the CE device during the development process. These are:

Reliability
Estimated Number of Faults Remaining
Test Coverage
Data Flow Complexity
Specification Completeness

Other more simple derived metrics need no real explanation. These include:

Errors per thousand lines of code (Errors Per KLoc)
Errors per development phase
Modules per chip
IPs per chip
Gates used versus free space

Other various items could also be used to measure progress and quality. The project will determine the items that will be measured and should insure that these items are listed in the Complex Electronics Assurance Plan.

Reliability

This derived metric gives you a reliability number based on either the sample size or number of runs done in a given time period. The following primitives are needed for this derived metric.

Nr = Number of runs in a given sample

Nt = Number of runs for a given period of time

Nc = Number of error free runs in a given sample

St = Total sample space (Inputs and Outputs)

Ss = Sample space selected

Derived metric of:

Rf = Reliability factor in percent

For a Sample Space

Rf = (Nr / Nc) x 100

or

Rf = (Nt / Nc) x 100

For the Total Design

Rf = (Nr / Nc) x (Ss / St) x 100

or

Rf = (Nt / Nc) x (Ss / St) x 100

Estimated Number of Faults Remaining by Seeding

This derived metric gives you an estimated number of faults remaining based on testing results. These results are obtained by seeding the part with known faults. You then run your test bed against the seeded part and count how many of the faults were found by the tests. The following primitives are needed for this derived metric.

Ns = number of seeded faults

Sf = number of seeded faults found

Nf = number of unseeded faults found

The faults remaining (FR) in a design can be approximated using:

FR = (Nf x Ns) / Sf

Test Coverage in Percent

This derived metric gives you a test coverage number based on the part functions and number of I/O. The following primitives are needed for this derived metric.

If = Implemented Functions

Rf = Required Functions

Iu = I/O primitives used

It = Total number of I/O on design

Derived metric of:

TC = Test Coverage

TC = (If / Rf) x ( Iu / It) x 100

Data Flow Complexity (DFC)

This derived metric lets you measure the complexity of the design. The following primitives are needed for this derived metric.

Lfi = local flows into the procedure

Datain = number of data structures from which the procedure retrieves data

Lfo = local flows out of the procedure