Benchmarking

Benchmarking helps IT managers monitor trends in application quality and compare metrics against a baseline. The standards developed by CISQ for software size and software structural quality provide a common basis for benchmarking software quality across systems, technologies, vendors and companies. The metrics are used to develop baselines and thresholds that are specific to technology, industry vertical, application type, and other demographic characteristics.

 

CISQ’s quality characteristic measures for Security, Reliability, Maintainability and Performance Efficiency are displayed in a benchmark report as density metrics. The density metric is comprised of violations per Function Point or violations per 1,000 Lines of Code (KLOC).  Software quality may also be reported in Sigma level for the analysis of large systems. Sigma levels are determined by the number of times a software flaw is introduced out of the number of opportunities to introduce such a flaw. As an example, the number of empty catch blocks out of all the catch blocks in the program.

 

 

Benchmarking Example

 

Quality characteristic Your application Peer group average Delta
Reliability 4.46σ 4.83σ -7.7%
Security 4.65σ 4.56σ 1.97%
Performance Efficiency 3.98σ 4.23σ -5.9%
Maintainability 4.89σ 4.78σ 2.3%