Anyone who has had any significant contact with software development has been involved in projects that have had problems. It's in the very nature of software to have bugs. On the other hand, defining software quality has proved elusive.

We all know what we mean by software quality; the trouble is that we all mean something slightly different. One company might talk about the number of defects found per function point -- but how do they define a function point? And how are the defects found? Another company might use a different measure entirely or use the same terms to describe completely different methodologies.

[ Keep up with app dev issues and trends with InfoWorld's Fatal Exception and Strategic Developer. ]

I discussed these issues with Dr. Bill Curtis, director of the new Consortium for IT Software Quality (CISQ), and co-author of the Capability Maturity Model (CMM) framework. According to Curtis, there is currently no accepted standard for internal software quality, and there is a crying need for one now that outsourcers and other software development organizations are seeing service-level agreements that include measures of reliability, scalability, security, and other aspects of quality.

We're not talking about functional or unit testing here -- we're talking about the sorts of problems that occur when large, complicated, multitier, multilanguage systems are integrated. Perhaps one component is written in Java and another component is written in C#: What happens when they pass each other data structures? Do they share all their assumptions? What happens when a value is null? What happens when one component passes a very long buffer to the other component? What happens when one component encounters an error condition that could affect another component?

We're not talking about a process here, so much as firm definitions of metrics. CMMI is a set of standards for software development processes; CISQ will develop a set of standards for measuring software quality and defining quality metrics.

CISQ was born of two organizations: the Software Engineering Institute (SEI) at Carnegie Mellon University and the Object Management Group (OMG). According to Curtis, OMG is good at creating standards, while SEI is good at bringing executives to the table and getting people to use standards.

The inaugural meeting of CISQ in the United States will be at the SEI Office in Arlington, Va., in October; a second meeting in November will be at the SEI Office in Frankfurt, Germany. CISQ expects to have a draft IT Software Quality Standard in the fourth quarter of 2010, and initial licensing of software quality evaluation service providers in the fourth quarter of 2011.

More information can be found at the CISQ Web site.