Bonnie: An Open Source Clinical Quality Measure Testing Tool

Bonnie is a new open source software tool that MITRE has developed and released in April 2014 that allows Meaningful Use (MU) Clinical Quality Measure (CQM) developers to test and verify the behavior of their CQM logic.  The goal of Bonnie is to reduce the number of defects in CQMs by providing a robust and automated testing framework. Bonnie allows measure developers to independently load measures that they have constructed using the Measure Authoring Tool (MAT). Loading the measures into Bonnie converts the measures from their Extensible Markup Language (XML) eSpecifications into executable artifacts and measure metadata.

Bonnie Dashboard Page

The measure eSpecification format that Bonnie loads is Health Quality Measure Format (HQMF) XML. The HQMF specification provides the metadata and logic that describe the specifics of calculating a CQM. Bonnie can load the HQMF describing a measure and programmatically convert the HQMF specification into an executable format that allows calculating the measure directly from the specification. 

The measure metadata loaded into Bonnie is then used to allow developers to rapidly build a synthetic patient test deck for the measure using the clinical elements defined during the measure construction process. By using measure metadata as a basis for building synthetic patients, developers can rapidly and efficiently create a test deck for a measure. 

Once a CQM has been loaded into Bonnie, a user can inspect the measure logic and then build synthetic test records and set expectations on how those test records will calculate against a measure. This capability to build synthetic test patient records, set expectations against those records, and calculate the measures using those patient records provides an automated and efficient testing framework for CQMs. 

Using the Bonnie-supported CQM testing framework allows measure developers to more clearly understand the behavior of the measure logic, validate that the measure logic encodes their intent, and allows for multiple iterations of measure updates to be validated against a test deck. 

Bonnie Measure Page

Additionally, the development of a test deck as part of measure development provides benefits after the measures are finalized. The test deck build as part of measure development can be used to demonstrate the intent of the measure though the use of patient examples included in the test deck. Furthermore, the test deck provides systems that implement the measures with a means to validate the development of their systems. This is provided in the form of a base set of synthetic patient records with known expectations for calculating against the implemented measures. Finally, the test deck could be used as a basis for the test deck used as part of the Meaningful Use certification program. 

Bonnie has been designed to integrate with the nationally recognized data standards used by the Meaningful Use program for expressing CQM logic for machine-to-machine interoperability. This provides enormous value to the CQM program and federal policy leaders and stakeholders: this software tool verifies that the new and evolving standards for the Meaningful Use CQM program are tractable and can be implemented in software.   

Additionally, Bonnie was designed to provide an intuitive and easy-to-use interface based on feedback from the broader measure developer community. A key goal of the Bonnie application is to deliver a user experience that provides an efficient and intuitive method for constructing synthetic patient records for testing and validating CQMs. 

The Bonnie software is freely available via an Apache 2.0 open source license. The Meaningful Use program makes all or parts of the Bonnie software available for inspection, verification, and even reuse by other government programs or federal contractors. 

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2014.

Example of CDA limitations to interoperability: time intervals

The HL7 Clinical Document Architecture (CDA) is an XML-based markup standard intended to specify the encoding, structure and semantics of clinical documents for exchange. CDA is an ANSI-certified standard from Health Level Seven (HL7).  The CDA is highlighted as a flexible framework that can contain any type of clinical content.  Additionally, the details of the encoding of clinical data and associated aspects of that data are intentionally designed to be flexible.

That flexibility provides freedoms in various different systems' ability to export clinical data.  However, that same flexibility is an increasing barrier to the interoperability of data as systems need to import that same data.  My biggest pet peeve with the CDA and this problem is the flexibility that the CDA provides in the encoding of time intervals.

Based on clinical reason, the CDA provides the freedom to encode time intervals in eight (8) (VIII) different representations.

<low>
<width>
<high>
<low> <width>
<low> <high>
<center>
<center> <width>

This amount of flexibility in expressing something as simple as a time interval is an obstacle for any receiving system hoping to import and parse an HL7 CDA-based XML document without knowing the way that the generating system is going to express something as basic as a time interval. This permissive nature of the CDA's artifacts is common beyond this one basic example.

What is needed, and hopefully addressed in the emerging FHIR specification, is a more constrained approach to the foundational aspects of clinical data, such as how to encode time intervals.  To reach a point with more interoperability of healthcare data, analysis is needed of the presence of types of structured clinical data concepts and associated clinical codes used operationally.

I feel that the healthcare standards community ultimately needs to identify a strict and simple constrained set of ways of expressing clinical concepts that healthcare Standards Development Organizations (SDOs) like HL7 should use to constrain existing permissive and complex standards.  This could also be done to guide a stricter and simpler implementation to support interoperability via FHIR.

This could introduce significant and radical improvements in the interoperability of patient data in the US healthcare industry.  This will better enable disparate healthcare software systems to work together without requiring point-to-point coordination.  This could reduce, and eventually eliminate, these problems of point-to-point coordination that result in islands of automation.

This "loose coupler" approach will encourage HL7, or possibly new healthcare SDOs, to embrace a core set of strict and simple required attributes, over the current state of the practice using permissive and complex attributes.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2014.