Example QRDA Category 1 XML for Meaningful Use Stage 2 Clinical Quality Measure Reporting

Last fall, I wrote about the QRDA Category 1 XML Standard as a way of expressing inputs to Clinical Quality Measures for the Meaningful Use Stage 2 program.  More recently, I have become immersed in the QRDA Category 1 while leading the open source Cypress project for the Office of the National Coordinator of Health Information Technology (ONC) and the Meaningful Use Stage 2 CQM Testing and Certification Program.

Under ONC, the Meaningful Use Program tests and certifies EHR technologies as part of the Meaningful Use Program via Authorized Testing Laboratories (ATLs), who are the organizations that use the tools and policies identified by ONC.  The Cypress project is the authoritative testing and certification tool used for the MU Stage 2 Clinical Quality Measures, and includes testing for the QRDA Category 1 XML specification.

Based on all the work our team has been supporting with the QRDA Category 1 XML, I thought it would be helpful to demonstrate how a CQM can define the data that would need to be expressed in a QRDA Category 1 XML file.

As an example, I am selecting NQF 0018 "Controlling High Blood Pressure", which measures the percentage of patients, aged 18-85 years old at the time of the CQM's measurement, and who had some sequence of encounters with a provider, and a diagnosis of hypertension, and whose blood pressure was adequately controlled (<140/90mmHg) during the measurement period.

If you wanted to see the full logic of this particular CQM, the visualization that is automatically generated from the open source popHealth project that I am leading is below:

Denominator Logic

individual characteristicbirth date

>= 18 years starts before start of measure period

and

individual characteristicbirth date

<= 85 years starts before start of measure period

and

diagnosis condition problemessential hypertension

<= 6 months starts after start of measure period

or

diagnosis condition problemessential hypertension

starts before start of measure period

and

encounteroffice visit

during measure period

or

encounterface-to-face interaction

during measure period

or

encounterpreventive care services - established office visit, 18 and up

during measure period

or

encounterpreventive care services-initial office visit, 18 and up

during measure period

or

encounterhome healthcare services

during measure period

or

encounterannual wellness visit

during measure period

Numerator Logic

physical examdiastolic blood pressure < 90 mm Hg

duringrecent of

encounteroffice visit

during measure period

or

encounteroutpatient consultation

during measure period

or

encounterpreventive care services-initial office visit, 18 and up

during measure period

or

encounterpreventive care services - established office visit, 18 and up

during measure period

or

encounterface-to-face interaction

during measure period

or

encounterhome healthcare services

during measure period

or

encounterannual wellness visit

during measure period

and

physical examsystolic blood pressure < 130 mm Hg

duringrecent of

encounteroffice visit

during measure period

or

encounteroutpatient consultation

during measure period

or

encounterpreventive care services-initial office visit, 18 and up

during measure period

or

encounterpreventive care services - established office visit, 18 and up

during measure period

or

encounterface-to-face interaction

during measure period

or

encounterhome healthcare services

during measure period

or

encounterannual wellness visit

during measure period

Consider some realistic but minimal health data applied against that particular CQM and let's see what what a QRDA Category 1 XML file for me would look like.  Assume I have the following notional clinical data assigned to me in an Electronic Health Record system:

First Name: John
Last Name: Doe
DoB: June 24, 1975
Address: 123 Main Street Gardner, MA 01440
Work phone: 781-271-7102
HL7 Gender: Male
Spoken Language: English
CDC Race: White
CDC Ethnicity: Not Hispanic or Latino
Conditions: Hypertension diagnosed on March 1st, 2012
Encounters: Office Visit on March 1st, 2012
            Office Visit on July 1st, 2012
            Blood Pressure Visit on November 1st, 2012
Systolic BP: 127 mmHg on March 1st, 2012
             123 mmHg on November 1st, 2012
Diastolic BP: 79 mmHg on March 1st, 2012
              81 mmHg on November 1st, 2012

Now, lets see how that clinical data for this patient should be expressed in the (very verbose) QRDA Category 1 XML format.  As an FYI for how the QRDA Category 1 XML specification works, the data expressed in the QRDA Category 1 XML is in response to a CQM request.

Assume that the QRDA Category 1 XML below were generate for that particular CQM NQF 0018 "Controlling High Blood Pressure" for the Meaningful Use Stage 2 program.  Also, assume a reporting period ending on December 31st, 2012 23:59.  I am highlighting in yellow within the XML below where the clinical data that I enumerated for our notional patient is expressed in the QRDA Category 1 XML:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="cda.xsl"?>
<ClinicalDocument xmlns="urn:hl7-org:v3" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:voc="urn:hl7-org:v3/voc"
  xmlns:sdtc="urn:hl7-org:sdtc">
  <realmCode code="US"/>

  <typeId root="2.16.840.1.113883.1.3" extension="POCD_HD000040"/>

  <templateId root="2.16.840.1.113883.10.20.22.1.1"/>
  <templateId root="2.16.840.1.113883.10.20.24.1.1"/>
  <templateId root="2.16.840.1.113883.10.20.24.1.2"/>
  <id root="5b010313-eff2-432c-9909-6193d8416fac"/>
  <code code="55182-0"
   codeSystem="2.16.840.1.113883.6.1" 
   codeSystemName="LOINC" 
   displayName="Quality Measure Report"/>
  <title>QRDA Report for John Doe</title>
  <effectiveTime value="20130427145038"/>
  <confidentialityCode code="N"
    codeSystem="2.16.840.1.113883.5.25"/>
  <languageCode code="eng"/>
  <recordTarget>
    <patientRole>
      <id extension="12345" root="2.16.840.1.113883.4.572"/>
      <addr use="HP">
        <streetAddressLine>123 Main Street</streetAddressLine>
        <city>Gardner</city>
        <state>MA</state>
        <postalCode>01440</postalCode>
        <country>US</country>
      </addr>
      <telecom use="WP" value="tel:+1-781-271-7102"/>
      <patient>
        <name>
          <given>John</given>
          <family>Doe</family>
        </name>
        <administrativeGenderCode code="M" 
          codeSystem="2.16.840.1.113883.5.1" 
          codeSystemName="HL7 AdministrativeGender"/>
        <birthTime value="19750624120000"/>
        <raceCode code="2106-3"
          displayName="White" 
          codeSystemName="CDC Race and Ethnicity" 
          codeSystem="2.16.840.1.113883.6.238"/>
        <ethnicGroupCode code="2186-5" 
          displayName="Not Hispanic or Latino" 
          codeSystemName="CDC Race and Ethnicity" 
          codeSystem="2.16.840.1.113883.6.238"/>
        <languageCommunication>
          <templateId root="2.16.840.1.113883.3.88.11.83.2" 
            assigningAuthorityName="HITSP/C83"/>
          <templateId root="1.3.6.1.4.1.19376.1.5.3.1.2.1" 
            assigningAuthorityName="IHE/PCC"/>
          <languageCode code="eng"/>
        </languageCommunication>
      </patient>
    </patientRole>
  </recordTarget>
  <author>
    <time value="20130427145038"/>
    <assignedAuthor>
      <id extension="FakeNPI" root="2.16.840.1.113883.4.6"/>
      <addr>
        <streetAddressLine>234 Main Street</streetAddressLine>
        <city>Gardner</city>
        <state>MA</state>
        <postalCode>01440</postalCode>
        <country>US</country>
      </addr>
      <telecom use="WP" value="tel:(781)271-7102"/>
      <assignedAuthoringDevice>
        <manufacturerModelName>AcmeEHR</manufacturerModelName>
        <softwareName>AcmeEHR</softwareName>
      </assignedAuthoringDevice>
    </assignedAuthor>
  </author>
  <custodian>
    <assignedCustodian>
      <representedCustodianOrganization>
        <id root="2.16.840.1.113883.19.5"/>
        <name>Fake Custodian</name>
        <telecom use="WP" value="tel:(781)555-5555"/>
        <addr>
          <streetAddressLine>345 Main Street</streetAddressLine>
          <city>Gardner</city>
          <state>MA</state>
          <postalCode>01440</postalCode>
          <country>US</country>
        </addr>
      </representedCustodianOrganization>
    </assignedCustodian>
  </custodian>
  <legalAuthenticator>
    <time value="20130427145038"/>
    <signatureCode code="S"/>
    <assignedEntity>
      <id root="bc01a5d1-3a34-4286-82cc-43eb04c972a7"/>
      <addr>
        <streetAddressLine>567 Main Street</streetAddressLine>
        <city>Gardner</city>
        <state>MA</state>
        <postalCode>01440</postalCode>
        <country>US</country>
      </addr>
      <telecom use="WP" value="tel:(781)271-3000"/>
      <assignedPerson>
        <name>
          <given>Mike</given>
          <family>Doe</family>
        </name>
      </assignedPerson>
      <representedOrganization>
        <id root="2.16.840.1.113883.19.5"/>
        <name>AcmeEHR</name>
      </representedOrganization>
    </assignedEntity>
  </legalAuthenticator>
  <documentationOf typeCode="DOC">
    <serviceEvent classCode="PCPR">
      <effectiveTime>
        <low value="20100601"/>
        <high value="20100915"/>
      </effectiveTime>
      <performer typeCode="PRF">
        <time>
          <low value="20020716"/>
          <high value="20070915"/>
        </time>
        <assignedEntity>
          <id root="2.16.840.1.113883.4.6" extension="111111111"/>
          <representedOrganization>
            <id root="2.16.840.1.113883.4.2" extension="1234567"/>
            <id root="2.16.840.1.113883.4.336" extension="54321"/>
          </representedOrganization>
        </assignedEntity>
      </performer>
    </serviceEvent>
  </documentationOf>
  <component>
    <structuredBody>
      <component>
        <section>
          <templateId root="2.16.840.1.113883.10.20.24.2.2"/>
          <templateId root="2.16.840.1.113883.10.20.24.2.3"/>
          <code code="55186-1" codeSystem="2.16.840.1.113883.6.1"/>
          <title>Measure Section</title>
          <text>
            <table border="1" width="100%">
              <thead>
                <tr>
                  <th>eMeasure Title</th>
                  <th>Version neutral identifier</th>
                  <th>eMeasure Version Number</th>
                  <th>NQF eMeasure Number</th>
                  <th>Version specific identifier</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Controlling High Blood Pressure</td>
                  <td>ABDC37CC-BAC6-4156-9B91-D1BE2C8B7268</td>
                  <td>1</td>
                  <td>5177f5798538a2f952caf324</td>
                  <td>8A4D92B2-397A-48D2-0139-C6208B875109</td>
                  <td/>
                </tr>
              </tbody>
            </table>
          </text>
          <entry>
            <organizer classCode="CLUSTER" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.24.3.98"/>
              <!-- This is the templateId for eMeasure Reference QDM -->
              <templateId root="2.16.840.1.113883.10.20.24.3.97"/>
              <statusCode code="completed"/>
              <reference typeCode="REFR">
                <externalDocument classCode="DOC" moodCode="EVN">
                  <id root="8A4D92B2-397A-48D2-0139-C6208B875109"/>
                  <text>Controlling High Blood Pressure</text>
                  <setId root="ABDC37CC-BAC6-4156-9B91-D1BE2C8B7268"/>
                  <versionNumber value="1"/>
                </externalDocument>
              </reference>
            </organizer>
          </entry>
        </section>
      </component>
      <component>
        <section>
          <templateId root="2.16.840.1.113883.10.20.17.2.1"/>
          <code code="55187-9" codeSystem="2.16.840.1.113883.6.1"/>
          <title>Reporting Parameters</title>
          <text>
            <list>
              <item>Reporting period: January 1st, 2012 - 
                    December 31st, 2012</item>
            </list>
          </text>
          <entry typeCode="DRIV">
            <act classCode="ACT" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.17.3.8"/>
              <code code="252116004" 
                codeSystem="2.16.840.1.113883.6.96"
                displayName="Observation Parameters"/>
              <effectiveTime>
                <low value="20120101000000"/>
                <high value="20121231235900"/>
              </effectiveTime>
            </act>
          </entry>
        </section>
      </component>
      <component>
        <section>
          <templateId root="2.16.840.1.113883.10.20.17.2.4"/>
          <templateId root="2.16.840.1.113883.10.20.24.2.1"/>
          <code code="55188-7" codeSystem="2.16.840.1.113883.6.1"/>
          <title>Patient Data</title>
          <text/>
          <entry>
            <observation classCode="OBS" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.2"/>
              <templateId root="2.16.840.1.113883.10.20.24.3.57"/>
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce61000287"/>
              <code code="8462-4" 
                codeSystem="2.16.840.1.113883.6.1" 
                sdtc:valueSet="2.16.840.1.113883.3.526.3.1033">
                <originalText>Physical Exam, Finding:
                              Diastolic BP</originalText>
              </code>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20120301120000"/>
                <high value="20120301120000"/>
              </effectiveTime>
              <value xsi:type="PQ" value="79" unit="mmHg"/>
            </observation>
          </entry>
          <entry>
            <observation classCode="OBS" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.2"/>
              <templateId root="2.16.840.1.113883.10.20.24.3.57"/>
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce6100028d"/>
              <code code="8462-4"
                codeSystem="2.16.840.1.113883.6.1"
                sdtc:valueSet="2.16.840.1.113883.3.526.3.1033">
                <originalText>Physical Exam, Finding:
                              Diastolic BP</originalText>
              </code>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20121101120000"/>
                <high value="20121101120000"/>
              </effectiveTime>
              <value xsi:type="PQ" value="81" unit="mmHg"/>
            </observation>
          </entry>
          <entry>
            <encounter classCode="ENC" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.49"/>
              <templateId root="2.16.840.1.113883.10.20.24.3.23"/>
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce6100027f"/>
              <code code="99201"
                codeSystem="2.16.840.1.113883.6.12" 
               sdtc:valueSet="2.16.840.1.113883.3.464.1003.101.12.1001">
                <originalText>Encounter, Performed:
                              Office Visit</originalText>
              </code>
              <text>Encounter, Performed:
                    Office Visit</text>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20120301120000"/>
                <high value="20120301130000"/>
              </effectiveTime>
            </encounter>
          </entry>
          <entry>
            <encounter classCode="ENC" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.49"/>
              <templateId root="2.16.840.1.113883.10.20.24.3.23"/>
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce61000280"/>
              <code code="99201"
                codeSystem="2.16.840.1.113883.6.12"
                   sdtc:valueSet="2.16.840.1.113883.3.464.1003.101.12.1001">
                <originalText>Encounter, Performed:
                              Office Visit</originalText>
              </code>
              <text>Encounter, Performed:
                    Office Visit</text>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20120701120000"/>
                <high value="20120701130000"/>
              </effectiveTime>
            </encounter>
          </entry>
          <entry>
            <encounter classCode="ENC" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.49" />
              <templateId root="2.16.840.1.113883.10.20.24.3.23" />
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce61000281"/>
              <code code="185349003"
                codeSystem="2.16.840.1.113883.6.96"
                sdtc:valueSet="2.16.840.1.113883.3.464.1003.101.12.1001">
                <originalText>Encounter, Performed:
                              Blood Pressure Visit</originalText>
                <translation code="99202" codeSystem="2.16.840.1.113883.6.12"/>
              </code>
              <text>Encounter, Performed:
                    Blood Pressure Visit</text>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20121101120000"/>
                <high value="20121101130000"/>
              </effectiveTime>
            </encounter>
          </entry>
          <entry>
            <encounter classCode="ENC" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.49" />
              <templateId root="2.16.840.1.113883.10.20.24.3.23" />
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce61000281" />
              <code code="185349003"
                codeSystem="2.16.840.1.113883.6.96"
              sdtc:valueSet="2.16.840.1.113883.3.464.1003.101.12.1048">
                <originalText>Encounter, Performed: 
                              Blood Pressure Visit</originalText>
                <translation code="99202"
                  codeSystem="2.16.840.1.113883.6.12"/>
              </code>
              <text>Encounter, Performed:
                    Blood Pressure Visit</text>
              <statusCode code="completed" />
              <effectiveTime>
                <low value="20121101120000" />
                <high value="20121101130000" />
              </effectiveTime>
            </encounter>
          </entry>
          <entry>
            <observation classCode="OBS" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.2" />
              <templateId root="2.16.840.1.113883.10.20.24.3.57" />
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce61000285" />
              <code code="8480-6" 
                codeSystem="2.16.840.1.113883.6.1" 
                sdtc:valueSet="2.16.840.1.113883.3.526.3.1032">
                <originalText>Physical Exam, Finding:
                              Systolic Blood Pressure</originalText>
              </code>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20120301120500"/>
                <high value="20120301120500"/>
              </effectiveTime>
              <value xsi:type="PQ" value="127" unit="mmHg"/>
            </observation>
          </entry>
          <entry>
            <observation classCode="OBS" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.2" />
              <templateId root="2.16.840.1.113883.10.20.24.3.57" />
              <id root="1.3.6.1.4.1.115" extension="5177f5807938a7ce6100028b" />
              <code code="8480-6"
                codeSystem="2.16.840.1.113883.6.1"
                sdtc:valueSet="2.16.840.1.113883.3.526.3.1032">
                <originalText>Physical Exam, Finding:
                              Systolic Blood Pressure</originalText>
              </code>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20121101120000"/>
                <high value="20121101120000"/>
              </effectiveTime>
              <value xsi:type="PQ" value="123" unit="mmHg"/>
            </observation>
          </entry>
          <entry>
            <observation classCode="OBS" moodCode="EVN">
              <templateId root="2.16.840.1.113883.10.20.22.4.4" />
              <templateId root="2.16.840.1.113883.10.20.24.3.11" />
              <id root="1.3.6.1.4.1.115" 
                extension="5177f5807938a7ce6100027e"/>
              <code code="282291009"
                displayName="diagnosis"
                codeSystem="2.16.840.1.113883.6.96"
                codeSystemName="SNOMED-CT"/>
              <text>Diagnosis, Active: Hypertension</text>
              <statusCode code="completed"/>
              <effectiveTime>
                <low value="20120301123000"/>
                <high nullFlavor="UNK"/>
              </effectiveTime>
              <value code="10725009" 
                codeSystem="2.16.840.1.113883.6.96" 
                xsi:type="CD"
                sdtc:valueSet="2.16.840.1.113883.3.464.1003.104.12.1011">
                <originalText>Diagnosis, Active: Hypertension</originalText>
                <translation code="401.1"
                  codeSystem="2.16.840.1.113883.6.103"/>
              </value>
              <entryRelationship typeCode="REFR">
                <observation classCode="OBS" moodCode="EVN">
                  <templateId root="2.16.840.1.113883.10.20.22.4.6"/>
                  <templateId root="2.16.840.1.113883.10.20.24.3.94"/>
                  <id root="da6b50c0-9177-0130-01b7-12313d02bdec" />
                  <code code="33999-4" 
                    codeSystem="2.16.840.1.113883.6.1" 
                    codeSystemName="LOINC" displayName="status" />
                  <statusCode code="completed"/>
                  <value xsi:type="CD"
                    code="55561003"
                    displayName="active"
                    codeSystem="2.16.840.1.113883.6.96"
                    codeSystemName="SNOMED CT"/>
                </observation>
              </entryRelationship>
            </observation>
          </entry>
        </section>
      </component>
    </structuredBody>
  </component>
</ClinicalDocument>

If your first impression that this is bloated XML... you are not alone.

The primary cause of this "bloat" really traces its routes back to the HL7 Clinical Document Architecture and the HL7 RIM for expressing effectively anything (from individual patient records to Clinical Quality Measure procedural logic!) in XML.  However, as much as I hate HL7 XML-based documents, there is a little bit of goodness with respect to the QRDA Category 1 XML for expressing patient-level data as inputs to Clinical Quality Measures.

Several HL7 standards have proven to demonstrate failures as inputs to Clinical Quality Measures; the HL7 Continuity of Care Document (CCD), the HITSP C32, or the Consolidated CDA.  These are all general XML standards meant for expressing patient-level data for continuity of care from one provider to another via an XML format.

The silver lining on the QRDA Category 1 cloud is that once the clinical data that is needed for a particular CQM has been identified in an EHR system, the way to express that clinical data on a patient-by-patient basis is clear(er) and more tractable for systems that will need to parse and interoperate with the structured data.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2013.

Complexity Analysis of MU Stage 2 Eligible Professional Clinical Quality Measures

I recently wrote about the initial release of the MITRE open source Kamira project to assess the quality of Clinical Quality Measures (CQMs).  The initial focus of the Kamira project has been the Cyclomatic Complexity analysis of the CQM logic.

The Kamira project has listened to my evangelizing the use of Kiviat charts as the visualization technique used with CQMs.  In addition to being fairly useful for understanding the CQM complexity results, I have found that the Kiviat visualizations also have a "sexiness" factor that I have been leveraging to continue the research.  See the Kamira Cyclomatic Complexity dashboard design below:

Kamira complexity dashboard design

What we did with the Kiviat visualization for the complexity results is leverage the five attributes that could always be used for parts of the CQM algorithmic logic:

• Initial Patient Population (IPP)
• Denominator 
• Numerator 
• Exclusion 
• Exception

We then associated ranges for the complexity metrics based on the Carnegie Mellon University paper that I identified a few months ago.  These ranges are:

• 1-10: Very Simple, Low Risk (green)
• 11-20: Nominal, Moderate Risk (yellow)
• 21-50: Complex, High Risk (orange)
• >50: Untestable, Extreme Risk (red)

Since we wanted to highlight the worst/highest component of the CQM, we opted to color in the area of the five metrics with the worst/highest section of the algorithmic logic.

Because some of the MU Stage 2 CQM logical sections far exceeded CMU's threshold of 50 for "Untestable, Extreme Risk" we found that there were some CQMs that had logical sections that were "off the scale" when it came to Cyclomatic Complexity.  The worst violator is "NQF 0038: Childhood Immunizations" which is actually part of the core set of MU Stage 2 for Eligible Professional CQMs.

Since we couldn't use a linear scale for those CQMs, so we capped the scale at 60, and switched the presentation of the results around by providing a red background around the Cyclomatic Complexity value.  Normally, there isn't a background and the results are just black text.  The idea here was to try and call out that there was something fundamentally wrong with complexity values that are so high.

See the 6 most complex CQMs for MU Stage 2 Eligible Professionals (EP) below:

6 Most Complex CQMs for MU Stage 2 Eligible Professionals

From a software engineer's perspective, if you have to manually implement these CQMs in software code, you are going to be very hard pressed to test and validate that any system implementing this logic accurately.  Ie. you have a challenging job ahead of you to test that any implementation of these 6 CQMs are in fact accurate.  This also means that there's a higher probability of having a lurking software bug in your implementation.

Also, for healthcare providers who need to understand these CQMs in order to improve the quality of care that they are providing to their patients, they will also have a challenge.  Providers needing to maintain a mental model of the CQMs and understand what actions should be taken for their patients will likely face challenges.

Ideally, the complexity of the CQMs should never even get this high.  What I would want to see is some consideration by measure stewards on Cyclomatic Complexity during the development process.  Another consideration for the policy leaders would be refusal to accept any CQM submitted that had a Cyclomatic Complexity value that exceeded a threshold.

The Kamira project's work has initially focused on the Meaningful Use Stage 2 Eligible Professional CQMs.  The reason that these were initially selected is because we were easily able to instrument the JavaScript code that is implemented in the popHealth and Cypress project's quality measure engine.  However, the work can be expanded to include the MU Stage 2 Eligible Hospital (EH) CQMs when those are implemented in popHealth and Cypress, currently planned for late April 2013.  Unfortunately, I expect that the EH CQMs will actually be more complex than the EP CQMs.

Additionally, I have plans to expand the work into financial analysis and feasibility of implementing CQMs, but for now the lowest hanging fruit was the complexity analysis.

The full visualization of the Meaningful Use Stage 2 CQM Complexity Data is available here, and the detailed results in a JSON file is available here.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2013.

Complexity and Certification of MU Stage 1 Eligible Professional Clinical Quality Measures

While working in the Clinical Quality Measure space on the two open source projects popHealth and Cypress, I have observed trends in the adoption of EHR vendors of various Clinical Quality Measures (CQMs).  For a little background on the role of CQMs in the Meaningful Use program, read the beginning of the entry that I wrote about Applying Kiviat Visualization to Meaningful Use Clinical Quality Measures. 

For Meaningful Use Stage 1, there are minimal requirements by EHR vendors to support the 6 Core and Core Alternate CQMs, and any 3 of the remaining 38 Meaningful Use Stage 1 CMS.  This allows for some malleability by the commercial EHR vendors to select CQMs based on either their ability to implement CQM logic in their product or based on customer demand for specific CQMs.

The Office of the National Coordinator for Health Information Technology (ONC) hosts the Certified HealthIT Product List (CHPL pronounced "CHaPeL") service.  You can view the list of certified products through the CHPL web interface.  Compiling the results for the EHR products against various Meaningful Use Stage 1 CQMs, there are some interesting results:

Meaningful Use Stage 1 Ambulatory Clinical Quality Measures:
Adoption by EHR vendors from data collected via the CHPL service

In addition to the work I am leading via Cypress, I am also leading a research project to assess the quality of Clinical Quality Measures called "Kamira".  The Kamira project can provide metrics on the quality of CQMs.  For early 2013, this has included automated Cyclomatic Complexity calculation of the CQM algorithmic logic based off of the JavaScript code that the Cypress and popHealth projects use to calculate the CQM results.

If you then compare the results of the CQMs that were tested and certified by EHR vendors against the complexity score of the CQMs, you can see a weak correlation between the two.  You can download the full file with the results here.

Meaningful Use Stage 1 Ambulatory Clinical Quality Measures:
Adoption by EHR vendors from data collected via the CHPL service
compared against Cyclomatic Complexity Analysis of the CMQ logic

It's worth noting that the correlation here is weak, but there does appear to be a trend toward vendors opting to implement the less complex CQMs in their products when they have some latitude to choose.

FYI, the ranges for the CQM complexity (the colored diamonds) are:

• 1-10 Very Simple/Low Risk (green)
• 11-20 Nominal/Moderate Risk (yellow)
• 21-50 Complex/High Risk (orange)
• >50 Untestable/Extreme Risk (red)

These metrics are somewhat arbitrary, but I picked ranges for CQM complexity from a Carnegie Mellon paper that had a good number of citations, so I think that the values and thresholds are fairly defensible.

Where this work might have a few vulnerabilities is that I am 100% certain that EHR vendors do not use complexity as their only consideration when selecting CQMs to implement in their products.  For instance some of the red, highly complex CQMs which were in the middle when it came to adoption by EHR vendors are cardiac CQMs.  From my perspective, it's a safe assumption that some of these EHR vendors were going to bite bullet and implement the cardiac CQMs regardless of the complexity associated with them because there is more demand in the marketplace from providers that need the cardiac CQM results, vs. say the behavioral health CQMs.  However, I think that CQM developers need to start tracking complexity of CQMs as they are developed for MU Stage 3 or beyond.

Lastly, the Kamira project just launched last week.  I plan on posting the MU Stage 2 complexity results for the Eligible Professional CQMs in the coming weeks.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2013.

Longitudinal Visualization Techniques for Clinical Quality Measures

I recently wrote about Applying Kiviat Visualization to Meaningful Use Clinical Quality Measures.  One additional item that is a limitation to using Kiviat diagrams is the visualization of longitudinal trending of metrics over time.  This same limitation exists with Clinical Quality Measures (CQMs).

I am a big fan of Edward Tufte, and his evangelism on the use of sparklines.  Below is an interpretation of how that same sparkline visualization could be applied to a family of diabetic CQMs  from the Meaningful Use program, that is a modified version of Juhan Sonin's HealthCard:

This technique does make some assumptions.  It appears to look decent with two years of notional data.  However, I think that changes in CQM results will probably be needed over decades.  Also, I am not sure if someone like Tufte would violently object to the introduction of date under the illustration.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2012.

The QRDA Category 1 XML Standard

The Quality Reporting Document Architecture (QRDA) Category 1 standard is a new XML standard designed for communicating patient-level clinical data that will be used to calculate Clinical Quality Measures (CQMs). This is an HL7 XML standard that is part of the Clinical Document Architecture  (CDA), which is an overall framework for expressing healthcare data in XML that is also stewarded by HL7.  

The QRDA Category 1 XML standard was created because systems needing to calculate Clinical Quality Measures (CMSs) could not guarantee that patient-level data would appear in the existing continuity of care standards such as the HITSP C32, HL7 Consolidated CDA (CCDA), and the ASTM Continuity of Care Record (CCR).  The QRDA Category 1 standard has been designed so that all of the clinical information needed for each specific CQM will need to be expressed within the QRDA Category 1 XML document. 

The QRDA Category 1 is a clinical document representing a single patient. It defines sections containing measure information and patient data that meets criteria for a measure. It also contains information on the reporting period. The markup for the patient data is the same as a CCDA document.  

In a nutshell, the QRDA Category 1 XML standard will include the templates needed for each attribute in each CQM that a patient could provide clinical data to be used to calculate any number of CQMs.  Additionally, more than one CQM can be included in the request for the clinical data associated with one patient.  

If you view the illustration below, you can see that an Electronic Health Record (EHR) system may be requested to generate QRDA Category 1 XML for patients to be used to drive the calculation of the Diabetes Blood Pressure Management CQM AND the Diabetes LDL Test CQM.  In that case, all clinical data that could be used to drive the calculation of either of those CQMs must appear in the QRDA Category 1 for each patient in the EHR.  Further, additional clinical information associated with each patient is not required, and will result in warnings (but not errors) in the QRDA Category 1 for each patient.

The C32/CCDA vs QRDA Category 1
Only relevant data is expressed in the QRDA Category 1
based on the need for specific CQMs

I personally feel that the QRDA Category 1 reflects the failures of the CCR, C32, and CCDA standards.  It is difficult to explain why a single standard for expressing patient-level data that is interoperable and can provide sufficient fidelity of data for calculating CQMs does still not exist.  It's 2012.

Where problems are identified with a CQM when the CCR... the C32... the CCDA do not provide sufficient information for calculating MU CQMs, CQMs could be adjusted to be simpler, and conform to data that must exist in the CCR... the C32... the CCDA.  This lack of a good continuity of care standard for a single patient record is what I feel is the root cause of all of this work.  

Knowing that the QRDA Category 1 isn't going away anytime soon, I have found that the use of the QRDA Category 1 XML standard has been difficult due to lack of validation tests and example QRDA Category 1 XML files.  However, I suspect that these two shortcomings will be addressed in time.

The objective of using the QRDA Category 1 XML standard as a mechanism to present clinical quality data associated with a patient should unambiguously define clinical attributes so that there’s no confusion about what is expected of an EHR system's ability to export in XML artifacts.  Time will tell if this objective is realized in EHR systems... I remain skeptical.  I think we will know a lot more by 2013, when this standard is officially introduced into the Meaningful Use Stage 2 program.


This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. © Rob McCready, 2012.