Validity of the ICD-9-CM Based Patient Safety Indicators
Haytham MA Kaafarani, MD, MPH, Amy K. Rosen, PhD, Ann Borzecki, MD, MPH, Kamal M.F. Itani, MD
VA Boston Healthcare System, Boston, MA
PURPOSE OF STUDY
The Agency for Healthcare Research and Quality (AHRQ) recently designed the Patient Safety Indicators (PSIs) to detect safety-related adverse events. The National Quality Forum has endorsed several of these ICD-9-CM based indicators as quality of care measures. We examined the positive predictive value (PPV) of three surgical PSIs: 1) postoperative pulmonary embolus/deep vein thrombosis (pPE/DVT), 2) iatrogenic pneumothorax (iPTX) and 3) accidental puncture/laceration (APL).
We applied the AHRQ PSI software (v.3.1a) to Veterans Health Administration 2003-07 administrative data to identify (flag) cases suspected of having a pPE/DVT, iPTX or APL. Two trained chart-abstractors reviewed a sample of 336 flagged medical records (112 records/PSI) using a standardized instrument. Inter-rater reliability was assessed.
SUMMARY OF RESULTS
Of 2,343,088 admissions, 6,080 were flagged for pPE/DVT (0.26%), 1,402 for iPTX (0.06%) and 7,203 for APL (0.31%). For pPE/DVT, PPV was 44% (95% CI, 34-53%); 21% of cases represented inaccurate coding (e.g. arterial not venous thrombosis), and 35% featured thromboembolism present on admission or preoperatively. For iPTX, PPV was 80% (95% CI, 73-88%); 13% represented inaccurate coding (e.g. spontaneous pneumothorax) and 7% were pneumothoraces present on admission. For APL, PPV was 85% (95% CI, 78-91%); 10% of cases represented coding inaccuracies and 5% indicated injuries present on admission. However, 27% of true APLs were minor injuries requiring no surgical repair (e.g. small serosal bowel tear). Inter-rater reliability was >90%.
Until adequate coding revisions are implemented, PSIs should only be used for screening and case-finding. Their utility as quality measures may need to be re-assessed.