Development, Reliability, Validity, and Item-Level Psychometric Measurements of the Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) to Evaluate Registered Dietitians’ Competence in Performing Nutrition Focused Physical Exam on Adult Patients

Abstract

Alarmingly high rates of disease-related malnutrition have persisted among hospitalized patients with prevalence estimated as high as 30% to 50%. Malnutrition is associated with several negative clinical outcomes and hence, proper diagnosis and treatment are essential to mitigate complications. To improve malnutrition diagnostic accuracy and to determine other nutrient deficiencies, registered dietitian nutritionists (RDNs) recently integrated the nutrition-focused physical exam (NFPE) as part of nutrition assessment. Increased use of this essential skill among RDNs and the transformation of dietetics education to a competency-based model in the near future calls for appropriate measurement tools to gauge RDNs’ NFPE competence. However, published NFPE competency tools that are reliable and validated are sparse. This study aimed to fill this gap through the development and testing of an Interactive Nutrition-Specific Physical Exam Competency Tool (INSPECT). The initial design of the INSPECT contained 70 NFPE items identified through expert focus group discussions. The second phase tested the preliminary version of the INSPECT for content and face validity utilizing 17 RDN experts over two Delphi rounds. The expert consensus revealed acceptable face validity (α = 0.71) and excellent content validity for the INSPECT, with an internal consistency of α = 0.97 in the first round and α = 0.96 in the second round. The inter-rater agreement was also excellent with ICC =0.95 for each of the Delphi rounds. The overall Delphi consensus resulted in the retention of 41 NFPE items within the INSPECT. The third phase of the study tested the final version of the INSPECT utilizing multi-site observations of clinical supervisors rating RDNs’ NFPE competency during patient assessment. The INSPECT exhibited good inter-rater reliability (ICC = 0.78 for the first assessment and ICC =0.68 for the second assessment observed after 2 weeks), moderate to strong intra-rater reliability for 37 of 41 items (Spearman rho = 0.54 to 1.0), and excellent internal consistency (Cronbach’s α = 0.86 for the first assessment and α = 0.92 for the second assessment after 2 weeks). Ten out of 11 INSPECT subsets showed good to excellent internal consistency (α ranging from 0.70 to 0.98). Item-level properties were evaluated with Rasch analysis utilizing only the first assessment observations. Rasch analysis confirmed that the INSPECT measured a single construct as all items fit the established criteria for clinical observations of >0.5 and <1.7, had positive point measure correlations, met the Wright unidimensionality index criteria of ≥0.9, and exhibited one latent construct with a sub-dimension for the principal component analysis of Rasch residuals. Rasch rating scale analysis revealed that the rating scale and majority of the items (39/41) fit the Rasch model. Rasch item hierarchy analysis matched the a priori hypothesized hierarchy for the top-most and bottom-most items. Rasch reliability assessment demonstrated high person reliability (0.86), high item reliability (0.96), and separated person ability into a little more than 3 levels (3.56 ability levels). Generally, Rasch analysis revealed acceptable item-level psychometric properties. Overall, the results demonstrated the INSPECT to be a reliable and valid tool that can be utilized to measure, advance, and maintain RDNs’ NFPE competence in acute care clinical settings. Continued testing of the INSPECT with RDNs of different ability levels to determine cut-off scores ranging from novice to expert and evaluating the INSPECT in other clinical settings such as long-term care, rehabilitation, and outpatient clinics will further enhance the utility of the tool.