Standardized Assessment of Cognitive Function, Mood, and Pain Among Patients Who Are Unable to Communicate

Victoria Shier PhD; Maria O. Edelen PhD; Tara L. McMullen PhD; Michael S. Dunbar PhD; Patricia Bruckenthal PhD; Sangeeta C. Ahluwalia PhD; Emily K. Chen PhD; Sarah E. Dalton MA; Susan Paddock PhD; Anthony Rodriguez PhD; Stella Mandl BSW, BSN, RN; Teresa Mota BSN, RN; Debra Saliba MD, MPH, AGSF

Disclosures

J Am Geriatr Soc. 2022;70(4):1012-1022.

Abstract and Introduction

Abstract

Background: Assessments of patients have sought to increase the patient voice through direct patient interviews and performance-based testing. However, some patients in post-acute care (PAC) are unable to communicate and cannot participate in interviews or structured cognitive tests. Therefore, we tested the feasibility and reliability of observational assessments of cognitive function, mood, and pain for patients who are unable to communicate in PAC settings.

Methods: We conducted a national test of observational assessments of cognitive function, mood, and pain in 143 PAC facilities (57 home health agencies, 28 Inpatient Rehabilitation Facilities, 28 Long-Term Care Hospitals, and 73 Skilled Nursing Facilities) across 14 U.S. markets from November 2017 to August 2018. For the 548 patients identified as unable to make themselves understood, we assessed descriptive statistics, percent of missing data, time to complete, and inter-rater reliability (IRR) between paired research nurse and facility staff assessors.

Results: Most sampled non-communicative patients were administered all three observational assessments. Among assessed patients, overall missing data was high for some items within the Staff Assessment for Mental Status (2.9% to 33.5%) and Staff Assessment of Patient Mood (12.4% to 44.3%), but not the Observational Assessment of Pain or Distress (0.0% to 4.4%). Average time to complete the data elements ranged from 2.4 to 3.5 min and IRR was good to excellent for all items (kappa range: 0.74–0.98).

Conclusion: The three observational data elements had acceptable reliability. Although results revealed varying feasibility, there was support for feasibility overall in terms of implementing a standardized observational assessment of pain for patients in PAC settings. Additional work is needed for the Staff Assessment for Mental Status and the Staff Assessment of Patient Mood to improve the observable nature of these data elements and enhance instructions and training for standardizing the assessments.

Introduction

The Centers for Medicare & Medicaid Services (CMS) has sought to ensure that the clinical assessment instruments are patient-centered and consider the "voice" of the patient. Therefore, the assessment instruments have included patient interviews. For example, the testing work on the Minimum Data Set (MDS) 3.0^[1] developed and tested data elements that served to increase the resident's own voice and perspective, bolstering person-centered care. However, a number of patients in post-acute care (PAC) are unable to make their needs known through verbal or written communication. In 2018, the nursing home (NH) resident interviews for cognitive status, mood, and pain were not attempted among approximately 10%, 11.5%, and 8.5% of residents, respectively.^[2]

The Improving Medicare Post-Acute Care Transformation Act of 2014's (IMPACT Act) mandated CMS to develop and implement standardized patient assessment data.^[3] The IMPACT Act requires standardized patient assessment data for home health agencies (HHAs), inpatient rehabilitation facilities (IRFs), long-term care hospitals (LTCH), and skilled nursing facilities (SNFs) with respect to the following categories: (1) cognitive function and mental status; (2) special services, treatments, and interventions; (3) medical conditions and comorbidities; (4) impairments; and (5) other categories.^[4] (For readability, we use "patient" to indicate patients of HHAs, IRFs, and LTCHs and residents of SNFs.) The national field test of candidate standardized assessment data elements (SADEs) included patient interviews or direct performance-based testing for cognitive function, mood, and pain, among others. This included the Brief Interview for Mental Status (BIMS), a performance-based testing of cognitive function, that improves identification of cognitive ability and can be used as a brief performance test to expand the assessment of signs and symptoms of delirium.^[5,6] The national field test also included the Patient Health Questionnaire–9 (PHQ-9), a structured interview of mood tested in a wide range of populations.^[5,7–11] Patient self-report is thought to be foundational for assessing symptoms of depression.^[12] Similarly, self-reported pain has widely been seen as the gold standard for assessment^[13] and the national field test included a structured pain interview.^[14] These three data elements performed well in the national field test and were found to be feasible and reliable in PAC settings.^[15–17]

Data elements that can assess observed behaviors are required to assess the patients receiving PAC who are unable to participate in interviews because of cognitive impairment or difficulty communicating. Assessment of these patients will provide staff with important information about how to better meet the patients' unmet needs, including pain or physical discomfort, mental discomfort, loneliness, or need for social interaction. This paper presents the results of feasibility and reliability testing of the three observational assessments for standardization across HHAs, IRFs, LTCHs, and SNFs: Staff Assessment for Mental Status, Observational Assessment of Patient Mood, and Observational Assessment of Pain or Distress. These assess cognitive function, mood, and pain for patients in PAC settings who are unable to communicate.

1 2 3 4

Next Section

Abstract and Introduction
Methods
Results
Discussion
References
Sidebar

Table 1. Frequencies for observational assessments of patients who are unable to communicate: by post-acute care (PAC) setting and overall
	HHA	IRF	LTCH	SNF	Overall
Staff Assessment for Mental Status
# of assessments	32	103	169	209	513
% Short-term memory ok	25	20	24	8	16
% Long-term memory ok	54	32	24	16	24
% of patients normally able to recall^a
Current season	20	38	12	5	15
Location of own room	50	20	7	17	17
Staff names and faces	46	50	33	37	39
In care facility	63	63	41	22	39
Ability to make decisions regarding everyday tasks
% Independent	0	1	1	1	1
% modified independence	7	7	3	1	3
% moderately impaired	27	35	15	16	20
% Severely impaired	67	57	82	81	76
Staff Assessment of Patient Mood
# of assessments	32	98	155	216	501
Depression categorization
% No depression	53	60	66	64	63
% Minor depression	33	15	25	23	22
% Major depression	13	25	10	13	15
Observational Assessment of Pain or Distress
# of assessments	32	107	183	223	545
Observed indicators of pain or distress^a
% non-verbal sounds	34	24	21	23	23
% vocal complaints	41	24	19	22	23
% facial expressions	44	35	48	30	37
% body movements or postures	50	22	35	19	27
% No observed indicators of pain or distress	31	50	36	50	44
Frequency of evidence of pain past 3 days
% Less than daily	30	55	32	52	43
% Daily	25	22	24	19	22
% More than daily	45	24	43	29	35
% for whom indicators of pain resolve/diminish with medication or treatment	100	83	83	81	83

^aMore than one may be selected.

Table 2. Range of percent missing data for observational assessments of patients who are unable to communicate: by post-acute care (PAC) setting and overall
	HHA	IRF	LTCH	SNF	Overall
	% Missing	% Missing	% Missing	% Missing	% Missing
Staff Assessment for Mental Status^a	6.3%; 12.5%–21.9%	2.9%; 15.5%–30.1%	3.6%; 36.7%–55.0%	1.9%; 10.5%–23.4%	2.9%; 23.4%–30.8%
# assessments	32	103	169	209	513
Staff Assessment of Patient Mood	3.1%–37.5%	6.1%–38.8%	16.1%–58.7%	16.1%–58.7%	12.4%–44.3%
# assessments	32	98	155	216	501
Observational Assessment of Pain or Distress	0.0%–6.3%	0.0%–7.5%	0.0%–3.8%	0.0%–4.5%	0.0%–4.4%
# assessments	32	107	183	223	545

Abbreviations: HHA, home health agency; IRF, inpatient rehabilitation facility; LTCH, long-term care hospital; SNF, skilled nursing facility.
^aStaff Assessment for Mental Status presents the % missing for Ability to make decisions regarding everyday tasks separately from the other data elements because the percentage was much lower than the other data elements in the data element.

Table 3. Time to complete (in minutes) the observational assessments of patients who are unable to communicate: by post-acute care (PAC) setting and overall
	HHA	IRF	LTCH	SNF	Overall
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Staff Assessment for Mental Status	3.4 (2.3)	2.4 (1.5)	2.6 (1.4)	2.5 (1.6)	2.6 (1.6)
Staff Assessment of Patient Mood	4.5 (2.4)	3.7 (1.9)	3.5 (1.8)	3.3 (1.5)	3.5 (1.7)
Observational Assessment of Pain or Distress	2.4 (1.7)	2.4 (1.9)	2.5 (1.6)	2.3 (1.6)	2.4 (1.7)

Abbreviations: HHA, home health agency; IRF, inpatient rehabilitation facility; LTCH, long-term care hospital; SNF, skilled nursing facility.

Table 4. Inter-rater reliability kappa or weighted kappa for observational assessments of patients who are unable to communicate: by post-acute care (PAC) setting and overall
	HHA	IRF	LTCH	SNF	Overall
Staff Assessment for Mental Status
Short-term memory ok	0.82	0.64	0.77	0.72	0.74
Long-term memory ok	0.84	0.85	0.74	0.82	0.82
Patient is normally able to recall current season	0.86	0.84	0.89	N/A	0.90
Location of own room	0.93	0.87	N/A	0.98	0.94
Staff names and faces	0.79	0.84	0.94	0.90	0.89
in care facility	0.83	0.80	0.92	0.91	0.89
Ability to make decisions regarding everyday tasks	1.00	0.78	0.92	0.92	0.89
Staff Assessment of Patient Mood
Sum of all symptom frequencies (PHQ-9-OV)^a	1.00	0.98	0.92	0.98	0.97
Observational Assessment of Pain or Distress
Observed indicators of pain or distress:
Non-verbal sounds	0.86	0.65	0.80	0.87	0.81
Vocal complaints	0.94	0.73	0.87	0.90	0.86
Facial expressions	0.80	0.76	0.82	0.85	0.83
Body movements or postures	0.88	0.80	0.80	0.87	0.83
No observed indicators of pain or distress	1.00	0.75	0.91	0.92	0.89
Frequency of patient complaints or evidence of pain past 3 days	0.89	0.78	0.90	0.85	0.87
Indicators of pain resolve/diminish with medication or treatment	N/A	N/A	0.82	0.95	0.90

Note: Inter-rater reliability not shown (N/A) for data elements with proportions out of range for stable kappa estimate (per study power calculations). Interpretation of kappa or weighted kappa is as follows: 0.00–0.20 is slight/poor, 0.21–0.40 is fair, 0.41–0.60 is moderate, 0.61–0.80: substantial/good, and 0.81–1.00 is excellent/almost perfect.
Abbreviations: HHA, home health agency; IRF, inpatient rehabilitation facility; LTCH, long-term care hospital; SNF, skilled nursing facility.
^aAll data elements within the PHQ-9-OV had kappas ≥0.84 or N/A overall and by settings.

Authors and Disclosures

Victoria Shier PhD¹, Maria O. Edelen PhD^2,3, Tara L. McMullen PhD⁴, Michael S. Dunbar PhD⁵, Patricia Bruckenthal PhD⁶, Sangeeta C. Ahluwalia PhD^7,8, Emily K. Chen PhD⁹, Sarah E. Dalton MA⁶, Susan Paddock PhD¹⁰, Anthony Rodriguez PhD², Stella Mandl BSW, BSN, RN¹¹, Teresa Mota BSN, RN¹² and Debra Saliba MD, MPH, AGSF^7,13,14

¹Leonard D. Schaeffer Center for Health Policy & Economics, Sol Price School of Public Policy, University of Southern California, Los Angeles, California, USA
²RAND Corporation, Boston, Massachusetts, USA
³Patient Reported Outcomes Value and Experience (PROVE) Center, Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts, USA
⁴Pain Management, Opioid Safety, and PDMP Program Office, Veterans Health Administration, Washington, District of Columbia, USA
⁵RAND Corporation, Pittsburgh, Pennsylvania, USA
⁶Stony Brook School of Nursing, Stony Brook, New York, USA
⁷RAND Corporation, Santa Monica, California, USA
⁸UCLA Fielding School of Public Health, Los Angeles, California, USA
⁹RAND Corporation, Arlington, Virginia, USA
¹⁰NORC at the University of Chicago, Chicago, Illinois, USA
¹¹Division of Health Care Financing, Office of Health Policy in the Office of the Assistant Secretary for Planning and Evaluation, Department of Health and Human Services, Washington, District of Columbia, USA
¹²Abt Associates, Cambridge, Massachusetts, USA
¹³UCLA Borun Center, Los Angeles, California, USA
¹⁴Los Angeles VA GRECC, Los Angeles, California, USA

Correspondence
Victoria Shier, Leonard D. Schaeffer Center for Health Policy & Economics, Sol Price School of Public Policy, University of Southern California, 635 Downey Way, Los Angeles, CA 90089, USA. Email: vshier@healthpolicy.usc.edu

Conflict of Interest
The authors declare no conflicts of interest.

Author Contributions
Study concept and design: Maria O. Edelen, Sangeeta C. Ahluwalia, Emily K. Chen, Sarah E. Dalton, Susan Paddock, Anthony Rodriguez, Debra Saliba, Victoria Shier, Michael S. Dunbar, Patricia Bruckenthal, Tara L. McMullen, Stella Mandl, Teresa Mota. Acquired subjects and/or data: Maria O. Edelen, Sangeeta C. Ahluwalia, Sarah E. Dalton. Analysis and interpretation: Victoria Shier, Maria O. Edelen, Michael S. Dunbar, Patricia Bruckenthal, Sarah E. Dalton, Anthony Rodriguez. Manuscript preparation: Victoria Shier, Maria O. Edelen, Tara L. McMullen, Debra Saliba.

processing....

Standardized Assessment of Cognitive Function, Mood, and Pain Among Patients Who Are Unable to Communicate

Abstract and Introduction

Abstract

Introduction

Tables

Tables

Tables

Tables

References

Authors and Disclosures

Authors and Disclosures

Sidebar

Sidebar

Key Points

Most Popular Articles

Email This

Feedback