Evaluating a Homemaking Assessment for Broader Application to Practice

Nicole Matichuk, BKIN1, Liv Brekke, MPA1, Hilary Drummond, OT(C) 2, & Susan Forwell,

PhD, OT(C) 1

1Department of Occupational Science & Occupational Therapy, University of British Columbia

2Creative Therapy Consultants

1

This manuscript was prepared for the purpose of RSOT 547, in accordance with CJOT

article guidelines, with the exception of blinding the manuscript. The manuscript was not

blinded for RSOT 547 project clarity.

2

Abstract

Background. In response to third party payer needs, an assessment was developed to provide a

fair, objective, and consistent approach to quantifying the assistance an individual requires with

homemaking tasks as a result of accident or disability. Purpose. To evaluate psychometric

properties of this homemaking assessment for use with community-dwelling adults with injury or

disability. Methods. A retrospective chart review of 113 cases examined content validity and

internal consistency, and a prospective design of 9 cases tested interrater reliability. Findings.

Retrospective data demonstrated that assessment subscales have good internal consistency, items

are categorized appropriately, and the assessment measures a single construct overall. Several

latent constructs representing types of task demands were identified across assessment items.

Prospective data showed early evidence of acceptable interrater agreement. Implications.

Preliminary evidence is established for the psychometric integrity of a novel homemaking

assessment, providing the foundation for broader use of this assessment in community practice.

Keywords: Housekeeping, Measurement, Occupational Therapy, Psychometrics, MVA (Motor

Vehicle Accident)

3

Introduction

Instrumental activities of daily living (IADLs), such as homemaking, are integral to daily

life. IADLs are habits or routine tasks that allow individuals to be independent and engaged

within their homes and communities (Clemson, Bundy, Unsworth & Singh, 2009). They go

beyond personal care to include tasks requiring higher level cognitive processes and interactions

with varying environments and people. The ability to perform these everyday tasks has been

directly related to quality of life and well-being (Clemson et al., 2009). Aging, falls, motor

vehicle accidents (MVAs), and other injuries and disabilities have the potential to impact a

person’s ability to perform IADLs (Clemson et al., 2009). Decreased performance on IADL

tasks, like homemaking, can also be an indicator of higher care needs or supports (Clemson et

al., 2009). Occupational therapists are essential in assessing IADL function of their clients, yet

no assessment exists that evaluates homemaking in depth.

The CTC Homemaking Assessment was developed in direct response to the need to

provide a fair, objective, and consistent approach to assessing clients who might require supports

for housekeeping and home management tasks due to accident or disability. It is a

comprehensive, in-depth assessment tool designed by occupational therapists to determine the

percentages of housework tasks an individual can perform. While other IADL assessments

existed at the time of its development, there was no tool which focused on homemaking

specifically and which provided a concrete percentage of tasks an individual could complete. A

percentage was necessary because the majority of clients undergoing the CTC Homemaking

Assessment had been involved in MVAs, and the primary insurer provides supports only for

those individuals who can no longer perform the majority (i.e. 50 percent) of their previous

homemaking tasks.

4

The population of individuals injured in an MVA is large and increasing. Between July

2014 and June 2015, the number of reported new injury claims as a result of an MVA in British

Columbia increased by 10 per cent over the previous 12 months, to a total of 67,700 new claims

(ICBC, 2015). In addition, the costs associated with these claims (which cover payouts for pain

and suffering, future care, and loss of wages) have increased by 64 per cent since 2008 (ICBC,

2015). It has also been suggested that fraudulent claims may be on the rise (ICBC, 2015).

Together, the increase in bodily injury claims, the associated costs on the system, and concerns

about fraud all reinforce the need for valid and reliable assessments to be used in determining

what benefits an injured individual should receive. To date, however, there has been no

consensus on what method is best used for determining the need for these supports. Since its

development, the CTC Homemaking Assessment has been well accepted by clinicians, lawyers

and insurers; however, the measurement properties of the assessment have yet to be

systematically evaluated.

Issues in IADL Measurement

To date, there has been no research into the development and application of assessment

instruments specifically related to homemaking. For the purposes of informing this study, we

instead examined the literature on the assessment of IADLs, a broader construct which generally

includes homemaking tasks. Within the literature, what exactly constitutes IADLs remains the

subject of debate. IADLs are typically defined as activities that help facilitate independent living,

such as meal preparation, taking medication, managing money, using a telephone, shopping, and

housekeeping (Gold, 2012; Law, 1993; Law, Baum, & Dunn, 2005). Various ways of further

categorizing these activities have been discussed. Some authors have sorted IADLS into three

categories: getting about, household-based activities, and leisure-oriented activities (Law, Baum,

5

& Dunn, 2005). Leung et al. (2011) highlighted that IADLs have been theoretically divided into

physical and more complex cognitive tasks. However, empirical studies on the physical-

cognitive division of IADLs have been inconclusive; some show that IADLs can be expressed as

two separate constructs, while others show that IADLs are highly correlated and therefore

unidimensional (Leung, Chi, & Leung, 2011). Reuben et al. (1990) proposed advanced activities

of daily living (AADLs), another construct distinct from IADLs. These activities “are volitional,

influenced by cultural and motivational factors, expressing a personal engagement in satisfying

activities which are beyond what is needed to be independent” (De Vriendt et al., 2012, p. 975).

Examples of AADLs have included the use of household appliances (e.g. washer and dryer) and

advanced kitchen activities such as preparing a meal with a large number of ingredients (De

Vriendt et al, 2012). In contrast to all these construct theories, the CTC Homemaking

Assessment categorizes homemaking tasks according to task difficulty. It includes 29 items

divided into three categories: light, medium, and heavy tasks. (See Table 1 for a complete list of

assessment items).

Existing IADL assessments have been criticized for the number of different tests for

different populations, reliance on self-report rather than clinician observation, being cumbersome

and lengthy to administer, being based on varying conceptual frameworks, and relying on

outdated or specific cultural perspectives (Bottari, Dassa, Rainville & Dutil, 2010; Law, 1993;

Letts et al., 1994). Unlike the CTC Homemaking Assessment, which has been used with

individuals with a wide range of physical, cognitive, and/or mental health issues, the content of

many assessments is often developed for specific impairments, not on a broader disability basis

(Law, 1993). Some IADL assessments have also received only limited psychometric testing

(Clemson et al., 2009; Law & Letts, 1989). Instrument studies typically need a sample size of at

6

least 50 to support reliable conclusions, yet few occupational therapy instrument studies have

met this threshold (Yuen & Austin, 2014). In short, no reliable gold standard for IADL

assessment has yet to clearly emerge (Law, Baum, & Dunn, 2005).

IADLs can be evaluated via self-report, proxy report, direct observation, and/or medical

chart data extraction (Law, Baum & Dunn, 2005). Performance-based assessments are often

considered more effective. However, some evidence has suggested that these are not

psychometrically superior to self-report questionnaires, and that self-report measures generally

correlate well with performance-based ones when measuring the same domain of disability

(Coman & Richardson, 2006; Myers, Holliday, Harvey & Hutchinson, 1993). A more recent

review of the psychometric properties of instruments used to assess workers applying for

disability benefits concluded that “a combination of patient self-reports, performance tests and

medical examination […] seems the most solid tool for assessing functional limitations in work

disability benefits” (Spanjer et al., 2011, p. 2149). In keeping with this perspective, the CTC

Homemaking Assessment relies on information obtained both through therapist observation of

client performance in some homemaking tasks, as well as client-therapist interview.

Another important criterion for developing a measurement strategy is relevance and

appropriateness to the client’s life (Law, Baum, & Dunn, 2005). Research comparing assessment

methods highlights the importance of developing measurement tools which are ecologically

valid. From an occupational therapy perspective, this reflects a core assumption that engagement

in activity occurs within an individual’s unique physical, social, institutional, cultural and

temporal context. Therefore, “when conducting an evaluation [...] the environmental context in

that situation will have an influence on the person’s performance” (Law, Baum, & Dunn, 2005,

p. 25). In examining the effect of assessment environment, Rogers et al. (2003) found that self

7

and proxy reports were more concordant with a patient’s actual in-home performance of IADLs

than clinical judgment or performance testing in clinic. For physical IADLs specifically, they

found that self-report, proxy report, clinical judgment and clinic-based performance all

significantly underestimated a patient’s actual abilities. This suggests that, ideally, IADL

assessments should be conducted in the environment in which the client performs them. In

addition, IADL assessments need to account for the fact that tasks can be both gender and culture

specific (Law, Baum, & Dunn, 2005) and that individuals demonstrate different patterns of

occupational performance across their day and lifetime (Law, 1993). The CTC Homemaking

Assessment is administered in the client’s own home and is therefore grounded in the client’s

own context. Similarly, the assessment can account for the particular performance patterns of the

individual client. If an item is not applicable or appropriate for a client, it is scored as “not

applicable” and simply not used in calculating the assessment results.

Instrument Development and Psychometrics

In order to systematically examine the reliability and validity of any IADL assessment,

the purpose of the tool must first be established; which specific psychometrics to evaluate for a

certain instrument depends, in part, on the purpose of the instrument itself (Law, 1987; Law &

MacDermid, 2014). Descriptive measures give a snapshot of a person at one moment in time,

predictive measures set criteria against which to compare a person’s current status, and

evaluative measures assess change over time (Law & Letts, 1989). Many IADL tools can be used

for more than one purpose. The CTC Homemaking Assessment has historically been used for

both descriptive and evaluative reasons. This study focused on examining the measurement

properties of the CTC Homemaking Assessment primarily from a descriptive perspective.

8

Descriptive instruments should have evidence of content and construct validity, internal

consistency, and interrater reliability (Law, 1987).

The validity of an instrument is the degree to which an assessment measures what it is

intended to measure (Law & Letts, 1989). Content validity assesses the extent to which an

instrument’s items comprehensively represent all the characteristics of a construct it is designed

to measure (Law, 1987; Mokkink et al., 2010; Yuen & Austin, 2014). Methods of establishing

content validity include judgment by clinicians or patients, or statistical methods, such as

exploratory or confirmatory factor analysis (Law, 1987; Mokkink et al., 2010). With the CTC

Homemaking Assessment, a number of occupational therapists were involved in an iterative

process of item development and refinement over the course of several years. In addition, the

assessment has been well accepted by insurers and lawyers. However, statistical tests of content

validity have not yet been conducted.

Reliability speaks to the stability, consistency, and dependability of an instrument’s

measurements (Law, 1987), and includes properties such as internal consistency and interrater

reliability. Internal consistency refers to the “homogeneity of items or scores within an

instrument” (Law & MacDermid, 2014), while interrater reliability assesses how consistently an

instrument performs when administered by different practitioners (Vroman & Stewart, 2014).

Neither of these properties has yet to be examined in the CTC Homemaking Assessment.

Study Purpose

While the CTC Homemaking Assessment has been used with strong acceptance to date,

there is increasing potential for it to be used beyond a single private practice. Systematic

evaluation of the measure is necessary so that other clinicians can understand the properties of

the assessment and determine whether they wish to adopt it in their own practice. Therefore the

9

purpose of this study was a preliminary evaluation of aspects of the validity and reliability of the

CTC Homemaking Assessment for use by occupational therapists with community-dwelling

adults. Specifically this study sought to answer the following questions: (1) what characteristics

and performance patterns can be identified among clients with whom the assessment has been

used; (2) what constructs can be identified among the assessment items; (3) to what extent are

scores on items within each of the assessment’s subscales homogenous; and (4) how consistently

does the assessment perform when scored by different practitioners?

Methods

Study Design

This study focused on instrument testing of the CTC Homemaking Assessment using

quantitative methods. The overall design included both a retrospective chart review and a

prospective component.

Participants

The target population for this study was community-dwelling adults with injury or

disability. Two convenience samples were used, comprised of past and present adult (19+)

clients of Creative Therapy Consultants who had the CTC Homemaking Assessment

administered to them at least once. The sample for the retrospective phase included inactive

clients for whom records were available, while the sample for the prospective phase included

current clients who provided consent for their data to be used. The study excluded participants

under the age of 19. Participants in the prospective component were recruited by CTC clinicians

from among their caseload. Prior to being assessed, potential prospective participants received an

introductory letter describing the study and provided signed, informed consent.

Data Collection

10

All data collection was facilitated by CTC clinicians and staff and provided to the

research team in de-identified form. Unique identifiers were created for each participant and

clinician, and the research team was not able to associate these unique identifiers to client or

clinician names.

Retrospective Phase. A CTC research assistant (RA) reviewed charts of inactive clients

assessed between 2005 and 2016 in order to identify participants who met the study criteria; a

total 113 charts were identified during the study timeframe and all were included in the study

analysis. From these charts, the RA extracted relevant demographic information and captured it

in an audit form which was then provided to the research team, along with de-identified copies of

the assessment results. For retrospective participants, all assessments had been administered and

scored by trained CTC clinicians.

Prospective Phase. With participant consent, a CTC clinician administered the

Homemaking Assessment to clients (n=9) with an occupational therapy student present to serve

as a second rater. The student was oriented to the assessment by a CTC clinician, and was

provided with written scoring instructions. The student observed and independently scored the

assessment, and did not share or discuss their findings with the clinician. Fully independent

assessment administration was not feasible within the study timelines, although it is recognized

that it is recommended practice (Mokkink et al., 2010).

Similar to the retrospective phase, the CTC RA extracted relevant demographic

information from client charts and captured it in an audit form which was then provided to the

research team, along with de-identified copies of the assessment results from both the clinician

and the student.

Data Analysis

11

Descriptive statistics were calculated using Microsoft Excel 2010 (Microsoft

Corporation, 2010). All other statistics were completed with IBM SPSS Statistics Version 24

(IBM Corporation, 2016).

Retrospective. Demographic data, including age, gender, and reasons for referral, were analysed

from the sample. Detailed diagnosis information was reviewed and summarized into six general

categories: pain, fracture and/or joint issues, soft tissue injuries, mental health diagnoses, sensory

changes, head injuries and/or cognitive issues, and chronic disease.

The distribution of total weighted scores on the assessment was examined using a

histogram. Performance trends were also identified and recorded for each participant from their

unweighted percentage scores on each assessment subscale. For example, if a participant’s score

on light tasks was higher than their score on medium tasks, and their score on medium tasks was

in turn higher than their score on heavy tasks, then the performance trend was recorded as

“consistent downward trend.” The number of participants whose results demonstrated each

identified trend was totalled in order to characterize typical performance across assessment

subscales.

For all further statistical calculations, variables (assessment items) with greater than 45

percent missing data were removed from the analysis in order to achieve significant results. Ten

variables were excluded from analysis in total, including: hand-washing clothes, using the

microwave, using the oven, and drying dishes (light tasks); ironing, handling pots and pans,

reaching into upper and lower cupboards, and using the dishwasher (medium tasks); and

vacuuming stairs, and cleaning the freezer (heavy tasks). Some missing values were the result of

the sample including two slightly different versions of the assessment; therefore, not all cases

included all variables. Other missing values were the result of the design of the assessment tool

12

itself, which does not assess an individual’s performance on tasks they typically do not perform.

As a result, tasks such as ironing, drying dishes, and using the dishwasher were scored as “not

applicable” for a large percentage of study participants.

Using the remaining 24 variables (see Table 1), Pearson correlation coefficients

(Pearson’s r) were calculated between all assessment items and between unweighted percentage

scores on each subscale, and the results provided in correlation matrices. Production of these

matrices was designed to enable the research team to identify the extent to which items in a

single subscale were related to each other, as well as the extent to which items in different

subscales did not correlate, in order to help inform the categorization of scale items. R values

around 0.10 were interpreted as a weak relationship, 0.30 as moderate, and 0.50 and above as

strong (Kellar & Kelvin, 2013). The significance level was defined as p > 0.01.

[Table 1 here].

An exploratory factor analysis was performed to help assess the unidimensionality of the

assessment and of each subscale, and to examine for latent constructs within the data and

(Mokkink et al., 2010; Yuen & Austin, 2014). Due to the necessity of excluding a number of

variables, further item analysis and reduction was not carried out prior to running the factor

analysis. A principal axis factor extraction and Varimax rotation were used in order to help

simplify interpretation (Kellar & Kelvin, 2013). The required eigenvalue for a factor was set at 1

and missing data was excluded listwise. The researchers then examined the produced factor

loadings from an activity analysis perspective (Crepeau, 2014) in order to come to agreement on

the potential underlying constructs that might be represented by these factors.

Internal consistency was analyzed by calculating Cronbach’s alpha, the most commonly

used statistic for this purpose (Law & MacDermid, 2014; Mokkink et al., 2010; Vroman &

13

Stewart, 2014). As recommended by Mokkink et al. (2010), Cronbach’s alpha was calculated

separately for each subscale in the assessment in order to determine how well the subscale items

measured the same construct (i.e. light tasks, medium tasks, heavy tasks). Missing data was

excluded listwise. For the purposes of this study, a Cronbach’s alpha value above 0.70 was

considered adequate, 0.8-0.9 good, and above 0.9 excellent (Vroman & Stewart, 2014).

Prospective. Basic participant demographics only (gender and age) were analysed for the

prospective sample (n=9). Due to the small sample size, Cohen’s kappa was not calculated in

order to assess interrater reliability as might be expected (Mokkink et al., 2010; Vroman &

Stewart, 2014). Instead, a table was created allowing for visual comparison of the two raters’

results. The mean difference between scores on each subscale was also calculated, excluding

observations from one case which was determined to be an outlier.

Findings

Retrospective

Demographics. Participant demographic information is summarized in Table 2. Mean

participant age was 50.6 ± 15.2 years. The youngest participant was 19 years old, while the

oldest was 92. Three quarters of participants were female. The most frequent reasons for referral

to CTC for a Homemaking Assessment were MVA and cost of future care assessment. The

majority of clients had diagnoses of pain, fracture and/or joint issues, and soft tissue injuries. Of

the sample analyzed, 86.7 per cent of clients had two or more diagnoses at the time of

assessment.

[Table 2 here].

Overall patterns of response. Total weighted scores for participants were normally

distributed, with the greatest clustering of scores between the 40 to 60 per cent mark. Five

14

distinct performance patterns were identified across assessment subscales: consistent downward

trend, partial downward trend, no trend, flat trend, and consistent upward trend. Consistent

downward trend indicated that performance diminished as task difficulty increased, meaning

clients generally performed best on light tasks and worst on heavy tasks. Over two-thirds of

participants (67.3 per cent) performed with a consistent downward trend. A partial downward

trend was identified among 4.4 per cent of participants, indicating that their performance level

diminished only with heavy tasks.

No trend was identified across 22.1 per cent of participants. For these participants,

performance varied across light, medium, and heavy tasks. For example, a participant might

score well on light tasks, poor on medium tasks, and mid-range on heavy tasks. A flat trend,

indicating consistent scores across all three subscales, was identified for 5.3 per cent of

participants. Finally, 0.9 per cent of participants (n=1) performed with a consistent upward trend,

meaning their performance improved as subscale difficulty increased.

Intercorrelation of subscales and items. All three assessment subscales (light, medium,

heavy tasks) were significantly and positively correlated with each other. The light and heavy

subscales were moderately correlated (r of 0.491). The light and medium subscales demonstrated

a stronger relationship (r of 0.649), while the medium and heavy subscales were also strongly

correlated (r of 0.773).

In terms of assessment items, all tasks within the light subscale were weakly to

moderately positively correlated with each other and most of these relationships were significant.

The most strongly related items were folding clothes and putting clothes away (r of 0.573).

Generally, items in the medium subscale were also weakly to moderately positively correlated

with each other, with the strongest significant relationship between taking clothes out of the

15

dryer and transferring clothes from the washer to the dryer (r of 0.572). The relationship between

cleaning the sink and transferring clothes from washer to dryer was negative (r of -0.024) but

very weak and not statistically significant. All items in the heavy subscale were also weakly to

moderately positively correlated and virtually all of these relationships were significant.

Several particularly strong positive relationships were also observed between items from

different subscales. Cleaning the tub (a heavy task) and cleaning baseboards (a medium task)

reported an r value of 0.694. Sweeping floors (a light task) and washing floors (a medium task)

had an r value of 0.601. Only a single negative correlation was observed between items across

different subscales (cleaning windows and cleaning mirrors); however, this relationship was very

weak (0.10) and not statistically significant (See Supplementary Materials for complete

correlation matrices).

Extracted factors. Table 3 shows the results of the rotated Varimax factor loadings. Six

factors were extracted in total, accounting for 68.4 per cent of the cumulative variance.

Approximately half of the analyzed variables loaded onto Factor 1, suggesting a common

underlying construct among the tasks on the assessment. Variables which loaded onto Factor 1

were similar in that they were all tasks which could require bending at the waist or stooping to

complete. Factor 1 was therefore assigned the label of “Bending Tasks.” These bending tasks

included a mix of items from both the heavy and medium subscales.

[Table 3 here].

Four variables loaded most strongly onto Factor 2. These variables were determined to be

tasks that involved primarily use of the hands and arms to complete (without bending). Thus

Factor 2 was labelled “Upper Extremity Tasks.” These upper extremity tasks were primarily

items from the light subscale, with one medium task. Two variables loaded onto Factor 3,

16

representing perhaps a distinct category of “Clothing Tasks.” These variables did not load

strongly onto any other factors, and were both items from the light subscale. Similarly, a further

two variables loaded onto Factor 4. No commonalities could be surmised from these variables

and therefore Factor 4 was not assigned a label and was not considered useful in instrument

development considerations. Factor 5 was labelled “Load Tasks.” Both variables which loaded

most strongly onto this factor involved the manipulation of a potentially heavy load (i.e. full

garbage bags and wet laundry). Factor 6 was labelled “Planning Tasks” to reflect the potentially

more cognitively complex nature of the activities, e.g. grocery shopping, as well as the potential

need to plan these activities further in advance (e.g. cleaning the windows might require

preparing special equipment and/or determining a day in the future to complete this activity).

In general, the exploratory factor analysis suggested alternative groupings of items on the

CTC Homemaking Assessment. However, these results were not used to definitively change the

organization of the assessment itself. This was due to the fact that the sample size did not reach

the recommended minimum ratio of 10 cases to each variable (Kellar & Kelvin, 2013).

Reliability of Subscales. All three existing subscales demonstrated good internal consistency.

Cronbach’s alpha values were as follows: 0.78 for light tasks; 0.77 for medium tasks; and 0.80

for heavy tasks. No items were identified that, if deleted, would improve the alpha value of their

respective subscales.

Prospective

Demographics. Mean age for the prospective sample (n=9) was 45.1 ± 9.5 years, with a range of

27 years. Two-thirds were female (n=6).

17

Interrater agreement. Table 4 compares the subscale scores assigned by each rater to the same

client. The smallest mean subscale difference between the two raters was 6.3 ± 6.3 per cent, and

the largest was 8.1 ± 11.5 per cent.

[Table 4 here].

Discussion

The results of this study suggest that the CTC Homemaking Assessment reliably

measures the occupation of homemaking across different levels of difficulty. Based on tests of

internal consistency, each of the three assessment subscales can be said to reliably assess the

constructs of light, medium, and heavy homemaking tasks. Moreover, over two-thirds of clients

with whom the assessment has been administered demonstrated decreasing performance as task

difficulty increased, thus reinforcing the distinction between the subscales. In addition, the three

subscales are not perfectly correlated with each other, and the most strongly positively correlated

items occurred within a single subscale.

That being said, the results also suggest that the entire assessment measures a single

construct overall and that, rather than being distinct categories, task difficulty occurs along a

continuum. All three subscales demonstrated a moderate to strong positive correlation, while

individual assessment items also showed consistently positive correlations, suggesting that the

scale overall is generally unidimensional (i.e. assesses homemaking and nothing else). Similarly,

over half of the assessment items loaded strongly onto one factor. This is consistent with

previous studies which suggest that IADLs are all in fact a single construct (Leung et al., 2011).

However, the correlations between subscale scores also showed that there is a stronger

correlation between light and medium scores, and between medium and heavy scores, suggesting

a gradient in task difficulty.

18

In terms of occupational therapy practice, these results suggest that clinicians can use the

CTC Homemaking Assessment and expect it to reliably measure their client’s performance in

homemaking tasks across different levels of difficulty. Future research could focus on looking

for those items which help distinguish better between individuals’ homemaking abilities. The

distribution of total scores clustered around the 50 percent mark which is the cut-off for

homemaking supports from the primary motor vehicle insurance agency. Because of this, and

because descriptive measures should discriminate effectively between individuals (Mokkink et

al., 2010; Law, 1987), finding ways to make the assessment more discriminatory is warranted.

While the study provided support for the reliability and validity of the assessment

subscales related to task difficulty, additional latent constructs were identified from among the

study sample’s results. These underlying constructs, or factors, suggested a second dimension to

the assessment items: type of task demands. These included bending, upper extremity work,

manipulating loads, and planning. To some degree, this is consistent with studies that have made

the distinction between cognitive and physical IADLs (Leung et al., 2011). However, this study

suggests that even finer distinctions can be made among primarily physical tasks, even within the

narrower construct of homemaking. In terms of the difference between IADLs and AADLs

(Reuben et al., 1990), this study’s results were inconclusive about whether this distinction also

occurs within the construct of homemaking. Assessment items that might have been considered

AADLs were excluded from analysis due to a high number of missing values. With a larger

sample size, future studies might be able to provide more insight into whether this IADL-AADL

distinction is applicable to homemaking or not. Finally, the emergence of a “clothing task” factor

might suggest that individuals being assessed view folding clothes and putting clothes away as

very different from cleaning and home management tasks. We speculate that these tasks could

19

potentially be perceived as a more personal or individual, rather than a task that serves the needs

of the household in general. Although a range of latent constructs were identified within the data,

the structure of the assessment was not modified, due to an insufficient sample size. It would be

prudent to re-run the factor analysis with a larger sample before making any changes to the

existing assessment.

The content of the assessment was also not changed based on findings from the item

correlation matrix. These results showed that two items could potentially be removed due to their

strong and significant correlation with another item in the same subscale. These pairs, which also

loaded strongly together onto the same factors, were: folding clothes and putting clothes away;

and cleaning tub and cleaning baseboards. Removing one item from each of these pairs might

appear to be in keeping with recommendations that the number, range, and content of items in a

measurement tool be clearly justifiable (Yuen et al., 2014). However, removing items would

undermine a key strength of the CTC Homemaking Assessment, which is its client-centeredness.

Having a broader range of tasks structure the assessment helps clinicians better account for the

individual occupational performance patterns of clients. If an item is not applicable to a client, it

is simply not scored; therefore, a larger number of items is not inappropriate in these

circumstances.

Rather than changing the construction of the assessment itself, the results of this study

should inform how the assessment is administered and interpreted by clinicians. In particular, our

results have implications for the selection of tasks for observation. As per the design of the CTC

Homemaking Assessment, items are scored based on either self-report or direct observation of

task performance. Which tasks are to be observed is left to the discretion of the clinician. This

decision is particularly important as the literature suggests that direct observation in a client’s

20

home can be superior to, or an important adjunct to, self-reported performance status (Spanjer et

al., 2011). Based on our results, clinicians should consider not only the task difficulty, but also

the underlying type of task demand when selecting tasks to observe. Clinicians should observe

homemaking tasks that place varying types of physical and cognitive demands on clients in order

to obtain the most comprehensive picture of the client’s overall homemaking ability. For

example, if a client struggles to complete a medium bending task (e.g. cleaning the toilet), the

clinician administering the assessment may wish to also observe a medium upper extremity-

based task (e.g. washing dishes) because the two tasks may reveal different performance

abilities. Moreover, if clinicians are aware of these underlying task type constructs, it can assist

them in identifying and summarizing the specific functional limitations of individual clients.

In terms of whether the assessment can be administered reliably by different individuals,

very preliminary analysis is promising, suggesting it can be used by student occupational

therapists as well as clinicians with many more years of experience. Results from our interrater

comparison showed generally small differences in scores between raters for most participants.

The outlier that was excluded from our calculations was only the second assessment performed

by the student rater. Thus the difference in scores might be attributable to a learning curve effect

which has been observed among novices (Vroman & Stewart, 2014). In order to minimize future

differences in scoring, a more explicit standardization of the assessment (e.g. the creation of an

administration handbook) would be beneficial, particularly for new or emerging clinicians and/or

clinicians outside of CTC. Further research should also be conducted to enable the calculation of

formal statistics on the assessment’s interrater reliability.

With respect to generalizability, our results are promising in terms of the CTC

Homemaking Assessment’s use as a more global measure of disability for the occupation of

21

homemaking specifically. Results of this study suggest that the CTC Homemaking Assessment

can discriminate between a range of performance abilities, as the total weighted scores were

normally distributed. In addition, the assessment has been used with clients with a wide variety

of diagnoses and for a variety of referral reasons. This is unlike many other assessments that are

designed for administration with a very specific population or are more general but, to date, have

only be tested with a subset of potential populations (Law, 1993).

Though our study’s sample included a diversity of diagnoses, referral reasons, gender,

and ages, not all aspects of these were significantly represented in the sample population. Our

sample included primarily female participants. This was not unexpected because the assessment

is used mainly to determine the need for homemaking supports after an MVA. In BC, an

individual is only eligible for these supports if they were the primary homemaker prior to their

accident (Insurance [Vehicle] Regulations, 2014). Thus many individuals assessed are women.

In addition, the largest number of referrals in our sample was for individuals injured in an MVA,

closely followed by cost of future care assessments. Finally, diagnoses of pain, fracture/joint

issues, and/or soft tissue injuries were most frequently observed within the study’s sample. In

short, our study’s findings should be considered most generalizable to women, those who have

experienced an MVA, and for individuals with pain and/or musculoskeletal injuries or

conditions.

Study Limitations

The CTC Homemaking Assessment has been modified over time, and therefore the

retrospective sample included results from two slightly different versions of the measure. Most

of the assessments included in our study were from a previous version, meaning that some items

on the more recent version could not be included in our analysis due to a limited number of

22

cases. Therefore this study cannot make conclusions as to the effect of these additional variables

on the assessment’s factor structure and internal consistency. The sample size for our exploratory

factor analysis also limited our ability to make conclusive recommendations about altering the

structure of the CTC Homemaking Assessment. In addition, the sample size for the prospective

phase of this study did not allow us to make statistically significant conclusions about the

assessment’s interrater reliability. Further data collection to establish a larger overall sample

would ensure that the CTC Homemaking Assessment is generalizable to both men and women,

and would allow for a more conclusive factor analysis. If more data is collected using the current

version of the assessment, this would also allow for analysis of the additional variables that had

to be excluded by our study. Finally, further research could be helpful for determining interrater

reliability.

Conclusion

The CTC Homemaking Assessment is an ecologically valid, client-centered, and

occupation-based measure that fills an identified gap in the tools available to occupational

therapists to evaluate disability in a specific IADL domain. Its unique focus on homemaking

tasks and the provision of a percentage score have made it a valuable tool for clinicians working

in the legal system and with third party insurers. The results of our study provide preliminary

support for the psychometric integrity of the CTC Homemaking Assessment, suggesting that it is

a valid and generally reliable measure of homemaking ability, particularly among women and

those who have experienced an MVA. Given the increasing number of MVAs in Canada, the

need for such an assessment is only likely to continue to grow, especially within occupational

therapy private practice. With further study, the CTC Homemaking Assessment has the potential

23

to also be applied in other clinical settings to facilitate conversations with clients about their need

for supports, and to enhance the consistency and credibility of home support decisions.

Key Messages

● The CTC Homemaking Assessment is a novel, ecologically valid tool that can be used to

quantify a person’s ability to complete homemaking tasks ranging in difficulty and

performance demands.

● The assessment can identify a range of performance abilities on homemaking tasks and

occupational therapists with varying degrees of experience can administer the measure.

● Preliminary evaluation of this assessment demonstrates promising reliability and validity,

particularly when used with women and those who have experienced an MVA.

24

References

Bottari, C. L., Dassa, C., Rainville, C. M., & Dutil, E. (2010). The IADL profile: Development,

content validity, intra- and interrater agreement. Canadian Journal of Occupational

Therapy, 77(2), 90-100. doi:10.2182/cjot.2010.77.2.5

Clemson, L., Bundy, A., Unsworth, C., & Singh, M. F. (2009). Validation of the modified

assessment of living skills and resources, an IADL measure for older people. Disability &

Rehabilitation, 31(5), 359-369. doi:10.1080/09638280802105881

Coman L, Richardson J. Relationship between self-report and performance measures of function:

A systematic review. Can J Aging 2006;25:253e270.

Crepeau, E. B., et al. (2014). Analyzing occupations and activity. In Willard & Spackman’s

Occupational Therapy, B.A.B Schell, G. Gillen, & M. E. Scaffa (Eds). Philadephia:

Wolter Kluwers Health/Lippincott Williams & Wilkins.

De Vriendt, P., Gorus, E., Cornelis, E., Velghe, A., Petrovic, M., & Mets, T. (2012). The process

of decline in advanced activities of daily living: A qualitative explorative study in mild

cognitive impairment. International Psychogeriatrics, 24(6), 1-13.

doi:10.1017/S1041610211002766

Fisher, A. (1993). The assessment of IADL motor-skills - an application of many-faceted rasch

analysis. American Journal of Occupational Therapy, 47(4), 319-329.

Gold, D. (2012). An examination of instrumental activities of daily living assessment in older

adults and mild cognitive impairment. Journal of Clinical and Experimental

Neuropsychology, 34(1), 11-34.

Kottorp, A., Bernspång, B., Fisher, A. G., Arbetsterapi, Medicinska fakulteten, Institutionen för

samhällsmedicin och rehabilitering, & Umeå universitet. (2003). Validity of a

25

performance assessment of activities of daily living for people with developmental

disabilities. Journal of Intellectual Disability Research, 47(8), 597-605.

Insurance Corporation of BC (ICBC). (2015, August 31). ICBC starts 2015 basic rate

application process. Retrieved from http://www.icbc.com/about-

icbc/newsroom/Pages/2015Aug31.aspx

Insurance (Vehicle) Regulation, (2014). B.C. Reg. 447/83. Retrieved from

http://www.bclaws.ca/Recon/document/ID/freeside/447_83_00

Kellar, S. P., & Kelvin, E. A. (2013). Munro’s statistical methods for health care research.

Philadelphia: Wolter Kluwers Health/Lippincott Williams & Wilkins.

Law, M. C., & MacDermid, J. (2008). Evidence-based rehabilitation: A guide to practice.

Thorofare, NJ: Slack.

Law, M. (1993). Evaluating activities of daily living - directions for the future. American Journal

of Occupational Therapy, 47(3), 233-237.

Law, M. (1993). Evaluating activities of daily living - directions for the future. American Journal

of Occupational Therapy, 47(3), 233-237.

Law, M., & Letts, L. (1989). A critical-review of scales of activities of daily living. American

Journal of Occupational Therapy, 43(8), 522-528.

Law, M. (1987). Measurement in occupational therapy: Scientific criteria for evaluation.

Canadian Journal of Occupational Therapy, 54(3), 133-138.

doi:10.1177/000841748705400308

Law, M. C., Baum, C. M., & Dunn, W. (2005). Measuring occupational performance:

Supporting best practice in occupational therapy. Thorofare, NJ: SLACK Inc.

26

Letts, L., Law, M., Rigby, P., Cooper, B., Stewart, D., & Strong, S. (1994). Person-environment

assessments in occupational-therapy. American Journal of Occupational Therapy, 48(7),

608-618.

Leung, D., Chi, I., & Leung, A. (2011). An evaluation of the factor structure of the instrumental

activities of daily living involvement and capacity scales of the minimum data set for

home care for elderly chinese community dwellers in hong kong. Home Health Care

Services Quarterly, 30(3), 147-159.

Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., . . . de

Vet, H. C. W. (2010). The COSMIN checklist for assessing the methodological quality of

studies on measurement properties of health status measurement instruments: An

international delphi study. Quality of Life Research, 19(4), 539-549. doi:10.1007/s11136-

010-9606-8

Myers, A., Holliday, P., Harvey, K., & Hutchinson, K. (1993). Functional performance-measures

- are they superior to self-assessments. Journals of Gerontology, 48(5), M196-M206.

Neistadt, M. E. (2000). Occupational therapy evaluation for adults: A pocket guide. Baltimore,

Md: Lippincott Williams & Wilkins.

Rogers, J., Holm, M., Beach, S., Schulz, R., Cipriani, J., Fox, A., & Starz, T. (2003).

Concordance of four methods of disability assessment using performance in the home as

the criterion method.

Arthritis & Rheumatism-Arthritis Care & Research, 49(5), 640-647.

doi:10.1002/art.11379

27

Reuben, D. B., Laliberte, L., Hiris, J. and Mor, V. (1990). A hierarchical exercise scale to

measure function at the Advanced Activities of Daily Living (AADL) level. Journal of

the American Geriatrics Society, 38, 855–861

Spanjer, J., Groothoff, J. W., & Brouwer, S. (2011). Instruments used to assess functional

limitations in workers applying for disability benefit: A systematic review. Disability &

Rehabilitation, 33(22-23), 2143-2150. doi:10.3109/09638288.2011.570413

Tarbell, M., Henry, A., & Coster, W. (2004). Psychometric properties of the scorable self-care

evaluation. American Journal of Occupational Therapy, 58(3), 324-332.

doi:10.5014/ajot.58.3.324

Vroman, K., & Stewart, E. (2014). Occupational therapy evaluation for adults: A pocket guide.

Baltimore, Md: Lippincott Williams & Wilkins

Wales, K., Clemson, L., Lannin, N. A., & Cameron, I. D. (2012). Functional assessments used

by occupational therapists with older adults at risk of activity and participation

limitations: A systematic review and evaluation of measurement properties. Systematic

Reviews, 1(1), 45-45. doi:10.1186/2046-4053-1-45

Yuen, H. K., & Austin, S. L. (2014). Systematic review of studies on measurement properties of

instruments for adults published in the American Journal of Occupational Therapy, 2009-

2013. The American Journal of Occupational Therapy : Official Publication of the

American Occupational Therapy Association, 68(3), e97. doi:10.5014/ajot.2014.011171

28

Appendix A - Tables

Table 1. Tasks (Variables) on Current Version of CTC Homemaking Assessment

Variable Subscale Included in Statistical

Analysis?

Handwashing clothes Light

Folding clothes Light √

Putting clothes away Light √

Dusting Light √

Cleaning mirrors Light √

Sweeping the floor Light √

Chopping, cutting, putting away food Light √

Drying dishes Light

Transfer clothes from washer to dryer Medium √

Take clothes out of dryer Medium √

Ironing Medium

Cleaning toilet Medium √

Cleaning sink Medium √

Cleaning baseboards Medium √

Cleaning windows Medium √

Washing floors Medium √

Making beds, stripping sheets Medium √

Washing dishes Medium √

Loading/unloading dishwasher Medium

Reach into upper and lower cupboards Medium

Vacuum Heavy √

Vacuuming stairs Heavy

Cleaning bathtub Heavy √

Cleaning oven Heavy √

Washing walls Heavy √

Cleaning fridge Heavy √

Cleaning freezer Heavy

Grocery shopping Heavy √

Taking out garbage Heavy √

29

Table 2. Demographics for Retrospective Sample (n=113)

Characteristic (n= 113) Finding

Age (years) Mean ± SD Range

50.6 ± 15.2

73

Gender Male Female

28 (24.7%)

85 (75.2%)

Reason for referral* MVA Cost of Future Care Medical-legal Fall Functional Capacity Evaluation Return to Work

80

73

21

7

2

5

Diagnoses* Pain Fracture/Joint Issues Soft Tissue Injuries Mental Health Sensory Changes Head Injury/Cognitive Issues Chronic Disease

69

49

48

42

34

26

16

*Note: Many participants had multiple diagnoses and reasons for referral

30

Table 3. Rotated Factor Matrix for CTC Homemaking Assessment

Factor 1:

Bending

tasks

Factor 2:

Upper

extremity tasks

Factor 3:

Clothing

tasks

Factor 4

Factor 5:

Load

tasks

Factor 6:

Planning

tasks

Vacuuming .531 .216 .242 .107 .111 .147

Cleaning fridge .530 .229 .271 .261 .041 .196

Washing walls .642 -.003 .037 -.013 .128 .175

Cleaning oven .515 .126 .035 .181 .065 .158

Cleaning bathtub .751 .121 .167 .073 .225 -.018

Making beds, stripping

sheets

.459 .273 .381 .178 .231 .030

Washing floors .660 .263 .036 .144 .107 .192

Cleaning baseboards .741 .080 .118 .092 .016 -.027

Cleaning toilet .483 .263 .017 .480 .104 -.013

Dusting .408 .564 .096 .162 -.092 .132

Washing dishes .080 .387 .342 .143 -.001 .143

Cleaning mirrors .222 .798 .143 .037 .173 -.081

Sweeping floor .472 .483 .120 .263 .054 .004

Folding clothes .208 .209 .701 .225 .191 .230

Putting clothes away .136 .119 .895 .096 .020 .023

Cleaning sink .140 .293 .194 .610 -.002 .079

Take clothes out of dryer .277 -.093 .265 .725 .438 .131

Transfer clothes from

washer to dryer

.152 .062 .024 .160 .880 .074

Taking out garbage .429 .197 .278 .013 .524 .110

Cleaning windows .299 -.054 .069 .027 .034 .588

Grocery shopping .207 .461 .269 .053 .254 .478

Chopping, cutting, putting

away food

-.100 .476 .191 .307 .110 .515

*Kaiser-Meyer-Olkin Measure of Sampling Adequacy = 0.807

**Bartlett’s Test of Sphericity: p value less than 0.000

31

Table 4. Interrater Comparison of Subscale Percentage Scores

Light Tasks

Medium Tasks Heavy Tasks

Client # Rater 1 Rater 2 Rater 1 Rater 2 Rater 1 Rater 2

A1 87.5 100 90 90 41.7 41.7

A2 100 100 100 100 100 100

A3 100 83.3 95.5 83.3 78.6 80.1

A4 62.5* 100* 40* 100* 33.3* 100*

A5 80 90 85 100 66.7 80

A6 100 94.4 100 100 100 92.8

A7 95 100 72.7 90 28.6 62.5

A8 94.4 94.4 85 80 83.3 83.3

A9 88.9 88.9 90.9 93.2 91.7 83.3

Mean difference: 6.3 ± 6.3

Mean difference: 6.5 ± 7.3

Mean difference: 8.1 ±11.5

32

Appendix B - Supplementary Materials

Supplemental Table 1. Subscale Correlation Matrix

LIGHT % MEDIUM % HEAVY %

LIGHT % Pearson Correlation 1 .649** .491**

Sig. (2-tailed) .000 .000

N 112 112 112

MEDIUM

%

Pearson Correlation .649** 1 .773**

Sig. (2-tailed) .000 .000

N 112 112 112

HEAVY % Pearson Correlation .491** .773** 1

Sig. (2-tailed) .000 .000

N 112 112 112

** Correlation is significant at the 0.01 level (2-tailed).

Supplemental Table 2. Interitem Correlation Matrix

Due to the size of this table it is available upon request to the authors.