Community nurses are key to managing long-term conditions in patients by delivering care in people's own homes. They use a wide range of skills to assess, diagnose and manage a multitude of conditions, including wounds, pains and symptoms experienced at the end of life, to help maintain an individual's independence (NHS England, 2015a). Evidence from some high-profile enquiries and reviews attributes low staffing levels with adverse outcomes and poor patient experience (Francis, 2010; Keogh, 2013; Griffiths et al, 2016). Therefore, as patient and population needs change, new and innovative approaches to healthcare and support systems are needed (Department of Health and Social Care (DHSC), 2013). This care is becoming more complex, with increasing levels of acuity and dependency in patient care (McDonald et al, 2013), and a corresponding need to undertake caseload management (Reid et al, 2008). Caseload management has been defined as a management system in which care is delivered to a defined caseload according to patient need, while ensuring equity and cost-effectiveness (Pye, 2011). Bain and Baguley (2012) viewed caseload management as a way to ensure that patients' individual needs are met by an appropriately trained clinician at the right time. The focus on caseload classification within community nursing is reinforced by government policy that prioritises care closer to home (DHSC, 2012; NHS England, 2015b) and the provision of services that enable community care. However, there is also an apparent paradox between health policies that promote more care within and closer to home and the reported decline in community nursing services.
Workforce planning is integral to delivering efficient and high-quality services, and a number of methods have been described to undertake this, including professional judgement, population and health needs-based methods, caseload analysis and dependency-acuity methods (Reid et al, 2008). The National Quality Board (NQB) improvement resource for district nursing services (David and Saunders, 2018) recommends that a robust method be used for classifying patient acuity, frailty and dependency, and it sets out the requirements of an effective caseload tool. This process should help identify a safe caseload and aid in planning future workforce requirements, in terms of skill mix and capacity.
A literature review found that there was no single validated tool to distinguish caseload classification (Roberson, 2016), but consideration should be given to caseload analysis, measurement of workload and allocation and workforce planning. It has been stated that caseload profiling is a sub-set of caseload management (Harper-McDonald and Baguley, 2018), and this has been defined as a process to describe the number of variables in a community nursing caseload to show the complexity and type of the caseload (Kane, 2009). However, a systematic review of caseload profiling also found confusion as a number of different terms are used interchangeably in the literature, including ‘caseload analysis’, ‘profiling’ and ‘audit’ (Harper-McDonald and Baguley, 2018).
In Sheffield, an audit undertaken in 2016/17 found an average of 59 964 face-to-face contacts per 100 000 population by community nurses, representing an increase from an average of 1482–1541 contacts per whole-time equivalent nurse in the previous year (NHS Benchmarking Network, 2018). Once on the caseload, each service user was seen approximately 21 times for around 5 months, with the most common activity being wound care, followed by administration of medication (NHS Benchmarking Network, 2018).
As nursing caseloads have a wide range of complexities, there is a need to determine the most appropriate skill mix to deliver the right care. At present, there are no clear tools to determine staff capacity and skill mix in these community settings. To help understand and manage demand in a busy city-wide service, the Sheffield Caseload Classification Tool (SCCT) was co-produced by a group of community nurses and managers (Chapman et al, 2017). It was designed using a nominal group approach to define areas of care and the levels of complexity of care needs (Figure 1). The staffing skills required to deliver care within each group were also considered, based on grade and level of assigned responsibility. The tool consists of 12 care packages and three complexities of care (Table 1). A manual was produced with a series of working examples to help nurses standardise their responses when defining the level of complexity of care needed by each patient. The nurse classifies the patient's nursing need in terms of the area of care (i.e. care package) required and the level of complexity. The latter is based on a number of factors related to wellbeing, including the social situation of the patient. The data are then entered onto the electronic patient record, allowing the live daily capture of interventions and patient need across the entire community nursing caseload.
| 1 | Administration of intravenous medication |
| 2 | Administration of medication |
| 3 | Bladder management |
| 4 | Bowel management |
| 5 | Ear care |
| 6 | End-of-life care |
| 7 | Eye care |
| 8 | Long-term conditions and holistic care planning |
| 9 | Palliative care |
| 10 | Prevention, treatment and management of pressure ulcers |
| 11 | Tracheostomy care |
| 12 | Wound management |
Once designed, the SCCT was then piloted by 70 nurses, healthcare support workers and administrators, who were trained to use the tool to categorise over 3000 patient nursing needs during their assessment. Evaluation of the pilot results demonstrated that it was possible to use the tool to organise community nursing caseloads according to the complexity of patients' conditions and the needs of each patient while making use of an established electronic patient record (Chapman et al, 2017). The tool supports service leads to gain a more detailed understanding of the community nursing caseload, enabling the articulation of demand and complexity.
Aims
The purpose of the present study was to test the inter-rater reliability of the SCCT to ensure that the tool was robust.
Methodology
A one-day table-top validation exercise was designed using a handbook containing 69 different case studies to cover all the packages of care and complexity of need. These case studies were produced by nurse teams across Sheffield and were based on real examples (Figure 2).
Participants
Six community nurses from the same hospital trust were recruited via an email invite sent to all the staff. They then assessed 69 different case studies using the SCCT and graded each case against two criteria: care package (1 of 12) and complexity of need (routine = 1, additional = 2, significant = 3). This group of nurses had not been previously involved in designing the tool. All of them provided written informed consent prior to commencing with the validation exercise. Each nurse also supplied information relating to the length of time (in years) since qualification and the length of time (in years) that they had worked in a community setting. Their responses were then compared against pre-determined answers. The raters involved in the study had varying levels of familiarity with the tool and the accompanying manual shown. This ranged from no previous experience to a full working knowledge and understanding of using the SCCT. All raters were provided with a copy of the manual on the day, but were given no further introduction to or training on use of the tool.
Analysis
The nurses conducting the ratings were considered to be representative members of the community nursing team; hence, the model derived was a two-way random model with results that could be generalised to other nurses. The internal consistency of the average rating of each case study provided by the nurses was measured using the Cronbach's alpha statistic.
Alpha values, measuring scale reliability, were also determined for the set of nurses with each nurse's ratings deleted in turn, in order to identify any nurses whose ratings were detrimental to the overall reliability measure. An alpha value derived from all the nurses except one, which is greater than the alpha value original calculated for the entire scale, indicates that the responses of that nurse may not be consistent with those of other nurses, as the omission of this nurse from the calculation results in an improvement to reliability. However, small improvements in reliability resulting from the exclusion of a particular individual must be weighed against the corresponding loss of information that would result from that individual's exclusion.
Fleiss's kappa statistic (an extension of Cohen's kappa statistic applied to the assessments of three or more raters) was determined to assess the reliability of the care package and complexity ratings for the case studies.
For each case study, the number of care package and complexity ratings given by each nurse that tallied with the pre-determined answers were analysed. Additionally, the extent of any correlation with years of nursing experience or years employed in the community was noted.
Ethics considerations
The study was approved by the ethics committee of Sheffield Hallam University.
Results
A total of 69 cases studies were assessed by the community nurse raters. The nurses had been qualified between 2 months and 15 years and had worked in the community for between 2 months and 9 years. All nurses except Nurse 1 had only worked in the community since qualification.
The average consistency coefficient (Cronbach's alpha) for the care package was calculated to be 0.979 (using average measures); this represented very good inter-rater reliability. Alpha statistics for the data with each nurse's rating removed in turn ranged from 0.972 to 0.983, with reliability being marginally improved only by the deletion of the ratings of Nurse 1 (deletion of the ratings from all other nurses resulted in a slight reduction in overall reliability).
Fleiss's kappa coefficient for the care packages was determined to be 0.771, indicating substantial agreement among the nurses, and it was 0.423 for complexity ratings, indicating moderate agreement among the nurses.
Analysis of the agreement with the pre-determined answers for the case studies revealed that, on average, 4.84 out of the six nurses (standard deviation (SD)=1.87) agreed with the care package assessment that were pre-determined for each type of care package. The only care package that did not achieve agreement by at least three nurses was care package 10, for which a mean of 3.50 nurses agreed with the pre-determined assessment. However, only two instances of this care package were included in the list of case studies. The care packages showing the best agreement with the pre-determined assessment were care packages 1, 4, 5, 8 and 12. For all of these, a mean of five or more nurses agreed with the pre-determined assessment (Table 2).
| Care package | Number of nurses agreeing with the pre-determined care package assessment (mean; SD) |
|---|---|
| 1 | 5.6 (0.447; 5–6) |
| 2 | 4.13 (2.20; 0–6) |
| 3 | 4.67 (2.13; 0–6) |
| 4 | 5.20 (0.748; 4–6) |
| 5 | 5.00 (1.73; 2–6) |
| 6/9 | 4.44 (1.81; 0–5) |
| 7 | 4.00 (1.88 (1–6) |
| 8 | 5.33 (1.49; 2–6) |
| 10 | 3.50 (2.5; 1–6) |
| 11 | 4.50 (1.38; 2–6) |
| 12 | 5.50 (0.824; 4–6) |
Some of the above-mentioned care packages were considered to exhibit a certain level of self-similarity. The degree of similarity did not necessitate merging of categories. It is reasonable to assume that ‘near misses’ (i.e. identification of a similar care package to that of the pre-determined assessment) could be considered greater alignment than identification of a care package very different to that of the pre-determined assessment. Such groupings were defined to comprise the following:
If ‘near misses’ were considered to be correct care package classifications, the mean number of nurse agreements would be expected to improve slightly. There were nine instances of clinician assessments of a care package as being Administration of intravenous medication instead of Administration of medication or vice versa. Similarly, there were four instances of clinician assessments of a care package as being Prevention, treatment and management of pressure ulcers instead of Wound management or vice versa. There was one instance of nurse assessments of End-of-life care as being Long-term conditions and holistic care planning. However, the total number of such ‘near miss’ assessments was less than 3.5% of the total number of nurse assessments made; hence, improvements to the above reliability levels would be expected to be marginal.
Analysis of the agreement with the pre-determined complexity ratings revealed that the mean agreement was better for those case studies pre-determined as having significant complexity (mean=3.95 of 6 correct assessments) than for those case studies pre-determined as being routine or having additional complexity (Table 3).
| Complexity assessment | Number of case studies with the predetermined complexity | Number of nurses agreeing with the pre-determined complexity assessment (mean; SD) |
|---|---|---|
| Routine | 25 | 2.52 (2.10; 0–6) |
| Additional | 24 | 3.17 (1.87; 0–6) |
| Significant | 20 | 3.95 (1.91; 0–6) |
The number of care package assessments given by each nurse that tallied with those pre-determined for the study ranged from 52 to 60 out of 69 (i.e. 75.4%–87.0% agreement), with a mean number of correct assessments of 55.7 (SD=2.49). The number of complexity assessments given by each nurse that tallied with the pre-determined ones for the study ranged from 29 to 43 out of 69 (i.e. 42.0%–62.3% agreement), with a mean number of correct assessments of 36.3 (SD=4.23).
Hence, the proportions of care package assessments that tallied with those pre-determined for the case studies were substantially higher than the proportions of complexity ratings that tallied with those pre-determined for the case studies, despite the greater number of options available in the classification of care packages (Table 4).
| Nurse | Number of case studies with care packages assessed the same as the pre-determined rating | Number of case studies with complexity rating assessed the same as the pre-determined rating |
|---|---|---|
| 1 | 52 (75.4%) | 35 (50.7%) |
| 2 | 57 (82.6%) | 43 (62.3%) |
| 3 | 55 (79.7%) | 36 (52.2%) |
| 4 | 60 (87.0%) | 36 (52.2%) |
| 5 | 56 (81.2%) | 39 (56.5%) |
| 6 | 54 (78.3%) | 29 (42.1%) |
There was no clear relationship between the proportion of correct care package assessments and the proportion of correct complexity rating assessments by a particular nurse, except that Nurse 1 scored the lowest correct number of both care package and complexity rating assessments. Additionally, there was no correlation between the number of correct assessments and the time since qualification or time spent working in the community.
Discussion
Effective caseload management ensures that patients receive the right care by a nurse of the appropriate grade at the right time. Community nursing caseloads are constantly changing in complexity and size, which makes it difficult to measure caseloads accurately. The SCCT, using 12 classes of need and three levels of complexity, has shown good inter-rater reliability, with a high level of concordance to patient needs or demands and a very similar understanding of acuity. It covers the main themes that have already been identified in a previous literature review of caseload management classifications, where the system that is used (preferably, electronic), workload, staff time available and skill mix, delegation process and prioritisation process should all be considered (Roberson, 2016).
The Quest acuity tools (David and Saunders, 2018) have also been developed to measure acuity and frailty in district nursing, but these consist of a number of different measurement systems and have been used to audit practice and assess gaps in nursing provision. By comparison, the SCCT can be used to determine the capacity and skill mix that are required to meet the community nursing caseload needs on a daily basis. However, as agreement was higher across the sample of nurses for determining which package of care compared with the complexity, more training and clearer criteria may be required in the future to help identify complexity more accurately.
The nurses differed considerably in terms of the length of time for which they had worked in the community, from a newly qualified nurse (2 months) to one who had worked in the community for 15 years. The level of concordance achieved despite the variation in this parameter reflects the simplicity of the SCCT and its application in practice.
Strengths and limitations
The method used to test the reliability of the SCCT was simple, using 69 written cases that were all based on real examples. However, a paper-based appraisal may not accurately reflect complexity of practice, and only six nurses carried out the validation, which is a relatively small sample. Further studies are needed to investigate the reliability of the tool across clinical settings, in order to determine whether the tool is reliable in different service contexts.
The SCCT classification is easy to apply and is integral to the electronic patient record, allowing live use and reporting. The tool is able to assess the dependency of patients and indicate the staff grade required along with the predicted duration of time needed and cost of care. Thus, it is a very valuable tool to determine clinical demand and skill mix in a community setting. Additionally, it lends itself to further research by allowing specific targeted investigation of the caseload population, for example, the use of the tool to focus on the severity and complexity of wound management in community nursing.
Conclusion
The SCCT demonstrated good inter-rater reliability when assessed for care packages for an individual patient and moderate reliability when assessed for the complexity of care. On the basis of these results, a number of modifications will be made to the accompanying guide to help nurses when using the tool, in order to further improve its reliability. This tool provides a method to prioritise and plan workload on a community nursing caseload, ensuring the most efficient use of staff time to deliver the appropriate care to patients with very differing needs.