news
We recently offered an exclusive educational session on how to discover the power of digital assessments in brain health programs. This session was hosted by one of our research scientists Dr. Sc. Manuela Adcock. During this session Dr. Adcock spoke about the advantages of digital assessments, why quality criteria are vital when evaluating assessment tools, and why assessments are fundamental tools for any brain health program. If you wish to develop or strengthen standardized procedures to objectively measure the outcomes of your residents, patients, members, and clients, we highly recommend reading and/or watching this session.
Dr. Adcock’s background includes clinical work as a neuropsychologist in the diagnostics of brain diseases and many years of experience as a researcher in the field of neuroscience. In both areas, there is one thing that she has found to be crucial: to assess brain functions and performance. Especially when we talk about helping a diseased or aging brain with training and therapy in a brain health program, assessments are fundamental tools we need. Basically, this is not only valid/true for brain health programs but for any form of health interventions. With that, let’s dig in and discover the power and advantages of assessments, especially digital assessments.
What are assessments?
Assessments (often called “tests”) are standardized procedures that objectively measure performance (for example reaction time or balance or muscle strength).
Why are assessments used?
According to Dr. Adcock, assessments should be an essential part of any training and therapy measure and especially brain health programs. But why?
Standardization and accuracy: Assessments ensure standardization and accuracy. Assessments are valuable tools to accurately measure performance with a standardized and thus repeatable and comparable procedure.
Education and awareness: Assessments provide information about a person’s performance in a specific cognitive or physical function and therefore educate us about his/her health status (which is very crucial to know if we want to help this person and create for example an individually tailored brain health program). Moreover, education and awareness about one's own health status is also very crucial and important for the patient/client/resident. Assessments educate a person about his/her functional status and might serve as a kick-start for the intrinsic motivation to take care of their own health situation (i.e. getting better, more active, etc.).
Individually tailored training/therapy: At the beginning of a training period, assessments might help to define the current state of a trainee (meaning to understand his or her physical and/or cognitive status). Based on the results of these initial assessments (or so-called baseline measurements), training/therapy/brain health programs can be tailored specifically to the trainee or patient. For example, an assessment in a specific cognitive function (i.e. in attention or memory) shows low performance, meaning a weakness of this trainee/patient, so the individually tailored training program might include a focus on this specific function.
Progress monitoring: After a certain training period (i.e. several weeks), the assessments might help to evaluate training progress. Thus, the assessments which have been performed in the beginning of the training should be repeated in order to map the training success. Information about training progress might not only be important for the trainees/patients themselves but obviously also for their trainers and therapists to understand the success of the implemented training program and to further individually adapt the training. Moreover, information about the training progress and a program’s success might also be important for service providers and other stakeholders of the healthcare system.
Intrinsic motivation and compliance booster: Finally, let’s talk about the huge advantage and power of assessments in boosting motivation and compliance. Training progress measured by an assessment, especially when somehow visualized and easily understandable, might serve as a huge booster for further training motivation and compliance. It’s fulfilling to see that your hard work pays off and progress is visible and measurable.
Why not use training tasks as assessments?
Dr. Adcock often times gets asked “why not using a training task to track progress”? It is understandable that if a person committed to a training program repeats a training task many times, progress and improvement might be visible in this task. But keep in mind that we strongly recommend not to use a training task itself to evaluate a training effect, instead we recommend that you use a non-trained task to truly measure the transferable training success. This recommendation is mainly based on two reasons:
The goal of training is not only an improvement in the trained tasks but mainly a transfer effect of the functional improvement to untrained tasks, which is what the assessments represent. Meaning, it’s not surprising that we see an improvement in a training task that has been repeated again and again. The idea behind most training approaches is to train some targeted and “underlying functions” (i.e. some cognitive or physical functions). For example, you want to improve not only “game scores” but perhaps reaction time or balance. Thus, tasks other than training tasks should be used to evaluate this training success. To summarize, in order to measure the transferable training success, assessments are our recommended method.
Moreover, some training tasks are adaptive and change depending on the performance of the trainee (for example many computer-based or other digital training tasks have integrated algorithms which adapt the task and difficulty level mostly real-time to a trainee’s performance). Thus, they are not standardized procedures and results from one training session are not directly comparable with the results from another session/repetition of the task. Again, as assessments are standardized procedures, their results are comparable when they are repeated.
What is a good assessment?
When we talk about assessments, there is a wide range of different assessments and not all of them are equal. A good assessment fulfills some quality criteria. Quality criteria are a set of characteristics that are used to assess the quality of measurement procedures. The main quality criteria are: Objectivity, reliability and validity. In the course of the development of an assessment, studies and testing need to be performed to ensure that a newly developed assessment fulfills these criteria. Let us quickly explain the main quality criteria.
Objectivity refers to the independence of a measurement result from the person performing the measurement. Meaning that if you or your colleague perform an assessment with a patient should not have an influence on the result of the assessment.
Reliability means the consistency and stability of a measurement result over time and across different situations. Example: When an assessment is conducted twice (so called test-retest) and the results are compared (no training or therapy in between) -> the results should be more or less the same, meaning there should be a strong correlation if the assessment measures the proposed functions reliably.
Validity means that the test measures what it is supposed to measure. Example: When an assessment is supposed to measure balance, you can compare its results to a so-called “gold standard assessment” for balance, a well-known and well-established balance assessment -> results should be similar if the assessment measures the proposed function.
If you have options, always choose assessments which fulfill these quality criteria meaning they have been tested to fulfill these criteria. You usually will find this information in the user manual.
Some practical considerations: HOW should assessments be conducted?
Carry out assessments exactly as instructed and always under the same conditions
Educate and familiarize yourself with the procedure first
If the instructions are orally given, use the correct wording
If you need to calculate assessments scores, double check that you did so correctly
If you have the option, control the time of the day when assessments are performed
Create a comfortable environment for the patient with minimal distraction
Repeat assessments for progress monitoring but do not repeat them more than necessary
How can assessment results be interpreted?
Interpreting assessment results involves analyzing the data collected during the assessment process and making sense of what the scores or outcomes signify. This interpretation can provide valuable insights into an individual's performance, capabilities, and potential areas for improvement. One important aspect of interpreting assessment results is the use of a “frame of reference” or so called “reference norms”, which are standards or benchmarks against which an individual’s performance is compared. Basically, we distinguish between an individual and a social reference frame/norm.
Individual reference norm:
Basis for comparison: Individual reference norms compare an individual's current performance to their own past performance. Meaning the individual is compared to him/herself which we call an “intraindividual comparison”. The focus is on personal growth, progress, and development over time.
Goal: To track an individual’s improvement or change in abilities/skills (i.e. after a training period).
Social reference norm:
Basis for comparison: Social reference norms compare an individual’s performance to the performance of a select group or population. The focus is on how the individual’s performance compares to external standards. The creation of a social frame of reference is called normalization with the result of so-called norm tables.
Goal: To understand how an individual’s performance ranks relative to others (i.e. evaluating if an individual performance is above or below average performance of the comparison group).
What are the advantages of digital assessments?
Depending on the specific function you want to measure, you have a wide range of assessments to choose from. In today’s world, a decision you get to make is if you prefer to use so-called “paper-pencil tests” or digital assessments. Digital assessments involve the use of technology for the administration, scoring and analysis of assessments. So, what are the advantages of digital assessments?
Efficiency: Digital assessments can be administered and scored more efficiently than traditional assessments. Automated processes save time in terms of test administration and automated scoring/evaluation.
Error minimization: The standardized and automated procedures minimize execution and calculation errors.
Immediate feedback: Digital assessments allow for immediate feedback to participants. This quick turnaround can enhance the learning process and provide motivation.
Data storage: The measurement results can be automatically documented and stored.
Cost-effectiveness: Over the long term, digital assessments can be cost-effective. While there may be initial investments in technology and software, the reduction in paper, printing and manual grading costs can lead to overall savings.
Environmental impact: Digital assessments contribute to environmental sustainability by reducing the need for paper and minimizing the environmental footprint.
Security features: Digital assessments can incorporate advanced security features to prevent cheating or unauthorized access.
However, digital assessments need to be designed to be very user-friendly and enable safe and easy use. With this understanding, we developed our own research-based digital assessments, which can be performed on our Dividat Senso.
Dividat assessments
Considering the fundamental importance of physical-cognitive interplay for daily life, we strongly recommend utilizing assessments that require physical and cognitive resources. Dividat has developed such assessments in collaboration with ETH Zurich. These assessments can be conducted on the Dividat Senso. Each assessment focuses on a specific physical and cognitive function (i.e. balance or selective attention). The newly developed assessments were tested for compliance with quality criteria in studies conducted at ETH Zurich. The results from the assessments are integrated into the Senso’s digital case management system, Dividat Manager. This set of performance metrics gives a comprehensive view of a person’s physical and cognitive status. Moreover, they are designed to be very user-friendly and very easy to use, no need to be an expert for digital tests.
What is the advantage of measuring brain functions “in motion”?
Coming back to brain functions and brain health programs, many well established test procedures for measuring cognitive functions are paper-pencil procedures or computer-based procedures in which the test person sits at a computer. Analogous to our approach of cognitive-motor training, we also consider it highly relevant to measure brain performance "in motion", since our daily life is based on cognitive-motor interaction and thus cognitive-motor assessments have a higher ecological validity.
In summary, why should YOU use (digital) assessments?
Standardized procedures to objectively measure performance
Baseline measurement to create/adapt training
Repeatable assessments to monitor progress
Motivation and compliance booster
Efficient and easy to use
Crucial component o any health interventions and brain health programs