Computerized Cognitive Testing: The Good, the Bad and the Ugly

Baseline concussion testing is an often-cited solution to the concussion epidemic. Many believe that if a baseline concussion test is performed, it will serve as a foolproof marker to diagnose a concussion and subsequently identify when the athlete has recovered from a concussion. Part of the problem is that baseline concussion testing has become synonymous with computerized cognitive testing when, in fact, it is just one part of the concussion puzzle.

When considering baseline concussion testing, the National Athletic Trainers’ Association suggests including baseline symptom scores, physical and neurological evaluations, balance testing, and cognitive performance (1). Why? Each of these domains measures different functions. Measures of cognitive performance and balance are shown to be disproportionately affected by concussion and, therefore, can be used to assess an injured athlete and measure recovery.

More information is emerging about changes in eye movements after concussion. As such, programs are now including objective eye movements as part of their baseline concussion testing protocol, outside of the standard cranial nerve assessment. It is important to point out that these tests are best used in conjunction with each other, and not in isolation. Further, no single assessment should replace the clinical examination. As with any tool, it is important to understand the advantages and disadvantages of computerized cognitive testing.

The Good:

The gold standard for cognitive testing is a “paper and pencil” neuropsychological evaluation performed by a neuropsychologist. Unfortunately, there are neither the financial nor professional resources to have paper and pencil testing performed on large numbers of high school athletes. Most neuropsychological evaluations require a one-on-one presence in a neuropsychologist’s office, take hours to perform, and have a detailed interpretation of test findings. While this method is being performed in the National Football League and National Hockey League, it is impractical and too expensive to be done at the college and high school levels.

Computerized cognitive testing often includes measures of verbal memory, visual memory, attention, processing speed and reaction time. Computerized cognitive testing has the advantage of testing multiple individuals in a single setting, using a standardized format. Many computerized cognitive testing batteries have been around for many years and demonstrate a relative reliability, even in the setting in which a baseline test has not been performed and only a post injury test is available.

Another advantage of computerized cognitive testing is that an individual can establish a baseline, in which a post-injury comparison may be made to themselves, rather than the general population. As software-based tools, they are relatively inexpensive and can be repeated over time, in a non-clinical setting.

The Bad:

While computerized cognitive testing is inexpensive, it is not free. As a result, many programs turn to the Sport Concussion Assessment Tool (SCAT), which is now in its third version. It should be noted that this tool is best used in the initial assessment of concussion, such as the sideline or locker room, and decreases in reliability in the days following a concussion. Using the SCAT in conjunction with computerized testing for baseline and post-concussion testing is common.

There is a great deal of variability in post-concussion performance on computerized cognitive testing. Reasons for this include the fact that, unlike most baseline testing situations, post-concussion testing is done is isolation, which provides an ideal testing environment. Additionally, many athletes demonstrate greater motivation and concentration on post-concussion cognitive testing and sometimes exceed their baseline testing scores.

Administration of baseline testing is a critical factor in establishing valid computerized baseline and post-injury tests. When attempting to test too many athletes at a time, distractions arise and performance declines. Furthermore, administration of baselines by a non-healthcare provider is not appropriate and may result in liability issues.

Additionally, some athletes will attempt to “sandbag” their baseline test. Fortunately, many computerized cognitive tests have validity measures built in to identify invalid baseline tests (2). These same measures may also misidentify an invalid test in a child with a learning disability or attention deficit hyperactivity disorder (ADHD). In fact, baseline cognitive testing in these populations may be more valuable than in typical peers because post-injury scores compared to the general population may be less reliable.

The Ugly:

The misuse of computerized cognitive testing is problematic. Whether from ignorance or malfeasance, the misuse of baseline and post-injury concussion testing poses a health and legal risk to those who use the tools. It is important to recognize that computerized cognitive tests DO NOT DIAGNOSE CONCUSSION. As such, they should not be used in attempt to do so. The diagnosis of a concussion is a clinical decision made by a trained health-care professional.

Similarly, decisions for returning to athletics and returning to school should not be based on computerized cognitive tests alone. Like many tools in health care, they are used to assist in making judgements and recommendations about the health of the athlete. Self-administration of baseline and post-injury testing should also be strongly discouraged. Unwitnessed testing performed by other athletes or other family members has been reported in situations when athletes were permitted to self-administer computerized cognitive testing at home.

Final Recommendations:

Baseline and post-injury concussion tests can be useful tools when performed appropriately. Schools that employ the use of baseline concussion testing are less likely to return an athlete to play within 10 days (3). Computerized cognitive testing should be used in conjunction with other measures of neurological function and symptom reporting. Baseline testing can be done in small groups, but should be proctored by health-care providers using guidelines provided by the manufacturer and limit the number of participants.

Selection of which athletes to baseline test should be made by a health-care professional, such as an athletic trainer, but may be based on the incidence of concussion in a given sport. Attention should be made to male and female sports alike. The frequency of performing baseline tests is up for debate, but most recommend doing so every one to two years. Increasing that frequency in younger athletes should be considered based on the increased rate of cognitive development in youth. Care should be taken to ensure that baseline and post-injury tests are valid and should be repeated, if they are invalid. It is critical that such tests are not used to diagnose concussion, but are used as a tool to assist in the evaluation and management of an athlete determined to have a concussion. 

Citations:

1. Broglio SP, et al. National athletic trainers’ association position statement: Management of sport concussion. Journal of Athletic Training. 2014;49(2):245–265.
2. Schatz P. Glatts C. Sandbagging baseline test performance on ImPACT, without detection, is more difficult than it appears. Archives of Clinical Neuropsychology. 2013 May;28(3):236-44.
3. Meehan WP, et al. Computerized neurocognitive testing for the management of sport-related concussions. Pediatrics. 2012;129:38–44.