Most of the current paper-based cognitive assessment tests were developed and validated in psychology, neuroscience, and cognitive science lab settings over several decades. Long term funding and some creative freedom in basic research provide an environment to prototype experiments and tests that can take many years to show results and validate outcomes.
Basic research labs also pioneered the first computer-based cognitive tests, usually with custom software developed from scratch along with customized hardware like special keyboards and touchscreens. This environment has created a cottage industry of sorts, with dozens (if not hundreds) of computerized tests for memory/recall, attention/vigilance, reaction time, lexical fluency, and executive function being created independently from one another.
A few examples of free open source computer-based cognitive tests include the Psychology Experiment Building Language (PEBL), and www.cognitivefun.net, a collection of flash based cognitive tests. Most individual university labs also continue to develop and provide free implementations of cognitive tests, usually for a specific research focus – UCSD’s Cognitive Development Lab offers several computerized cognitive tests that support its focus on early childhood cognitive and language development as one example.
This creative, cottage industry approach in the research world also means that there isn’t much standardization on test implementations, scoring algorithms, and reporting formats.
A higher level of standardization can be provided by several commercial cognitive test vendors. A representative list includes Psychology Software Tools, Inc., Brain Resource (IntegNeuro), CANTAB (by Cambridge Cognition), CNS Vital Signs, and CogState. There is a tradeoff, however, between greater standardization and loss of flexibility with computerized cognitive tests, which is why many research labs continue to opt for customized tools.