These are two questions I’ve been exploring for the last three decades. Initially because I had a hard time staying focused and ultimately because I saw such huge changes in my (and others’) focusing abilities listening to certain types of music.
How do you validate it, though?
This became the persistent question I confronted these last few years, especially after accumulating tons of data and thousands of positive user experiences.
After the launch of my streaming music site, Brain Shift Radio where we noticed that almost half of the listening done was for focus, I began exploring quantitative measures that we could use to see unequivocally whether music can contribute to enhanced focusing abilities.
Exploring Attention Tests
As it turns, out two independent studies recently conducted in the Netherlands on REI (the core technique behind my music) used Continuous Performance Tests (CPTs) to test attention. Both studies showed significant improvements while listening to my music versus either a placebo recording or silence.
So we looked closely at CPTs and they seemed like a good option. I was drawn to these types of tests for several reasons:
- Although they are not used to diagnose attention deficits, CPTs are gaining ground as reliable ways to test attention. There are a handful of CPTs being used in research and for evaluating treatment approaches for people with attention issues. These tests are also computer-based – our platform of choice.
- These tests are pretty straight-forward, so designing a test that we could integrate with Brain Shift Radio was fairly easy (especially compared to the software we have designed in the past). As long as we dealt with the limitations and shortfalls that often accompany tests of this sort we could get quality data.
- These tests can be done remotely and can scale to any number of users. This would allow us to collect more data on the effects of music on attention than had ever been done before. The challenge here is getting people to our website and engaging them in the test itself. The only real drawback is that the test-taking process requires about 20 minutes.
- We would be able to track, compile, and interpret a wide breadth of data. Not only could we track clicks and taps but also, through a simple intake form, personal profiles (anonymously, of course). We could also track and compare user, location, and platform data (again anonymously).
- As an added benefit, the type of data collected from these tests can be integrated into our music-selection algorithms. This means that we can use this test data to give our listeners better mixes (assuming they take the test and attach it to their account).
Building the Software
Continuous Performance Tests come in a few varieties. You have auditory and visual, and you have different types of triggers. The auditory tests were off the table since we would be examining the effect of music on attention and the music would compete with the auditory trigger. This left us with choosing different visual trigger options.
Like all our software, we built the framework first, leaving ourselves open when it came to choosing trigger options. We tested a ton of different triggers and decided to offer two types: symbols or numbers. Both use the same timing, variability, and duration. Where they differ is in how quickly people are able to process the stimulus. Symbols – in our case various colored squares – are easier/quicker to process, whereas a series of six numbers takes more cognitive resources.
This makes the symbol test a little easier and also makes it a good place to start. The numeric test will result in higher error rates than the symbol test but, as we’re seeing after a few hundred tests, the relative changes between the silence and music portions of the tests are virtually the same.
Here’s a quick breakdown of our two tests types:
Symbol test (left, top image): This test displays a series of 6 randomly colored squares. When a white square appears in the group, tap the left button or the left arrow key on a keyboard. When a black square appears in the group, tap the right button or arrow key. If neither white nor black appear (pictured) you do nothing.
Numeric test (left, bottom image): This test presents a series of 6 numbers. If a 0 appears in the group (pictured), you tap the left button or left arrow key on a keyboard. If a 1 appears, you tap the right button or arrow key. If neither a 0 or a 1 appear, you do nothing.
Pretty simple. The trick is that speed is important. You need to act quickly because the stimulus only shows for a short time.
Taking the Test
Our attention tests are divided in four sections: An intake questionnaire, a test with silence, an intermission, and a test with music. Here is a breakdown of each of these sections:
- Intake questionnaire: The intake questionnaire is short, just eight questions. These include gender, age, history of ADD (ADHD, AD/HD), medication use, and perceived focusing abilities.
- Test with silence: This test is roughly 7.5 minutes and gives us a baseline for your focusing ability. We ask that you not have any other music playing while taking this test.
- Intermission: Right after the first test ends we start the Brain Shift Radio music. This music is chosen for you based on your answers to the intake and your performance with the silence test. The intermission lasts for 4 minutes to give your brain a chance to respond to the music.
- Test with music: The music we started during the intermission will continue as you complete the second portion of the test. This is the same length as the test with silence.
Interpreting Your Results
Once you’re finished with the entire test, we tally your scores. You will find two graphs, each presenting different data sets.
Raw Scores (left). The first are your raw scores. Here you will see each error type – detection, commission, and omission – and your response times – fastest, slowest, and average. Here’s what each error type means:
- Detection error: This error appears when you click the incorrect answer. In the case of the colored square test this would be when you click white when black was presented. In the case of the numbers clicking a 1 when a 0 is displayed results in an error. Errors of this sort can indicate inattention or impulsivity, depending on the speed of the click. For example, faster clicking may suggest impulsiveness whereas slower clicking may indicate inattention.
- Omission error: This is when you don’t respond to a slide displaying either positive response condition (a white/black box or 1/0). Omission errors can suggest inattention or slow cognitive processing.
- Commission error: A commission error occurs when you click either trigger when neither of the positive response conditions are present. In the case of the colored squares, neither black nor white appear and, for the numbers, the display is void of either a 1 or 0. Commission errors suggest impulsivity or inattention. Like with detection errors, fast clicking may indicate impulsiveness. Slow response may indicate inattention.
- Fastest click: This field shows your fastest click in tenths of a second.
- Slowest click: This field shows your slowest click in tenths of a second.
- Average click: This is the average speed of your clicking in tenths of a second.
This data can give you insights into your performance and whether you trend toward being inattentive or impulsive.
Weighted Scores (below). The second score is your weighted score. This balances your various errors against one another and factors in your response times.
Weighted Scores show you how you focus with and without BSR music. Lower numbers mean better focus.
The difference between these scores shows your performance, or more accurately, the music’s ability to help you perform.
In most cases, you should see a lower score for the music portion of the test, showing fewer errors and faster response times (as shown at right).
We’re finding that there are instances when a test-taker has not heard the BSR music before that his scores are not better on the music portion of the test. Past research has suggested that it can take up to 17 minutes for people with no experience with REI to see benefits.
To test this theory we are asking people who scored as well or better in the silence condition their first time that they either take the test again or listen to Brain Sift Radio for a couple of days before taking the test a second time.
Based on people’s responses to these two approaches so far, we’re seeing that the second test usually falls inline with the results we see from people who do respond positively to their first test.
Ensuring Quality Results
We worked very hard to make the tests accurate by addressing the deficits and limitations that many attention tests have. Here is a look at the major areas we had to address:
- Improvements from learning: We made sure the test is simple enough that there is no need to develop skill to perform well. We also offer a practice test for you to get used to the process and we throw out the first triggers of each test. This means that you won’t see improvements because you got better at the required task.
- Sufficient duration: We made each test long enough that it forces you to focus. One of the keys for these tests is that you do them long enough that the novelty wears off, requiring you to work to stay attentive. This is why it takes a total of twenty minutes to take our tests.
- Randomization and unpredictability: We randomly varied the presentation of objects within a consistent framework. This ensures that you can’t memorize or predict the pattern while also keeping the various elements balanced.
- User tracking: We are able to track if someone has been listening to BSR and whether someone has taken the test before and what their results were. This means we can interpret their data according to their experience with BSR.
- Music Tracking: As part of the intake and silence condition test process we choose the Brain Shift Radio music based on performance and answers to the intake answers. This will help see, not only whether BSR music can increase attention, but also if there are differences based on variations of the music. And when we have enough data we’ll be able to see how the music affects different gender and age groups as well as those diagnosed with attention deficits and/or those on medications.
Because of the databases and framework we have set up, we have the ability to parse the data in a variety of ways and hopefully see some patterns that we may not even consider at this time.
We have also built this test with the ability to rotate different music, even (gasp!) someone else’s.
Incorporating Your Scores into Brain Shift Radio
The Brain Shift Radio CPTs are for more than just showing you how our music can help you focus. We are also using the data to help us understand how you focus so we can better choose great mixes for you. This data can be stored in your profile if you wish and we will draw from it whenever you choose the Focus or Brain Boost categories in Auto Select mode.
If you log into your account (trial or subscription) when you take the test, we’ll keep track of all your results and you can periodically take the test again to track your progress and make sure you’re getting all you can from BSR.
The Brain Shift Radio attention tests are compatible with both web and mobile devices.