jaakkopasanen
Member
- Joined
- Jul 12, 2020
- Messages
- 87
- Likes
- 343
I present to you abxtests.com, a browser based app for double blind AB and ABX listening tests. This is effectively Listening Tests as a Service, meaning you'll be able to create your own listening tests easily and share it with other people.
The primary target user would be an audio scientist or an audio engineer but there are also many audiophiles who are interested in testing things blind instead of trusting their biases. I started building this app for myself because I need to conduct some listening tests without being able to have the participants in the same physical location. Since I realized others could use this too, the expectations went up quite fast from something scrappy just for my own use to this what I'm now very comfortable to share with the public and put place under a public scrutiny.
ABX app runs users through a test suite with any number of AB and ABX tests. Each test can be repeated arbitrary number of iterations for statistical significance. I put a fair amount of effort into ensuring the listener's user experience is very good, audio playback is smooth and that the app itself doesn't introduce any biases. The test suite starts with a welcome screen where the test author can give context and instructions to the participants. The welcome screen also has a survey form which is handy for collecting data about different demographics.
The last one in the test suite is results screen where the user is presented with all the choices the user made, simple statistics and a p-value. The p-value is calculated as multinomial PMF. The results also include statistics aggregated based on tags in the audio samples. This makes it possible to examine how different filters, codecs, etc stack up against different songs.
The tests are defined as YAML text files. YAML is a super simple syntax and requires no prior experience. Simply create the file, save it to a publicly hosted location (such as Dropbox) and share a test link with the participants. The test links look like this: https://abxtests.com/?test=https://www.dropbox.com/s/c00dylhi7ekqkfw/demo.yml?dl=0. The last part is the Dropbox share link of the YAML file. You would replace this with your own. See the project's Github page for full details.
Here's an example of a minimal YAML file
You obviously need to have the audio samples prepared too. Fortunately they too can be hosted in Dropbox.
I'd greatly appreciate any feedback about the app!
The primary target user would be an audio scientist or an audio engineer but there are also many audiophiles who are interested in testing things blind instead of trusting their biases. I started building this app for myself because I need to conduct some listening tests without being able to have the participants in the same physical location. Since I realized others could use this too, the expectations went up quite fast from something scrappy just for my own use to this what I'm now very comfortable to share with the public and put place under a public scrutiny.
ABX app runs users through a test suite with any number of AB and ABX tests. Each test can be repeated arbitrary number of iterations for statistical significance. I put a fair amount of effort into ensuring the listener's user experience is very good, audio playback is smooth and that the app itself doesn't introduce any biases. The test suite starts with a welcome screen where the test author can give context and instructions to the participants. The welcome screen also has a survey form which is handy for collecting data about different demographics.
The last one in the test suite is results screen where the user is presented with all the choices the user made, simple statistics and a p-value. The p-value is calculated as multinomial PMF. The results also include statistics aggregated based on tags in the audio samples. This makes it possible to examine how different filters, codecs, etc stack up against different songs.
The tests are defined as YAML text files. YAML is a super simple syntax and requires no prior experience. Simply create the file, save it to a publicly hosted location (such as Dropbox) and share a test link with the participants. The test links look like this: https://abxtests.com/?test=https://www.dropbox.com/s/c00dylhi7ekqkfw/demo.yml?dl=0. The last part is the Dropbox share link of the YAML file. You would replace this with your own. See the project's Github page for full details.
Here's an example of a minimal YAML file
Code:
name: Minimal listening test
welcome:
description: |-
### Minimal
Minimal listening test
options:
- name: lossless
audioUrl: https://www.dropbox.com/s/9e92quf9tr7aj8s/Wintersun%20-%20Sons%20of%20Winter%20and%20Stars%20lossless.wav?dl=0
- name: 64 kbps
audioUrl: https://www.dropbox.com/s/epvit0keu9w7emp/Wintersun%20-%20Sons%20of%20Winter%20and%20Stars%2064%20kbps.mp3?dl=0
tests:
- name: Lossless vs 64 kbps
testType: ABX
options:
- lossless
- 64 kbps
repeat: 5You obviously need to have the audio samples prepared too. Fortunately they too can be hosted in Dropbox.
I'd greatly appreciate any feedback about the app!
