• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

abxtests.com - Listening Tests in the Browser

jaakkopasanen

Member
Joined
Jul 12, 2020
Messages
36
Likes
138
I present to you abxtests.com, a browser based app for double blind AB and ABX listening tests. This is effectively Listening Tests as a Service, meaning you'll be able to create your own listening tests easily and share it with other people.

The primary target user would be an audio scientist or an audio engineer but there are also many audiophiles who are interested in testing things blind instead of trusting their biases. I started building this app for myself because I need to conduct some listening tests without being able to have the participants in the same physical location. Since I realized others could use this too, the expectations went up quite fast from something scrappy just for my own use to this what I'm now very comfortable to share with the public and put place under a public scrutiny.

ABX app runs users through a test suite with any number of AB and ABX tests. Each test can be repeated arbitrary number of iterations for statistical significance. I put a fair amount of effort into ensuring the listener's user experience is very good, audio playback is smooth and that the app itself doesn't introduce any biases. The test suite starts with a welcome screen where the test author can give context and instructions to the participants. The welcome screen also has a survey form which is handy for collecting data about different demographics.

The last one in the test suite is results screen where the user is presented with all the choices the user made, simple statistics and a p-value. The p-value is calculated as multinomial PMF. The results also include statistics aggregated based on tags in the audio samples. This makes it possible to examine how different filters, codecs, etc stack up against different songs.

The tests are defined as YAML text files. YAML is a super simple syntax and requires no prior experience. Simply create the file, save it to a publicly hosted location (such as Dropbox) and share a test link with the participants. The test links look like this: https://abxtests.com/?test=https://www.dropbox.com/s/c00dylhi7ekqkfw/demo.yml?dl=0. The last part is the Dropbox share link of the YAML file. You would replace this with your own. See the project's Github page for full details.

Here's an example of a minimal YAML file
Code:
name: Minimal listening test
welcome:
  description: |-
    ### Minimal
    Minimal listening test
options:
  - name: lossless
    audioUrl: https://www.dropbox.com/s/9e92quf9tr7aj8s/Wintersun%20-%20Sons%20of%20Winter%20and%20Stars%20lossless.wav?dl=0
  - name: 64 kbps
    audioUrl: https://www.dropbox.com/s/epvit0keu9w7emp/Wintersun%20-%20Sons%20of%20Winter%20and%20Stars%2064%20kbps.mp3?dl=0
tests:
  - name: Lossless vs 64 kbps
    testType: ABX
    options:
      - lossless
      - 64 kbps
    repeat: 5

You obviously need to have the audio samples prepared too. Fortunately they too can be hosted in Dropbox.

I'd greatly appreciate any feedback about the app!
 

Katruje

New Member
Joined
Dec 31, 2020
Messages
2
Likes
0
Wow, this is really great work. Congratulations on what you've created. I don't think many other people here in the Forums understands how useful this can be. And fully open-source on Github to boot! Following the repo for sure. I hope other people start playing around with this.
 
Top Bottom