How KnowBrainer.com Tests Microphones & Soundcards
How KnowBrainer Tests Microphones & Soundcards
Depending on what microphone we are testing, we work with up to 20 different testers and utilize a combination of in-house and independent sources. Our tests are conducted in 3 phases with the 2nd phase tests consisting of several sub stages noted below:
Phase 1: Testing is conducted with a trained user profile to see if continued testing is warranted. Some microphones perform so poorly that they never go beyond Phase 1 testing. In most cases, our accuracy cutoff point is 96% but we may make an exception for a specialized microphone or a microphone that is so inexpensive that we have to consider it.
Phase 2: We create a new user profile for each microphone and with the exception of Bluetooth microphones, skip general training. We NEVER use a trained user file, save a test profile or even make corrections. Because most of our testers are computer enthusiasts, we usually select a few paragraphs from technical journals but leave out unknown brand names. Most end-users dictate in free-form style (choppy dictation) which will typically be less accurate than reading text which is considerably easier because it requires little thinking. Although reading text into NaturallySpeaking will typically produce slightly higher than average free-form dictation results, we never use familiar text such as the DNS training files or the popular Rainbow Passage because it would skew the tests even more. Although the Rainbow Passage contains nearly all English phonemes, we do not find it ideally suited for microphone testing because the passage is simply too familiar. If you read a paragraph 4 times into the NaturallySpeaking without correcting, you will most likely appreciate slightly higher accuracy by the 4th reading because you're becoming familiar with the text and your dictation becomes smoother. For exactly this reason, we always initiate our tests with an unfamiliar reading.
We read a 200 word passage into NaturallySpeaking and we do not count punctuation. The process is conducted 4 times in a very quiet environment and we deduct .5 points for each error.
We repeat the previous test 4 additional times in a moderately noisy environment and if the microphone is noise canceling enough, we will conduct 4 more tests in a noisy environment. We test in 2 types of noisy environments.
(a) The 1st environment consists of an office setting with numerous voices speaking.
(b) The 2nd environment consists of music played at various volume levels.
Phase 2 Summation: A moderately noise canceling microphone may do reasonably well when playing music but not as well with human voices because of the frequency range of the microphone. You can more or less estimate how well a microphone will do in a noisy environment from our noise cancellation ratings but the accuracy ratings are calculated by summing the scores of the 1st 4 tests which are conducted in an absolutely pristine quiet environment. It might be better to think of our accuracy ratings as a test tube example. You could probably reduce all of our ratings by about 2% in real-world use but what's important is that we keep all of our tests uniform. In other words a microphone that we rate at 97% may only perform at 95% for the average end-user but another microphone that tests out at 98% will have a high probability of testing out at about 96% for the average end-user. Unfortunately, vocal characteristics and ranges vary enormously so this is not an exact science.
Phase 3: Testing is designed to compensate for the placebo effect as we sometimes fall victim to shiny objects. Even though we have finished our review by the end of Phase 2, we simply can't trust our results until they undergo real-world testing which means using the microphone for all our dictations for a period of 2 weeks. It's amazing how many times we run into something that wasn't obvious in the first two tests or even the first few days. For example, we found the Nady Encore II had a tendency to leak audio into NaturallySpeaking which resulted in NaturallySpeaking refusing to acknowledge our dictation for about 10 seconds after turning the microphone back on. Sometimes it's the little things that you don't notice until you seriously commit to using a microphone. In addition, sometimes a system that seems perfectly comfortable during early testing turns out to be uncomfortable in the long run.