![]() |
KnowBrainer Speech Recognition | ![]() |
Topic Title: Hardware vs. performance of DNS 11.5 Topic Summary: Folding all variables constant except for hardware configuration shows differences in performance Created On: 08/12/2011 06:04 AM Status: Post and Reply |
|
![]() |
![]() |
- Chucker | - 08/12/2011 06:04 AM |
![]() |
![]() |
- MDH | - 08/12/2011 07:52 AM |
![]() |
![]() |
- R. Wilke | - 08/12/2011 09:47 AM |
![]() |
![]() |
- MDH | - 08/12/2011 10:03 AM |
![]() |
![]() |
- R. Wilke | - 08/12/2011 10:46 AM |
![]() |
![]() |
- R. Wilke | - 08/12/2011 09:39 AM |
![]() |
![]() |
- txlaw1 | - 08/12/2011 10:20 AM |
![]() |
![]() |
- R. Wilke | - 08/12/2011 11:04 AM |
![]() |
![]() |
- MDH | - 08/12/2011 01:09 PM |
![]() |
![]() |
- monkey8 | - 08/12/2011 03:28 PM |
![]() |
![]() |
- R. Wilke | - 08/13/2011 05:19 AM |
![]() |
![]() |
- drfred123 | - 08/13/2011 10:42 AM |
![]() |
![]() |
- R. Wilke | - 08/13/2011 12:21 PM |
![]() |
![]() |
- MDH | - 08/16/2011 10:48 PM |
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Hi all, Operating System: Chuck's system: Operating System:
To interpret these results consider that the smaller the number, the faster the performance of the transcription process. That is, 0.33 represents faster performance vs. 0.55. Rüdiger may wish to provide additional information or further explanation. However, I tried to keep this as simple as possible so that everybody understands how hardware effects performance with DNS. Also, I wish to thank Rüdiger for providing both the testing utility and the test files. Although the results were a joint effort, Rüdiger did most of the grunt work making it easy to conduct the test. Therefore, all kudos to be directed to him. Chuck Runquist Technical Project Manager VoiceTeach LLC Home of VoicePower®: Simply powerful, powerfully simple ![]() ------------------------- VoiceComputer: the only global speech interface. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Rudiger, Ingenious experiment--Kudos!!! Chuck, given that you have about 7 different systems, you should try it now on your lowest performing one, and one with a Sandy Bridge chip and post the results. Rudiger, it occurrs to me that we frequently see people posting here asking advice about which of two or more systems would be most recommended for use with Dragon. Your tool would be an excellent commercial product for evaluating this objectively, rather that making a best-educated (or ignorant) guess. Also, it would give objective evidence as to comparison of systems to get the best "bang for the buck" to see if differences are real, substantial, or just mostly hype. You should make this into a commercial product for these purposes. Although designed with DNS in mind, your tool gives more fundamental information that would be applicable to anyone, even those not intending to use DNS (but potentially heavy users in other regards), in terms of an objective evaluation tool guiding informed purchase choices of computers. Great work. Great tool. MDH
------------------------- |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Mark, thanks for the compliments. I think you are basically correct in that doing it this way could even meet industrial standards somehow, however turning it into a commercial product would involve things I wouldn't necessarily like to face. The underlying theoretical concepts are not new, and I don't own the technology being used. What I did though was putting the pieces together and make them work. Just as an aside, the idea occured to me just recently when thinking of ways to do an immediate comparison between version 11.0 and 11.5 after my first intuitive impression was that there is an improvement in performance in 11.5. In short, there isn't any, so it was merely some sort of placebo effect. Upon further investigating it, I discovered that mrec.dll (the modular recognizer) which is the core modul in here is the same version number and size in version 11.0 and in 11.5. Rüdiger
-------------------------
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quote: What I did though was putting the pieces together and make them work. Yeah, like for a car, or plane, or computer--all of which proved useful commercially. Quote: the idea occured to me just recently when thinking of ways to do an immediate comparison between version 11.0 and 11.5 Did you compare DNS v. 10.1 versus 11.5? (especially since v11 is a significant re-write). MDH
------------------------- |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quote: Did you compare DNS v. 10.1 versus 11.5? (especially since v11 is a significant re-write). No I didn't, much closer I compared 11.0 versus 11.5. I did this by creating two identical virtual machines (the second one being a copy of the first), stored them in the same location (well next to each other), and the only difference they had was 11.0 installed on one and 11.5 on the other. I then created a digital recorder profile on each of them, using the same audio recordings on both, and ran the tests. Like I said, I did this when I first thought that 11.5 is a little snappier than 11.0 in free form dictation, but that was just because I was willing it to be obviously. However, audio recordings never lie. Rüdiger
-------------------------
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Chuck, thanks once again for taking the time to participate in the testings, thus helping to provide evidence as to the usability of the underlying methods and materials, but also for being so kind to post the results upon my request. Much appreciated. Rüdiger
-------------------------
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Thanks so much, Rudiger and Chuck, for these tests and the significant improvement demonstrated with better equipment. This makes it easier for me to justify my next purchase of hardware when I upgrade from XP to Win 7. So how would you all break down and attribute the performance improvement between (1) the L3 cache being 0 for Rudiger and 12,228 KB for Chuck and (2) Chuck's memory being three times what Rudiger has? Inquiring minds want to know.
------------------------- DNS DPI Pro 15.3, Win10, i7-3770K @ 3.7GHz, 16 Gb, 465 Gb SSD Boot drive, 3-in-1 TableMike - KnowBrainer Pro 2017 - Thanks so much, Lunis. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quote: (1) the L3 cache being 0 for Rudiger and 12,228 KB for Chuck and (2) Chuck's memory being three times what Rudiger has? Since my processor is a dual core, I don't have any L3 cache because they don't have it. All I have is L2 cache for that matter. But you cannot immediately compare the amounts of L2 cache and L3 cache with each other, because the concept is different. However, on a rough estimation (as I could prove by running relevant tests recently) one might say that half the amount of L3 cache equates for double the amount of L2 cache, more or less. In other words, if you have an i5 processor with 3 MB of L3 cache, it will perform about as fast as my system with 6 MB of L2 cache. Therefore, getting such a system now wouldn't mean any improvement over what I currently have. As far as the amount of memory installed, it doesn't have so much influence, if any, on short term performance, given that you have a certain minimum (2 GB in XP, and 4 GB in Windows 7), when looking at processing single utterances, but it will pay when running many applications with lots of RAM use along with each other. Rüdiger
-------------------------
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quote: Chuck's memory being three times what Rudiger has? To clarify--only as related to his computer! MDH ------------------------- |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Rüdiger/Chuck
from a very personal point of view I would love to see this tool developed further and used as some sort of standard although I appreciate what you say Rüdiger in terms of your possible lack of interest in going through all the necessary formalities etc. For my sins, and as a quadriplegic over many years, I get asked to do expert witnessing fairly regularly where I have to recommend assistive technology equipment (in which DNS and supporting package features prominently) for the disabled and other medical conditions, and also to justify such suggested equipment and software. It is usually a case of persuading a judge or panel why an insurance company should be forced to pay for a top spec machine or heading in that direction. I continually have to persuade the authorities that a more powerful PC is of a major benefit in terms of speed and speed/accuracy. To have such an official benchmarking tool adopted by even Nuance, and why not, would save me a lot of explanation and justification although even in its current form it would be more than useful to prove a point. So thank you and can you adapt it so I can justify why an RTA victim should have a 12 bedroom luxury house because quite simply DNS performs much better in such an environment. Lindsaywww.pcbyvoice.com ------------------------- |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Lindsay, interesting idea. I will see what I can do. I guess once you start considering what impact living in a luxury house may have on Dragon's performance, you shouldn't also neglect having a pool like an ocean in the garden as well as being transported in a modified Ferrari. Anyway, back to being serious, I added the results I received from you to the list. Unfortunately, due to technical problems, you had to cancel out of running the tests very early (was it to do with the electricity in your house once more not working?), but also in all its briefness it clearly demonstrates the processor being used, particularly the processor cache being available, being the determining factor as far as performance on a very systematical scale. Please keep in mind that Chuck's system has 1.5 MB of L2 Cache and 12 MB of L3 Cache. Your system, referred to as LA in the list, has 1 MB of L2 cache and 8 MB of L3 cache. Thinking of it, this is almost pure algebra. Have a look at it, please: Rüdiger
-------------------------
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
I would like to know whether there was an actual perceived difference in the responsiveness of both systems and whether it really made that much differnce to the user. Does .21 vs .36 really make a differrence in the real world experience?
------------------------- |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quote: I would like to know whether there was an actual perceived difference in the responsiveness of both systems and whether it really made that much differnce to the user. Does .21 vs .36 really make a differrence in the real world experience? Dr. Fred, in order to answer your question in some reasonable manner, I would need an entire chapter to explain, but for the sake of briefness, I will start by referring to some keywords being used in speech recognition research and some articles you or anyone might look up and start from there. The numbers you are seeing in the list represent the so-called Real Time Factor (which is a technical term in research, cf. http://en.wikipedia.org/wiki/Real_time_factor), what my program does is compute them automatically retrieving the relevant processing data coming straight from the Dragon engine and counting the times retrieved from the operating system. Therefore they are pretty accurate. Since the material being used is recorded speech (audio files) the results are absolutely replicable, and therefore valid. Now please have a look at the definition of the Real Time Factor: Quote: If it takes time P to process an input of duration I, the real time factor is defined as . RTf = P/I If, for example, it takes 8 hours of computation time to process a recording of duration 2 hours, the real time factor is 4. When the real time factor is 1 or less than 1 , the processing is done in real time. It is a hardware-dependent value. What this means is, that for instance given the examples that we have tested which were recorded utterances each of them about 10 seconds long (or entire sentences of 20 words on the average for that matter), Chuck's system can handle them in about two third of the time (on the average) that my system will need to do this. Just to demonstrate, this is a typical result for Chuck's system (file 2, run 2, slider 50%) : Utterance Length: 9090 msecs Unfortunately I deleted the details in here for my testings (the log files that are created by pressing a button in my program), but if the ratio 0.35 in this run for me according to the list, it must have looked something like this: Utterance Length: 9090 msecs Now this is just about a third less, or two thirds of it, or however you may like looking at it. You may also argue that this is just a difference of about 1.2 secondes or so, but consider that this is in relation to processing about 10 seconds of speech. And quite frankly, to me, it will make a difference. Rüdiger
-------------------------
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
I have already shared this idea with Rudiger, but I present it here as a possible method of using Rudiger's ingenious Performance Factor Tool to compare computers for use with speech recognition. I will present an example using the data that has been posted with Rudiger's and Chuck's results. We first need to have a "gold standard" with which to compare. This is determined by which computer registers the lowest cumulative best performance factor at the 50% setting of the accuracy versus speed slider. For now, we will assume this to be Chuck's computer, and we will compare Rudiger's. Using "best" results:
I welcome any comments, including criticisms (be nice
MDH ------------------------- |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
FuseTalk Standard Edition v4.0 - © 1999-2021 FuseTalk™ Inc. All rights reserved.