KnowBrainer Speech Recognition
Decrease font size
Increase font size
Topic Title: Could use some help with Regular List + Open Ended List command
Topic Summary: For the upcoming Auto Box© public demo
Created On: 10/11/2021 09:23 PM
Status: Post and Reply
Linear : Threading : Single : Branch
Keyword
 10/11/2021 09:23 PM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1432
Joined: 08/16/2010

I'm releasing a public Auto Box© demo in a couple weeks and I'm adding an optional "pseudo" Auto-Launching DVC list command.

 

The sample command is <dear> <dictation> .

For example saying "Dear Marie" will launch Auto Box© with the contents "Dear Marie". This is purely meant to be used as a starting point for writing a new email header/body, or starting a new paragraph in speech unfriendly Office application for instance.

 

This is the command (for testing purposes you can replace the program path with C:\Windows\notepad.exe):

 

<dear> <dictation>

 

If _arg1 = "Dear" then AppBringUp "C:\Program Files (x86)\Auto Box Demo\Auto Box.exe"

Wait 300

SendKeys "Dear " 

SendKeys _arg2 

SendKeys "{Home}"

Wait 100

HeardWord "\cap","that"

Wait 100

SendKeys "{Ctrl+End}"

Wait 100

SendKeys ","

 

Depending on general system specs I may need to add more latency.

 

However, what I want to do is add more arguments to the first list so that the user has a bunch of words to choose from (or add his own). That usually works in DVC as long as it's just one scripting line. But in this case any new argument I add will keep on following the events in the remaining nine script lines including line 3: SendKeys "Dear" .

 

Does anyone know how to accomplish this? Preferably in DVC. If it's not possible in DVC then I'll settle for Advanced Scripting .

 

Here's a short video example of the command in action:

 

https://youtu.be/6-2sH0uUW8o



-------------------------


Auto Box© Demo now available



 10/12/2021 11:30 AM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1202
Joined: 04/03/2009

AppBringUp "C:\Program Files (x86)\Auto Box Demo\Auto Box.exe"

 

Wait 300

 

SendKeys _arg1

 

SendKeys "{Space}" 

 

SendKeys _arg2 

 

SendKeys "{Home}"

 

Wait 100

 

HeardWord "\cap","that"

 

Wait 100

 

SendKeys "{Ctrl+End}"

 

Wait 100

 

SendKeys ","



-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors

 10/12/2021 02:38 PM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1432
Joined: 08/16/2010

Thank you! That works like a charm.

 

I will add the following sample command:

 

 

 



-------------------------


Auto Box© Demo now available

 10/12/2021 03:19 PM
User is offline View Users Profile Print this message

Author Icon
kkkwj
Top-Tier Member

Posts: 831
Joined: 11/05/2015

I think I would pass the words to Auto Box on the command line using ShellExecute. Then AB could fetch the words from the command line arguments easily without any guessing at system performance and "Wait 300" delays.

 

You would also have a lot more flexibility in doing interesting things in C# in AB, IMHO. (faster response, capitalization, punctuation smarts (Dear Dr. John,), and so on.



-------------------------

Win10/x64, AMD Ryzen 7 3700X, 64GB RAM, Dragon 15.3, SP 6 PRO, SpeechStart, Office 365, KB 2017, Dragon Capture, Samson Meteor USB Desk Mic, Klim and JUKSTG earbuds with microphones, 3 BenQ 2560x1440 monitors, Microsoft Sculpt Keyboard and fat mouse



 10/12/2021 04:13 PM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1202
Joined: 04/03/2009

Originally posted by: kkkwj I think I would pass the words to Auto Box on the command line using ShellExecute. Then AB could fetch the words from the command line arguments easily without any guessing at system performance and "Wait 300" delays.

 

 

 

You would also have a lot more flexibility in doing interesting things in C# in AB, IMHO. (faster response, capitalization, punctuation smarts (Dear Dr. John,), and so on.

 

I agree with all of this!



-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors

 10/12/2021 04:22 PM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1432
Joined: 08/16/2010

Definitely! In the next version (7) coming January most of this will already be implemented. This version will be a little bit more intelligent too.

 

Here's a little preview. In the graphic it looks bigger than it is. Actual default size is a convenient 670 x 180 on regular HD (it will adapt to other scalings and resolutions). And it's completely resizable of course:

 

 

 

 

 

 

For now the scripting approach is interesting because you can apply it to all kinds of other applications. Speaking of which, I wish it were possible to use two open ended lists! That would for sure give users a whole lot more options.   



-------------------------


Auto Box© Demo now available



 10/13/2021 11:11 AM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1202
Joined: 04/03/2009

Given current technology, I cannot see how it would be logically possible to have multiple open-ended lists in a single command as there would be no way for the software to figure out where the open-ended dictation ends. One would need some kind of a mechanism to tell the software where the first open-ended dictation "ended". The parser would need to watch for some termination "trigger" then start a new ListVar.

-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors

 10/13/2021 03:14 PM
User is offline View Users Profile Print this message

Author Icon
kkkwj
Top-Tier Member

Posts: 831
Joined: 11/05/2015

Hi Edgar, yes, if you have back-to-back open-ended lists, the first list sucks everything up and leaves nothing for the second list. But maybe if you put a keyword between the two lists, maybe Dragon would be smart enough to terminate the first list with the first instance of the magic keyword. In other words, " puff " might correctly parse "blah blah puff the magic dragon" to give "blah blah"=listvar1 and "the magic dragon"=listvar2. At least that's how any good programmer would do it. :-)

-------------------------

Win10/x64, AMD Ryzen 7 3700X, 64GB RAM, Dragon 15.3, SP 6 PRO, SpeechStart, Office 365, KB 2017, Dragon Capture, Samson Meteor USB Desk Mic, Klim and JUKSTG earbuds with microphones, 3 BenQ 2560x1440 monitors, Microsoft Sculpt Keyboard and fat mouse

 10/14/2021 11:03 AM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1202
Joined: 04/03/2009

Originally posted by: kkkwj if you put a keyword between the two lists, maybe Dragon would be smart enough to terminate the first list with the first instance of the magic keyword.

That's the way I do it in one of my parsers. The trick is coming up with a "keyword" which is both easy for Dragon to get right/recognize out of context yet still be unlikely to occur in any context. For this I like to use a made up word or a foreign language word.



-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors

 10/14/2021 02:55 PM
User is offline View Users Profile Print this message

Author Icon
R. Wilke
Top-Tier Member

Posts: 7703
Joined: 03/04/2007

Thing being, that both Edgar and Kevin, and not even accounting for diligence as that is far beyond his scope anyway, are limited to consider things (recognition results) AFTER the fact, therefore the tedious efforts in implementing parses in whatever direction once a recognition result has been passed to the foreground window. But, wouldn't it be nice having a more straightforward, and more technically advanced approach instead, sorting it out a priori?

However, although doable, it all boils down to Dragon programming at the core, which both of you will probably never achieve, and specifically Kevin, whatever you think about "how any good programmer would do this", please do us a favour, save ourselves the trouble, and just leave it unsaid.

-------------------------



No need to buy if all you want to do is try ...

DragonCapture KB Download (Latest)
DragonCapture Homepage

 10/14/2021 08:07 PM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1202
Joined: 04/03/2009

Originally posted by: R. Wilke wouldn't it be nice having a more straightforward, and more technically advanced approach instead, sorting it out a priori? However, although doable, it all boils down to Dragon programming at the core

Obviously! Maybe some future Dragon programmer will stumble on this thread...



-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors

 10/14/2021 11:43 PM
User is offline View Users Profile Print this message

Author Icon
kkkwj
Top-Tier Member

Posts: 831
Joined: 11/05/2015

I don't know how important it is to have a first open-ended list; if the first list is fixed (like ListNumbers1-100), then the second list could be open-ended, I think. And for sure if Dragon calls your command based on the first fixed list, then you can do whatever you like when it comes to parsing the recognized utterance. I suppose we should all be cognizant of the difference between the "recognition problem" and the "parsing with local knowledge of what the incoming recognized should/might look like problem." Dragon does recognition and has a much more difficult problem than parsing.


I don't recall seeing anyone making a good case for the need for back-to-back open lists. Is there a good case for such a thing? Where might it offer people an advantage that could not be done with other techniques?

-------------------------

Win10/x64, AMD Ryzen 7 3700X, 64GB RAM, Dragon 15.3, SP 6 PRO, SpeechStart, Office 365, KB 2017, Dragon Capture, Samson Meteor USB Desk Mic, Klim and JUKSTG earbuds with microphones, 3 BenQ 2560x1440 monitors, Microsoft Sculpt Keyboard and fat mouse

 10/15/2021 10:37 AM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1432
Joined: 08/16/2010

Originally posted by: kkkwj I don't recall seeing anyone making a good case for the need for back-to-back open lists. Is there a good case for such a thing? Where might it offer people an advantage that could not be done with other techniques?

 

Kevin,

 

If it were possible to have two open-ended lists in one command and the lists wouldn't need any kind of prefix, and the first one (like Edgar pointed out) knew where its contents ended, then you would have a simple: <dictation> <dictation> structure. This would allow complete auto launching of any kind of dictation box, notepad etc.

 

Of course such a command would only be useful in a very limited amount of applications/situations and it would most probably also need to be much more subtle than that.

 



-------------------------


Auto Box© Demo now available



 10/15/2021 12:09 PM
User is offline View Users Profile Print this message

Author Icon
kkkwj
Top-Tier Member

Posts: 831
Joined: 11/05/2015

Thank you for the example. But wouldn't a fixed list of app names in the first list work as well?

 

<applist> <dictation>

 

List applist:

AB

photoshop

word

etc

 

The script would be:

if Listvar1 = "word" then ShellExecute "path to word"

if Listvar1 = "photoshop" then AppBringUp "path to photoshop"

 

It wouldn't be a big deal to add spoken app names in the first list, and you need to put their pathnames in the script anyway, so the list is not a big deal in comparison. The spoken names and the pathnames need to reside somewhere.



-------------------------

Win10/x64, AMD Ryzen 7 3700X, 64GB RAM, Dragon 15.3, SP 6 PRO, SpeechStart, Office 365, KB 2017, Dragon Capture, Samson Meteor USB Desk Mic, Klim and JUKSTG earbuds with microphones, 3 BenQ 2560x1440 monitors, Microsoft Sculpt Keyboard and fat mouse

Statistics
32177 users are registered to the KnowBrainer Speech Recognition forum.
There are currently 1 users logged in.
The most users ever online was 12124 on 09/09/2020 at 04:59 AM.
There are currently 270 guests browsing this forum, which makes a total of 271 users using this forum.

FuseTalk Standard Edition v4.0 - © 1999-2021 FuseTalk™ Inc. All rights reserved.