 |
KnowBrainer Speech Recognition |
 |
 |  |
|
Topic Title: Scripting of mouse movements and clicks Topic Summary: Scripting of mouse movements and clicks Created On: 03/01/2011 03:08 AM Status: Post and Reply |
|
 |
 |
- csallada | - 03/01/2011 03:08 AM |
|
|
- Chucker | - 03/01/2011 10:24 AM |
|
|
- csallada | - 03/13/2011 12:15 AM |
|
|
- Chucker | - 03/13/2011 03:58 AM |
|
|
- Lunis Orcutt | - 03/01/2011 04:26 PM |
|
|
- Allan | - 03/02/2011 01:19 PM |
|
|
- monkey8 | - 03/13/2011 10:20 AM |
|
|
- csallada | - 03/13/2011 10:20 PM |
|
|
- monkey8 | - 03/14/2011 05:24 AM |
|
|
- csallada | - 03/14/2011 07:27 AM |
|
|
- monkey8 | - 03/14/2011 07:53 AM |
|
|
- csallada | - 04/10/2011 03:11 AM |
|
|
- monkey8 | - 04/12/2011 12:35 PM |
|
|
- Lunis Orcutt | - 04/12/2011 01:35 PM |
|
|
- gt1982 | - 01/19/2012 05:18 PM |
|
|
- Chucker | - 01/19/2012 07:03 PM |
|
|
- gt1982 | - 01/19/2012 08:18 PM |
 03/01/2011 03:08 AM
|
|
|
I use a practice management system called Genie. I have Dragon Medical 10.1 and knowbrainer software for commands. I use Dragon dictation to dictate my letters.
At the moment prior to dictating a letter i need to make a number of mouse clicks and movements. I have use a macro recording to record these movements to create a command however the problem is that this is relatively slow and does not work if i run the command on the different computer with a different screen size.
See the accompanying pictures: for example prior to dictating a letter i need to click on the quill icon, then click on the arrow need to the default letter, then choose new letter from the list, then click on the reviewed tab before pressing the microsoft icon button.
Ideally i would like to teach commands specific for the program so if doesn't matter which screen size the program is on.
the second option would be to create a script independent of the macro recorder that incorporate mouse click on certain parts of the screen.
chris allada ------------------------- Christopher Allada Cardiologist Dragon Medical 11, Knowbrainer 2011, Dell Precision T3500 PC, Xeon W3520 CPU at 2.8GHz, Windows 7 Professional 64 bit, 12 GB ram |
|
|
|
|
|
03/01/2011 10:24 AM
|
|
|
Chris, Chuck Runquist "At times we shall simply have to admit that, one way or another, what we can neither explain nor understand certainly doesn't cease to exist because we cannot see how it does or why it should." - Dr. Mark Hyman -------------------------
|
|
|
|
|
|
03/13/2011 12:15 AM
|
|
|
Thankyou for you comments, Chuck and others. I have updated my profile as i am using Dragon Medical 10.1.
I was unable to locate the application VoicePower. Where can i get this?
I cannot use commands and dictate freely in Genie as i can say in word as in says "dictating to a nonstandard window". Other than the top commands eg file, edit, open etc which is in the top menu of Genie I cannot command any other parts of the program. Only 2 of the icons have a key command.
I am happy to use mouse position commands as i do most dictating from this computer. I am not sure however how i do a mouse position command. Does anyone have an example of a simple script that uses a combination of keystrokes and mouse clicks in different positions with slight pauses between each comman? Or does the VoicePower program do this? I am happy to give advanced scripting a go but again i am not sure where i access mouse grid or mouse position commands. As you can see below Genie has made a couple of the icons have keystokes but the remainder i need to use mouse clicks.
Therefore to write a letter: Press Stethoscope symbol. There is a command- (CTRL-K) Slight pause Press Quill symbol. There is a command- (CTRL-R) Slight pause Press marked as reviewed box. There is a command- (CTRL-R) Slight pause Mouse click on arrow to the right of default letter Slight pause Mouse click on 'one of the letter templates' Slight pause Mouse click over word sign
Chris ------------------------- Christopher Allada Cardiologist Dragon Medical 11, Knowbrainer 2011, Dell Precision T3500 PC, Xeon W3520 CPU at 2.8GHz, Windows 7 Professional 64 bit, 12 GB ram |
|
|
|
|
|
03/13/2011 03:58 AM
|
|
|
Quote: I was unable to locate the application VoicePower. Where can i get this? Chris, Go to the following link: VoicePower® Ultimate Chuck Runquist Technical Project Manager VoiceTeach LLC Home of VoicePower® Ultimate  "Always do right - this will gratify some and astonish the rest". -- Mark Twain -------------------------
|
|
|
|
|
|
03/01/2011 04:26 PM
|
|
|
We believe that Chuck's answer is partially aimed at your old profile and tag which both state that you're using DNS Pro 8 but we updated your profile to reflect that you are using Medical 10.1 since you mentioned it in your 1st paragraph. We don't have the ability to edit your tagline. This will give you additional options such as VoicePower.
We would strongly recommend against using Macro Recorder commands because they waste a lot of space in your Command Browser and because they recorded in real time. They also play back in real time which makes them appear lethargic. Mouse Position commands are somewhat risky because they tend to only work well on 1 monitor at 1 specific resolution and even then, things can change and you wind up having to rewrite your commands. We minimally recommend creating step-by-step macros or if you're feeling a little braver, Advanced-Scripting commands. Although you can use either MouseGrid or Mouse Position commands in Advanced-Scripting, it's better if you can find workarounds such as activator keys or even pressing the {Tab} key multiple times. Try left-clicking into Heaolthscope app and pressing the {Alt} on your keyboard. If the developers of Healthscope did a decent job, you should see a activation keys which will typically look like a short line under one of the letters such as Add Consult, Edit Consult, Save etc… This will make your work easier. -------------------------
|
|
|
|
|
|
03/02/2011 01:19 PM
|
|
|
I agree with Lunis about using activator keys in an Advanced Scripting Macro. However, when stuck with mouse clicks, one needs to locate them for each compute/screen combination, and not change the screen layout. Then using the following script, and substituting the ID of your particular computers, you can write one macro that does not need to be edited.
If Environ ("computername" Y_offset = 0 X_offset = 0 ElseIf Environ ("computername" Y_offset = 15 X_offsset = 5 End If
SetMousePosition 1,628+ X_offsset,155 + Y_offset |
|
|
|
|
|
03/13/2011 10:20 AM
|
|
|
Chris I have attached several commands that will accomplish your tasks with advanced scripting. Simply import the XML file attached into your Command Browser and you will add to your repertoire 4 new voice commands, the first three will accomplish your tasks: "add advanced scripting command" First of all call "add advanced scripting command" and this will create your empty advanced scripting command. Then go to your application and make sure it is the active window, position the mouse exactly where you require and then call "record mouse position" You will receive a verbal confirmation that the mouse position has been recorded and then you just simply move back to your advanced scripting command and call "paste mouse position", this will paste in the mouse position, a Wait line and a button click command into your new script/voice command. E.g. SetMousePosition 1,222,314 Then just repeat the process with as many steps as you require and finally give your new voice command/script a name. Once you have used the commands a few times you will be able to create multiple mouse position voice commands very quickly. When you import the XML file it'll also import a voice command/script called "add mouse position command". Basically this command will create a complete script, for which you just need to add a name, but this will only record a single mouse position and then implement a left mouseclick for that position. It's very quick for creating single mouse position commands but for multiple mouse position commands you need to use the 3 commands above. Lindsay Edit: I just realised that the "add advanced scripting command" script that is in the multiple mouse positioning commands file will create global commands which will be recognised anywhere. If you use the "add advanced scripting command" from the separate scripting command update file it'll create the commands so that they are only recognised when the relevant application is active. If you just import this file last it'll give you the option to overwrite the existing "add advanced scripting command" and then use this. ------------------------- |
|
|
|
|
|
03/13/2011 10:20 PM
|
|
|
Thankyou Linday for those commands. They seem to work fine. Is it possible to have a combination of mouse clicks and keyboard commands in the same script? i am not sure how to do this.
i have tried a combination of mouse clicks and keyboard commands which doesn't work: Sub Main if i replace the keyboard commands with their equivalent mouse clicks it works fine however. Sub Main
Chris ------------------------- Christopher Allada Cardiologist Dragon Medical 11, Knowbrainer 2011, Dell Precision T3500 PC, Xeon W3520 CPU at 2.8GHz, Windows 7 Professional 64 bit, 12 GB ram |
|
|
|
|
|
03/14/2011 05:24 AM
|
|
|
Yes you can combine mouse positions and keyboard shortcuts in the same command, however as an example: SendKeys"{Ctrl+k}" will not work because they changed the syntax for this command. ------------------------- |
|
|
|
|
|
03/14/2011 07:27 AM
|
|
|
Dear Lindsay, when i create the advanced script and run it nothing seems to happen. I have tried having a space after SendKeys and no space after SendKeys with the same result. again nothing happens if i use the SendDragonKeys"{Ctrl+k}" instead. if i manually press on the keyboard CTRL-K, short pause, CTRL-R, short pause, CTRL-R short pause it works however. The following is an example of the script where nothing seems to happen when i say the command. Sub Main End Sub I have also noted that nothing happens when i add into a script the mouse click as seen in the second picture where i have tried to click onto the sidebar to allow me to scroll as the page to another template as seen in the third picture. i am not sure why i cannot recreate the mouse click for this sidebar. When i try the script below everything works except for the last click as i stated in the previous sentence and therefore i cannot scroll up the template list. Sub Main End Sub
any ideas? chris
------------------------- Christopher Allada Cardiologist Dragon Medical 11, Knowbrainer 2011, Dell Precision T3500 PC, Xeon W3520 CPU at 2.8GHz, Windows 7 Professional 64 bit, 12 GB ram |
|
|
|
|
|
03/14/2011 07:53 AM
|
|
|
Chris
------------------------- |
|
|
|
|
|
04/10/2011 03:11 AM
|
|
|
Dear Lindsay,
Thanks for all your help so far.
SendSystemKeys"{Ctrl+k}" has made this part of the command now work.
I now able to write the command completely. The issue of the command not working was related to me having two monitors and even though the recording of the mouse position was on the right monitor if the command editor was on the other monitor it was causing problems.
I now have 3 outstanding problems. -how do i do a right mouse click instead of a left mouse click? -is it possible to execute a command eg close file, save file within an advanced script command? -is it possible to to have commands that have mouse clicks across 2 different monitors? (at the moment if i try i get coordinates out of bounds)
Chris ------------------------- Christopher Allada Cardiologist Dragon Medical 11, Knowbrainer 2011, Dell Precision T3500 PC, Xeon W3520 CPU at 2.8GHz, Windows 7 Professional 64 bit, 12 GB ram |
|
|
|
|
|
04/12/2011 12:35 PM
|
|
|
Quote: how do i do a right mouse click instead of a left mouse click? Hi Chris The syntax for the mouse button clicks is: ButtonClick x, y if x = 1 then it is a left click so as an example a single right click would be: ButtonClick 2,1 Quote: -is it possible to execute a command eg close file, save file within an advanced script command? Yes and there are various ways of doing this, one example is using keyboard shortcuts so the following script, as an example, would save a file with Microsoft Word or most applications for that matter: Sub Main you can find information on all these commands in the help files, alternatively there is a very useful book by Larry Allen on advanced scripting, "Larry Allen's advanced scripting guide", available from various places including KnowBrainer and www.PCspeak.com. If you are in the UK then you can get it from my website below or www.ITspeaking.co.uk Quote: -is it possible to to have commands that have mouse clicks across 2 different monitors? (at the moment if i try i get coordinates out of bounds) yes you can do this but first you need to understand the syntax of the mouse positioning command: SetMousePosition z, x, y if z = 0 then the mouse position is relative to the top left-hand corner of your main monito x, y are obviously the mouse coordinates I don't know what type of command you are trying to create but in most cases these tend to be application-specific so the mouse would be positioned relative to the current active application window. In this case the command will work where ever the window is placed on any of the monitors as long as the window is the same size as when the command was created. The hardest part is working out the mouse coordinates and there are various ways of doing this. If you are talking about application-specific mouse positioning scripts then one possible solution is to do a screen print of the application window (Alt+PrtScr) and then open Microsoft paint and paste the screen print onto the Microsoft paint window (Ctrl+v). You can then manually position the mouse anywhere on the application window and Microsoft paint will show you the mouse coordinates. An even better way of doing it is to use another script, which I can post for you if you want, and using this script (which is called something like "add mouse position command" Lindsay ------------------------- |
|
|
|
|
|
04/12/2011 01:35 PM
|
|
|
You can also view the proper left-click and right-click mouse syntax by looking up the Left Click and Right Click commands in your KnowBrainer Command Browser. They are identical to the coding in NaturallySpeaking.
However, let's kick it up a notch. Begin by saying Import KnowBrainer Commands. This particular KnowBrainer command will open the NaturallySpeaking Command Browser and import about 24 specialty commands that only work in DNS Pro, Medical or Legal (which you are using). One of these commands is called Find Mouse Position. Just click into the appropriate Window (so that NaturallySpeaking knows which application to focus on), move your cursor to the exact position you wish to record and say Find Mouse Position. After this command records your Mouse Position it will verbally say “Mouse Position is ready to be pasted” because KnowBrainer talks. Yikes! When you paste the recorded Mouse Position into a DNS Advanced-Scripting command KnowBrainer will include the mouse X and Y Window coordinates, a slight pause and a mouse Left Click. All you will have to do is paste the code. Additional note to end users who are not using 1 of the professional versions of DNS but have KnowBrainer… You can also perform the same function within KnowBrainer 2011 via the following: 1. Say Start Paint to open Microsoft Paint 2. Maximize both Microsoft Paint and the document contained in Microsoft Paint 3. Minimize Microsoft Paint to temporarily move it out of the way 4. Position your cursor exactly where you want to enable a mouse Left Click 5. Either say Switch to Paint or press {Alt+Tab} to pull up Microsoft Paint 6. Write down the numbers that you see in the lower left-hand corner of Microsoft Paint (make sure you are viewing at 100%). The 1st digit is your x-coordinate and the 2nd digit is your y-coordinate. Just leave out the px and you can add this information to a KnowBrainer Mouse Area command. -------------------------
|
|
|
|
|
|
01/19/2012 05:18 PM
|
|
|
Lindsay, I have tried loading the "add advanced scripting command" by importing into Command Browser. It doesn't seem to be working for me. Does this command have to be a .dat file? Thanks for the help. Greg |
|
|
|
|
|
01/19/2012 07:03 PM
|
|
|
Greg, "Life's Rule #1: Once you pull the pin, Mr. Grenade is no longer your friend." (Variant of Murphy's Law - Edward A. Murphy, Jr) -------------------------
|
|
|
|
|
|
01/19/2012 08:18 PM
|
|
|
Lindsay, Thank you. I continue to get an error message when I try to make the new commands. Please show me what the commands in "multiple positioning commands" would like in the My Commands Editor. Thanks, Greg |
|
|
|
|
jpg.JPG)
= "243-GC-023-1C" Then
FuseTalk Standard Edition v4.0 - © 1999-2013 FuseTalk™ Inc. All rights reserved.