KnowBrainer Speech Recognition
Decrease font size
Increase font size
Topic Title: mouse to jump to cursor?
Topic Summary:
Created On: 11/01/2019 02:23 AM
Status: Post and Reply
Linear : Threading : Single : Branch
 mouse to jump to cursor?   - Anandits - 11/01/2019 02:23 AM  
 mouse to jump to cursor?   - Ag - 11/01/2019 05:32 PM  
 mouse to jump to cursor?   - Mphillipson - 11/03/2019 01:08 PM  
 mouse to jump to cursor?   - Lunis Orcutt - 11/03/2019 02:51 PM  
 mouse to jump to cursor?   - dilligence - 11/03/2019 09:44 PM  
 mouse to jump to cursor?   - Ag - 11/04/2019 03:24 PM  
 mouse to jump to cursor?   - dilligence - 11/04/2019 07:17 PM  
 mouse to jump to cursor?   - Ag - 11/06/2019 02:13 PM  
 mouse to jump to cursor?   - dilligence - 11/06/2019 09:19 PM  
 mouse to jump to cursor?   - Ag - 11/07/2019 12:40 AM  
 mouse to jump to cursor?   - Anandits - 11/07/2019 01:36 AM  
Keyword
 11/01/2019 02:23 AM
User is offline View Users Profile Print this message

Author Icon
Anandits
Power Member

Posts: 75
Joined: 03/25/2019

Is there a way to order the mouse position to jump to the cursor?

It happens occasionally that I navigate in a window by voice and am quite pleased to land where I want to go, and then actually have to execute a mouse command at that place – like a right click.
However, the mouse is still very happily roaming around somewhere else on the screen.

And so I have to do the whole thing again, but now by ordering the mouse to move, which of course is very slow.

Would there happen to be a command that orders the mouse position to get reconciled with where the cursor is... ?

 11/01/2019 05:32 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 326
Joined: 07/08/2019

I want this too. Both for AutoHotkey, and for Dragon/KnowBrainer

I don't know for KnowBrainer. I have tried to write the same thing in AutoHotKey, where they have variables %A_CaretX% and %A_CaretY%.

many folks write scripts using the caret, text cursor.

I haven't had much luck, but I use a very heterogenous multi-display system, which has often had problems with things like mouse coordinates.

If you have only have one monitor, or if you use the same scaling on every monitor, AHK caret may work for you

Many people use Dragon/KnowBrainer to invoke an AutoHotKey script.

it may be possible to do this inside Dragon/KnowBrainer without using an AutoHotKey script. I haven't tried yet.



-------------------------

DPG15.5 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, Dell Inspiron 7591 2-in-1, 64GB RAM, Intel Corei7-10510U CPU @ 1.80GHz, 2304 Mhz, 4 Cores...


 11/03/2019 01:08 PM
User is offline View Users Profile Print this message

Author Icon
Mphillipson
Advanced Member

Posts: 194
Joined: 09/22/2014

If you just simply want to perform a right click without the mouse you can create the following script and call it for example Context Menu. This is the applications menu that performs a keyboard-based right click without affecting the mouse.

 

Declare Function keybd_event Lib "user32.dll" (ByVal vKey As _

Long, bScan As Long, ByVal Flag As Long, ByVal exInfo As Long) As Long

'--- Hardware key constants And functions ---

Public Const KEYEVENTF_KEYDOWN As Integer = &H0

Public Const KEYEVENTF_KEYUP As Integer = &H2

Public Const VK_APPS As Integer = &H5D '---Applications key on a Microsoft Natural keyboard

 

Sub Main

 keybd_event(VK_APPS,0,KEYEVENTF_KEYDOWN ,0)

 Wait 0.1

 keybd_event(VK_APPS,0,KEYEVENTF_KEYUP ,0)

End Sub



-------------------------

Thanks Mark


 


 


 


Dragon Professional Advanced Scripting/KnowBrainer Scripts
Video Examples of Coding by Voice



 11/03/2019 02:51 PM
User is offline View Users Profile Print this message

Author Icon
Lunis Orcutt
Top-Tier Member

Posts: 37356
Joined: 10/01/2006

Wouldn't it be just as easy to say press {Shift F10} or use the KnowBrainer Context Menu command?



-------------------------

Forum Mission Statement
Trial Downloads
Dragon/Sales@KnowBrainer.com 
(615) 884-4558 ext 1

 11/03/2019 09:44 PM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1036
Joined: 08/16/2010

Shift + F10 can be taken one step further if you combine it with the open ended principle. In addition you can add some speed to the process by creating the commands in DVC (yes open ended commands can be done in DVC ). 

 

The open ended Windows Explorer Power Commands have been around since the early SP versions. This one for instance pseudo-right clicks any file or folder:

 

Right-click <dictation>

 

SendKeys ""+_arg1+""

Wait 250

SendKeys "{Ctrl+t}"

Wait 250

SendKeys "{Shift+F10}"

 

As with any open ended voice command the process can be somewhat tricky and you need to pronounce clearly to avoid misrecognitions. But all in all this command works great if there's not too many files or folders with similar names.

 

Note that the DVC Wait times are in milliseconds so 250 means 0.25 seconds. DON'T put Wait 250 in Advanced Scripting unless you want to take a (forced) coffee break  

 

Here's a very old demonstration video of the Windows Explorer Power Commands (I've just reuploaded it). Although I've ditched some of the commands in later versions, current SP editions still include the ones that have proven to work best:

 

https://www.youtube.com/watch?v=LHGBUnbVJWU&feature=youtu.be

 

I found these commands to work best when the Windows 10 Explorer ribbon is disabled. If you like you can disable it with the Ribbon Disabler tool which modifies explorerframe.dll. Of course you can always reenable it again but, at least on my systems, Dragon appears to perform a little bit more smoothly when the ribbon is disabled .

 

Note that any major Windows 10 upgrade will revert your changes and reinstalls the ribbon, grrrr.. But luckily you can then run Ribbon Disabler again.



-------------------------

www.speechproductivity.eu 


The Ultimate Dictation Box for Dragon - SP Ergonomic Text Editor - Hotkeys for Dragon - Better Text to Speech - Easy Guide to Dragon Scripting



 11/04/2019 03:24 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 326
Joined: 07/08/2019

just checking: Lunis: Mark: diligence:

 

what you are describing is how to use keyboard commands at the place where the keyboard focus is currently placed. or, in the example that Mark gave, emulating a right-click where the mouse is not.

 

That helps a lot, but...

 

As far as I can tell it does not actually move the mouse cursor over to where the text cursor/caret currently is.

 

Just checking.



-------------------------

DPG15.5 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, Dell Inspiron 7591 2-in-1, 64GB RAM, Intel Corei7-10510U CPU @ 1.80GHz, 2304 Mhz, 4 Cores...


 11/04/2019 07:17 PM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1036
Joined: 08/16/2010

https://superuser.com/questions/1037906/move-mouse-pointer-to-typing-area

 

I added right click and sleep to the AHK script:

 

#a:: 

CoordMode, Caret, Screen

CoordMode, Mouse, Screen

MouseMove, %A_CaretX%, %A_CaretY%

Sleep, 200

Click, right

return

 

Seems to work all right in speech friendly applications, not so in browsers.



-------------------------

www.speechproductivity.eu 


The Ultimate Dictation Box for Dragon - SP Ergonomic Text Editor - Hotkeys for Dragon - Better Text to Speech - Easy Guide to Dragon Scripting



 11/06/2019 02:13 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 326
Joined: 07/08/2019

@diligence: how many displays do you have? Are they homogenous, i.e. all the same, or at least all using the same display scaling?

I have four displays, extremely heterogenous, of different dimensions both physical and logical, with different display scalings.

The code at https://superuser.com/questions/1037906/move-mouse-pointer-to-typing-area works for certain apps but only on my primary display.

In the past I have written code that attempts to estimate what coordinates I need to use with MouseMove as calculated from the MouseSetPos coordinates, different scalings and window positions. So far I haven't been able to make that estimate in a single formula, and I've been needed to iterate until the MouseSetPos coordinates returned after the MouseMove correspond to the MouseSetPos coordinates desired. Typically, I am using this to return a window to a previously saved position.

I reiterate either until the desired coordinates are obtained, or until MouseSetPos, MouseGetPos, MouseSetPos reaches a fixed point, even if it's not what I desire.

unfortunately, this fails to work with A_CaretX/Y, because the caret coordinates seem to be in a completely different system than the two different coordinate systems for MouseSetPos and MouseGetPos. I have not yet tried to figure out if there is a linear mapping, although as noted above I haven't found such linear mapping even for MouseGetPos/MouseSetPos. Unfortunately, the iterative technique I use for MouseGetPos/MouseSetPos does not work to move the mouse to the caret, because I have no way to tell if the mouse is actually on top of the caret.

Instead, I have sometimes used the kluge "move mouse to somewhere in the active window". E.g. in the middle of the window, or one third down and across from the upper left-hand corner. Although even here the inconsistent MouseGetPos and most set of coordinates are annoying, and I have to use an iteration. Not to mention the possibility of windows that are all or partially offscreen.


-------------------------

DPG15.5 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, Dell Inspiron 7591 2-in-1, 64GB RAM, Intel Corei7-10510U CPU @ 1.80GHz, 2304 Mhz, 4 Cores...


 11/06/2019 09:19 PM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1036
Joined: 08/16/2010

Ag,

 

I only use one standard HD monitor (1920x1080) with 125% scaling. That works well in most cases but I have a couple of deviations compared to 100% scaling. Closing/minimizing some windows won't always work with the standard Dragon commands). I've also noticed that sometimes with selecting words Dragon selects an additional space after the last letter of the word or phrase. 

 

So you're using four different monitors? I didn't even know it was possible to come up with different scaling's for each of them... But at least the screen resolutions will be set accordingly to the one with the lowest resolution? 

 

The AHK script works well in Notepad, Wordpad, Word etc. on my system, but it does nothing in Chrome or Edge Dev. Looks like it has to do with the type of text control being used.

 

Anyway, a reliable solution for this may not exist.

 

But who knows maybe PG (AHK Maestro) steps in and surprises us once more



-------------------------

www.speechproductivity.eu 


The Ultimate Dictation Box for Dragon - SP Ergonomic Text Editor - Hotkeys for Dragon - Better Text to Speech - Easy Guide to Dragon Scripting

 11/07/2019 12:40 AM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 326
Joined: 07/08/2019

Originally posted by: dilligenceSo you're using four different monitors? I didn't even know it was possible to come up with different scaling's for each of them... But at least the screen resolutions will be set accordingly to the one with the lowest resolution?

 

No,  the actual screen resolutions are dramatically different, as are the effective screen resolutions. Windows does NOT require that all of the monitors connected have the same pixel counts.

 

In a perfect world I might be rich enough to have three external monitors matched in size. Instead, what I do is every so many years I buy a new monitor, typically physically larger with more pixels than my previous monitors, and my previous monitors get moved to become auxiliary monitors at the sides of my largest monitor.

 

---

 

Right now I am looking at 4 monitors:

 

Monitor #1 is my Surface Book LCD, 13", 3000 x 2000 pixels, 200% scaling, landscape.

Monitor #2 is 30"in landscape,  2560 x 1600 pixels, 150% scaling. it is physically placed directly above monitor #1

Monitor #3 is 24", 1200 x 1920 pixels, portrait, 100%. it is physically placed to the left of monitor #2.

Monitor #4 is 24", 1200 x 1920 pixels, portrait, 150%. it is physically placed to the right of monitor #2.

Monitor #3 and #4 have different scalings, because at the moment I dedicate monitor #4 to speech stuff, like the KnowBrainer's sidebar, the Dragon bar, the Vocabulary Editor, and DragonPad. I find the KnowBrainer sidebar font too small to read at 100%. I know it can be changed in a saved configuration, but I use other monitors where I don't want a larger font in the sidebar, hence I have changed the scaling. also, the KnowBrainer command editor font size cannot be saved as far as I know, so it is also more readable on monitor  #4.

At work I use a different set of monitors except for my Surface Book LCD, all 4 different pixel dimensions, three at 100% and one at 200%.

 

this works surprisingly well, at least from the point of view of Windows. Unfortunately many tools like AutoHotKey really don't understand heterogenous multi-monitor systems.



-------------------------

DPG15.5 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, Dell Inspiron 7591 2-in-1, 64GB RAM, Intel Corei7-10510U CPU @ 1.80GHz, 2304 Mhz, 4 Cores...




 11/07/2019 01:36 AM
User is offline View Users Profile Print this message

Author Icon
Anandits
Power Member

Posts: 75
Joined: 03/25/2019

You've all lost me, but please keep going: good to hear there is interest in this!
I also use multiple monitors and when I travel I use only my laptop; so I would want a command that is independent from screen size.
It just seems to me that there should be a command possible that very easily reconciles the locations of the cursor and the mouse position. But it seems it is not that easy. Anyway, maybe someone will come up with it!
Thanks for thinking about it.
Statistics
31556 users are registered to the KnowBrainer Speech Recognition forum.
There are currently 3 users logged in.
The most users ever online was 3936 on 09/08/2015 at 11:44 AM.
There are currently 311 guests browsing this forum, which makes a total of 314 users using this forum.

FuseTalk Standard Edition v4.0 - © 1999-2020 FuseTalk™ Inc. All rights reserved.