KnowBrainer Speech Recognition
Decrease font size
Increase font size
Topic Title: Dragging and dropping in games
Topic Summary:
Created On: 09/04/2022 01:00 PM
Status: Post and Reply
Linear : Threading : Single : Branch
 Dragging and dropping in games   - cdanger - 09/04/2022 01:00 PM  
 Dragging and dropping in games   - Ag - 09/04/2022 05:57 PM  
 Dragging and dropping in games   - dilligence - 09/05/2022 12:08 AM  
 Dragging and dropping in games   - cdanger - 09/05/2022 02:47 PM  
 Dragging and dropping in games   - cdanger - 09/05/2022 02:43 PM  
 Dragging and dropping in games   - Lunis Orcutt - 09/05/2022 01:58 PM  
 Dragging and dropping in games   - cdanger - 09/05/2022 02:57 PM  
 Dragging and dropping in games   - Lunis Orcutt - 09/05/2022 03:55 PM  
Keyword
 09/04/2022 01:00 PM
User is offline View Users Profile Print this message

Author Icon
cdanger
New Member

Posts: 4
Joined: 09/04/2022

I'm helping a user who is 100% reliant on voice. He's an avid gamer including Songs of Syx. He's extremely fast with DNS's Mouse Grid as he's learnt over the years the combinations for any spot on the screen, we've looked at Know Brainer's Display Numbers but he's actually faster and more comfortable using Mouse Grid.

However he's been struggling to use Mouse Grid to pick up items and drag them in games, which requires holding down the left mouse button and sometimes the shift key as well. We tried an additional script for Know Brainer which uses the command "She" and "Shah" to hold down and release the button but it's not working reliably. I've searched the forums for this script but can't find it right now.

He can use the standard Mouse Move commands in Dragon but he find this labourious and often misses his target which is very frustrating. We've been trying eye-gaze too but we can't get a good enough accuracy.

Any advice for this young man?

 09/04/2022 05:57 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 887
Joined: 07/08/2019

If you have 1 of the non-official MouseGrids, e.g. 1 of the open source ones written in AutoHotKey, it should be possible to something like

MouseGrid 5463....
Drag to MouseGrid 5745...

or possibly even

MouseGrid Drag from 5675756... to 675675...

( I'm not sure which would be easier to say, yet)


Unfortunately I didn't find any of the open source AutoHotKey mouse grids were particularly speech friendly. most require you to type numbers.

fixing this has been on my TBD list for a while. low priority because I've never been a particular fan of MouseGrid - in fact I have it disabled at the moment - but my dislike of MouseGrid may be as much related to its problems with displays with heterogenous display scaling as to my dislike to speaking beastlike numbers. I've been meaning to up that priority, because I dislike display numbers even more, and watching skilled MouseGrid users is quite impressive.

Anyway, this post just to suggest that you might look at 1 of the open source MouseGrids and customize for speech If you are comfortable with AutoHotKey or C# or whatever other language it might be written in. the biggest trouble with open source MouseGrid is that there are so many of them.


-------------------------

DPG15.6 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, MS Surface Book 3, Intel Core i7-1065G7 CPU @ 1.3/1.5GHz (4 cores, 8 logical, GPU=NVIDIA Quadro RTX 3000 with Max-Q Design.

 09/05/2022 12:08 AM
User is offline View Users Profile Print this message

Author Icon
dilligence
Top-Tier Member

Posts: 1578
Joined: 08/16/2010

When it comes to mouse dragging and dropping, your user might be interested in SP Press Utility©. It's very reliable. Here's a Demonstration Video

 

Your user might also find the SP Pointer Grid© convenient. Here's a Demonstration Video of that.

 

Perhaps the fastest way would be to use VoiceComputer Overlays. I am demonstrating this feature in This Video



-------------------------

https://speechproductivity.eu


Turbocharge your Dragon® productivity with 40 Power Addons

 09/05/2022 02:47 PM
User is offline View Users Profile Print this message

Author Icon
cdanger
New Member

Posts: 4
Joined: 09/04/2022

Thanks Dilligence - I've seen your website but not tried the software yet. I'll give it a go and see if I can get it to work. After a brief look at the videos the numbers might be a bit small for him to be able to see (which is one reason why MouseGrid has worked so well for him) but the more options we can explore the better! A proper look by me when I'm back at work might indicate that this is perfect for him. 

 09/05/2022 02:43 PM
User is offline View Users Profile Print this message

Author Icon
cdanger
New Member

Posts: 4
Joined: 09/04/2022

Thanks I'll look into this. I have adapted some AutoHotKey scripts for lots of reasons in the past and so I can look into this - I'm not very codey but I know someone who can help :-)

 09/05/2022 01:58 PM
User is offline View Users Profile Print this message

Author Icon
Lunis Orcutt
Top-Tier Member

Posts: 40207
Joined: 10/01/2006

                    Welcome (See Mission Statement)

 

Dragon's MouseGrid feature is single monitor rubbish left over from the late 90s. Fortunately, it looks like the next Dragon release will include a new numbering technology that doesn't disappear after you use them. You will be able to Left Click, Right Click and Middle Click any number. KnowBrainer 2020/18, which is freely available to those with permanent physical disabilities, will kick it up a notch by adding artificial intelligence to Dragon's new numbering commands. We believe the new numbering technology will be a game changer

Nuance does not publicize ETAs but scuttlebutt has it to expect a Ver. 16 release as early as the next few weeks but possibly not until early spring.

KnowBrainer 2020 includes the ability to hold down and release Windows, Control, Shift, Alt and Space keys. Check out the <key name> Hold down command.



-------------------------

Change "No" to "Know" w/KnowBrainer 2022
Trial Downloads
Dragon/Sales@KnowBrainer.com 
(615) 884-4558 ex 1

 09/05/2022 02:57 PM
User is offline View Users Profile Print this message

Author Icon
cdanger
New Member

Posts: 4
Joined: 09/04/2022

Thanks so much! We have been trialling KnowBrainer for the past three or four weeks and so we will give the Hold Down command a go. 

We've looked at Click by Voice and KB's Display Numbers but he uses Mouse Grid with great effect and he's incredibly fast with it. He can't see the numbers provided by the other options but he can count the squares in the 3x3 MouseGrid to get an accurate click. He's not fussed about multiple monitors at this stage - he lies on his side on a special platform so it would be difficult to locate these anyway. He even prefers a (very fast) Mouse Grid command when changing and tabs in the browser - despite the additional commands in Know Brainer! 

Very interesting news/rumour about DNS16! I know there were concerns that this version would be the end of the line since Microsoft purchased Nuance.

Woukd KB work so that we can target a location with MouseGrid, press down left mouse button and shift, and then target a second location, and release those keys? DNS had a bit of a problem combining shift and mouse click - it pops up an error. I know you're probably thinking of better ways of accomplishing this (I've been through this with him such as copy/paste - it's the demands of the game that require this or a simliar approach.

Many thanks for you help - I work for a service that provides AT and ongoing support for free to individuals with complex disabilities so there's no need for us not to purchase it - but it's good to know it's there for those that don't have access to our sort of service :-)

 09/05/2022 03:55 PM
User is offline View Users Profile Print this message

Author Icon
Lunis Orcutt
Top-Tier Member

Posts: 40207
Joined: 10/01/2006

Dragon already includes the ability to hold Left Click and drag so we should be able to expand this capability. Of course we will have to wait for Ver. 16 to be released 1st. However, KnowBrainer is primarily aimed at workflow rather than gaming. Of course it also includes VerbalBasic II which allows you to verbally create commands with no coding knowledge. We created VerbalBasic II with Eric Trump in mind, WYSWYS (What You See Is What You Say) and lots of pictures.

-------------------------

Change "No" to "Know" w/KnowBrainer 2022
Trial Downloads
Dragon/Sales@KnowBrainer.com 
(615) 884-4558 ex 1

Statistics
32389 users are registered to the KnowBrainer Speech Recognition forum.
There are currently 0 users logged in.
The most users ever online was 12124 on 09/09/2020 at 04:59 AM.
There are currently 365 guests browsing this forum, which makes a total of 365 users using this forum.

FuseTalk Standard Edition v4.0 - © 1999-2022 FuseTalk™ Inc. All rights reserved.