KnowBrainer Speech Recognition
Decrease font size
Increase font size
Topic Title: Advanced script for dragging and copying text by coordinates
Topic Summary:
Created On: 12/12/2020 03:40 PM
Status: Post and Reply
Linear : Threading : Single : Branch
 Advanced script for dragging and copying text by coordinates   - afgban - 12/12/2020 03:40 PM  
 Advanced script for dragging and copying text by coordinates   - noblemd - 12/12/2020 06:37 PM  
 Advanced script for dragging and copying text by coordinates   - Lunis Orcutt - 12/12/2020 07:08 PM  
 Advanced script for dragging and copying text by coordinates   - Alan Cantor - 12/12/2020 09:34 PM  
 Advanced script for dragging and copying text by coordinates   - Mphillipson - 12/13/2020 12:29 PM  
Keyword
 12/12/2020 03:40 PM
User is offline View Users Profile Print this message


afgban
New Member

Posts: 6
Joined: 01/16/2011

I'm having a little problem scripting a command that does something that macro recorder does pretty well – but because the website I go to frequently changes its format frequently it would be  better to do this with the scripting command.

Basically the macro recorder command I created goes like this

mouse left button down  (at a certain coordinate)

mouse move (to a certain coordinate)

mouse left button up

control C down and up

 

Thus copying an address that, in this case, appears on the webpage at a certain XY coordinate

 

But I would like to do this with an advanced script command – I looked into dragtopoint and tried it  but I got a script error command. There's not a lot of guidance on this.

Any help would be appreciated.

 

 12/12/2020 06:37 PM
User is offline View Users Profile Print this message

Author Icon
noblemd
Junior Member

Posts: 29
Joined: 08/09/2019

I use the following Advanced Scripting code to maneuver around a Web Page and carry out certain tasks. You have to determine what buttons need to be pressed and how many times to press them to go to the different Web Page buttons and other controls/links. It is also possible that the Web Page designer has named a control or link - if that is the case you can go to the control/link by using the name. For instance, if you need to enter a user name, often the entry box for that can be accessed by saying "User Name" 


'#Language "WWB-COM"

Option Explicit

Sub Main
SetMicrophone 0 ' Turn off microphone
Wait .2
SendKeys "{Tab}",1 'Go to Menu Tabs
SendKeys "{Right}",1 'Go to Network Tab
SendKeys "{Tab 6}",1 'Go to List of items
SendKeys "{Down 2}",1 'Go to 2nd List Item
SendKeys "{Enter}",1 'Open 2nd List Item
SendKeys "{Tab 8}",1 'Go to Edit Button
SendKeys "{Enter}" 'Activate Edit
Wait .5 'Wait for Edit dialogue to open
SendKeys "{Tab 2}",1 'Move to Radio Buttons
SendKeys "{Up}",1 'Move up one RB
SendKeys "{Tab 5}{Enter}" 'Tab to "OK" Button and Press
End Sub

 

 

If you are talking about a Web Page address then Mouse Right Click over the address link to get a right-click menu - choose COPY LINK item with SendKeys "{Down x}" (x=number of Down presses). Then SendKeys "{Enter}" to select menu item.

 

If you are refering to simple text of a street address then SendKeys as below

• {Move x,y} - move the mouse to (x,y)

• {ClickLeft x,y} - move the mouse to (x,y) and click the left button. (This is the same as {DownLeft x,y}{UpLeft}.)

• {DoubleClickLeft x,y} - move the mouse to (x,y) and click the left button. (This is NOT the same as {ClickLeft x,y}{ClickLeft}.)

 

What you probably need:

• {DownLeft x,y} - move the mouse to (x,y) and push the left button down.

• {UpLeft x,y} - move the mouse to (x,y) and release the left button.



-------------------------

Michael Noble
-------------------------
Using $20 USB Mic on adjustable arm ($30 total cost - 99.9% recognition). 


DPI 15.61.200.010 - Windows 11 Pro for Workstations Build 22499.1010 / MS Office 2021 LTSC



 12/12/2020 07:08 PM
User is offline View Users Profile Print this message

Author Icon
Lunis Orcutt
Top-Tier Member

Posts: 39487
Joined: 10/01/2006

We recommend avoiding the Macro Recorder like the plague because it deploys in real time and requires a lot of space which can affect the Dragon Command Browser.

 

There is a formula for capturing the Mouse Position on this forum but we just couldn't find it. If you're not using KnowBrainer 2017 (w/2020 AI Commands) consider downloading a 30 day trial, which you will find in our signature tag. KnowBrainer 2017 includes the ability to quickly create Mouse Position commands  

 

1. After installing KnowBrainer, Mouse Left Click anywhere in your browser, to bring it into focus

 

2. Hover your mouse over the button or menu item you wish to click and say Find Mouse Position to capture the pointer X & Y coordinates

 

3. Create and name a new Dragon Advanced-Scripting command

 

3. Position your cursor between Sub Main & End Sub

 

4. Press {Ctrl+v} or say paste that to paste the contexts of Windows clipboard which includes the Mouse Position (X &Y coordinates), a 1/2 second wait time and a left mouse click. It also adds an additional 1/2 second wait time so that you can repeat this process. You might also want to check out KnowBrainer VerbalBasic which allows you to say what you see (SWYS).

 

You can verbally create the appropriate scripts by simply saying them. For example, when you say Left Click, KnowBrainer will type ButtonClick 1,1 and add a 1/10 second wait time under your script. When you say Up, KnowBrainer will type SendKeys "{Up}" and KnowBrainer always knows where your verbal scripts go. You can find out about VerbalBasic II in the  KnowBrainer Manual which includes video examples. However, if you really want to learn scripting, check out the late Larry Allen's Scripting for Dragon 



-------------------------

Change "No" to "Know" w/KnowBrainer 2020
Trial Downloads
Dragon/Sales@KnowBrainer.com 
(615) 884-4558 ex 1

 12/12/2020 09:34 PM
User is offline View Users Profile Print this message


Alan Cantor
Top-Tier Member

Posts: 4214
Joined: 12/08/2007

Unfortunately, writing Dragon commands that interact with web content is a complex undertaking. As you have already discovered, the position of screen objects is not constant. So any script that relies exclusively on simulated mouse movements and mouse clicks will be unreliable.

The ideal way to interact with web content is to do so programmatically. Rather than simulating mouse actions, the script works on the level of the underlying code for the webpage. But the ability to successfully interact programmatically depends on how well a page is coded, and most webpages, maybe even the majority, are chock-full of lousy code.

Furthermore, interacting programmatically requires programming skill. Even if one possesses the right skills, the resulting script may work in one browser, but not others. For example, developing Dragon scripts that interact programmatically with Internet Explorer is do-able. But the scripts won't work in Firefox or Chrome.

Nevertheless, there are non-programmatic techniques for interacting with web content - although it helps if one has even a little programming experience.

The method that requires the fewest technical skills is figuring out the steps to perform a task, but without touching the mouse. Do it by keyboard. For example, navigate to the field you want to copy by pressing the Tab key. How many times? 5? 10? 22? 50?

SendKeys "{Tab 22}", true ' Navigate to the "Name" field
SendKeys "^c", true ' The field is selected. Copy it.

Alternatively, can you reach the target by navigating backwards, bottom to top instead of top to bottom?

SendKeys "+{Tab 99}", true ' Shift + Tab to the "Name" field

A script that tabs or back tabs through a page might work well enough, but I can almost guarantee that the scripts will occasionally fail.

A related approach, which works a little more reliably, is to use the browser's "Find" feature to reach, or get close to, the target. Let's assume there is a field labelled "Address:" that you want to copy:

SendKeys "^f" ' Find
Wait .5 ' You may need a short delay here
SendKeys "Address:{Enter}"
SendKeys "{Esc}" ' Cancel the search
Wait .5 ' A short delay cannot hurt here!
SendKeys "{Tab}" ' Tab into the Address field. But the field isn't selected.
SendKeys "{Home}" ' So let's select it. go to the start of the field...
SendKeys "+{End}" ' ... and select to the end of the field
SendKeys "^c" ' Copy it

There are more advanced non-programmatic methods that are more reliable, but one can get far using only the methods I've illustrated above.

It may take lot of trial-and-error experimentation before scripts like these are good enough for daily use.

One complicating consideration: you might find that you need to substitute SendSystemKeys or SendDragonKeys for SendKeys. The syntax for SendSystemKeys and SendDragonKeys is virtually the same, but not it's not exactly the same as for SendKeys.
 12/13/2020 12:29 PM
User is offline View Users Profile Print this message

Author Icon
Mphillipson
Top-Tier Member

Posts: 280
Joined: 09/22/2014

With the Click by Voice Google Chrome extension, which also works with Microsoft Edge, it has the ability to copy a link if you append :s to the command.

For more info please see the following link:

https://github.com/mdbridge/click-by-voice/blob/master/doc/activation_commands.md 152

An example of the KnowBrainer script I use is at the following URL:

https://www.screencast.com/t/YntlSiQz176



-------------------------

Thanks Mark


 


Dragon Professional Advanced Scripting/KnowBrainer Scripts
Video Examples of Coding by Voice



Statistics
32213 users are registered to the KnowBrainer Speech Recognition forum.
There are currently 1 users logged in.
The most users ever online was 12124 on 09/09/2020 at 04:59 AM.
There are currently 343 guests browsing this forum, which makes a total of 344 users using this forum.

FuseTalk Standard Edition v4.0 - © 1999-2022 FuseTalk™ Inc. All rights reserved.