Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
At work, I have Windows XP, DNS v. 9.5, and a 17 inch monitor. At home, I have Windows 7, DNS v. 10.1, and a 21 inch monitor. When I write commands using an EMR (that I have available in test-until going live in April-that is not MSAA supported) at home using mouse positioning coordinates, the coordinates are off when I transfer these commands (in XML format) to my work computer. So I wind up having to do all of the mouse-positioning over again, which obviously is double work. When writing commands into our current EMR that is MSAA supported, the mouse coordinates are properly transferable between systems.
Is the problem:
1. Different monitor sizes? Are mouse-positioning commands usually relative to the open window, or to the monitor?
2. Does it have to do with screen resolution and orientation?
3. Is this difference in mouse positions a function of one program being MSAA supported and the other not?
4. Is there something that I can do to make all mouse-positioning coordinates the same regardless of operating system, DNS version, monitor size, screen resolution, and whether MSAA supported?
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
Thread has been edited.
Edit Count :
1 times.
Date Last Edited :
02/07/2010
Time Last Edited :
10:05 AM
MDH
simply the screen resolution (and orientation obviously) and nothing else. Screen size is irrelevant, MSAA is irrelevant. So you can have a 21 inch monitor and a 17 inch monitor no problem as long as both screen resolutions are set to e.g. 1600 x 1200.
Lindsay
There are slight differences in operating systems, in terms of window frame templates, styles etc which may put the coordinates out slightly (particularly between XP and Vista/Windows 7 but not much difference between Vista and Windows 7). Also make sure you have the same toolbars etc for the applications with the different systems e.g. the same toolbars for Internet Explorer.
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
Lindsay,
Thank you very much. But why are mouse coordinates transferable with commands written into my current EMR, and not into the new EMR? This is all independent of screen resolution and orientation.
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
Do you mean that you have two systems, same resolution and orientation, and your mouse commands for your old EMR are transferable between the two systems yet the mouse commands for your new EMR are not?
If so it would be best to compare screenshots (Alt+PrtScr) from the two systems to see what is happening. There are other things that affect the mouse position like the DPI, text sizes etc but if you're old EMR commands are transferable then it sounds like these settings must be a okay. What is the difference between the systems in terms of operating system and screen size and what resolution are you actually using?
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
Lindsay,
Thanks again. No, the screen resolutions, screen sizes, resolutions, and operating systems (XP versus Windows 7) are different home versus office, but the mouse positions done in commmands at home are transferable to my office system in our current EMR exactly, but not in our soon-to-be new EMR.
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
Thread has been edited.
Edit Count :
2 times.
Date Last Edited :
02/07/2010
Time Last Edited :
04:39 PM
MDH
if you imagine a widget on a maximised application window which is exactly in the centre of the screen and the resolution 1024x768 then the mouse position of that widget will be 512, 384
With the same maximised application on a screen with a resolution of 1600x1200 the very same widget will still be shown in the middle of the screen but the mouse coordinates will then be 800,600. Therefore I am surprised the mouse position commands are transferable, I imagine the resolutions on your two different systems must be close or they must be big target areas for the widgets:-)
Another example would be a widget at an example position of 1300, 900 on the 1600x1200 screen, a mouse position command to click such a widget would be invalid mouse coordinates on the 1024x768 screen. Otherwise beats me how you can transfer them with perfect results :-)
Anyway your best chance of success in transferring the mouse position commands would be with the same resolutions.
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
Lindsay,
Thanks for your help. My only explanation for this difference is that the current EMR mouse coordinates must for some reason be based on the window, and the new EMR on the screen resolution/orientation.
Group : Top-tier Member
Active : 09/07/2010 @ 09:52 AM
Posts : 693
I think Lindsay is mostly right. I have written macros that work on different sized monitors by making decisions based on the pixel width and height of the screen; in other words, screen resolution. But the position of screen targets can change when modifying display properties such as active window border width and menu font size. These properties are independent of the resolution.
Although scripting a single command that works under all conditions may be possible, it may not be worth the effort. How long does it take to adjust your macros for each PC? If it's only takes 20 minutes per computer, consider doing it manually. Only if it takes a significant chunk of time to calibrate the macros (or the macros require ongoing maintenance) would I consider going the extra mile. (And that mile will probably be a long one!)
Macros that act on the basis of pixel colours and mouse pointer changes may be the solution. They can be extremely reliable, although they take time to develop.
Group : Site Administrator
Active : 09/08/2010 @ 03:26 PM
Posts : 11717
Thread has been edited.
Edit Count :
1 times.
Date Last Edited :
02/07/2010
Time Last Edited :
10:18 PM
If you're adding Mouse Position commands into DNS, they are actually Window position commands. For example, the following:
Sub Main SetMousePosition 1,1,1 Wait 1 ButtonClick End Sub
will simply left mouse click the upper left-hand corner of the current Window it is deployed in. Although not a guarantee, you may still experience some universality between systems.
MouseGrid commands such as the following:
Sub Main MouseGrid 1,5 Wait .5 MouseGrid 1,7 Wait .5 ButtonClick End Sub
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
Lunis,
Thank you for your help. What seems to be happening is that the mouse=positioning commands (as per your first example) are indeed Window specific commands in my current EMR, but not in the new one.
Group : Top-tier Member
Active : 09/07/2010 @ 09:52 AM
Posts : 693
I don't know how to search for pixel colours or mouse cursors via Advanced Scripting. I use either Macro Express or AutoHotkey to do the grunt work, and then activate the command via Advanced Scripting.
Schematically, the Macro Express or AutoHotkey command looks something like this:
Name: Activate OK button Description: Click "OK" if colour 12345 appears anywhere on the screen Activation: Alt + O
Search_Entire_Screen_for_Pixel_Colour 12345 If Found Mouse Move 100, 200 Left Click Else Do nothing
The Advanced Scripting Command looks something like this:
Group : Top-tier Member
Active : 03/29/2010 @ 02:22 PM
Posts : 224
Another thought if you have to often modify screen location parameters for mouse clicking &etc. is to write a simple "register position" command or sub routine.
E.g., I work with several different document reading/coding software programs on several different computers (and sometimes through a remote desktop, VNC or citrix window). The most frustrating of them have no keyboard shortcuts (?? -- I know!!) for simple things like turning pages and going to the next document. Yet they all have a button somewhere for navigating next/previous page/document (so four buttons if you're playing along at home) and sometimes the programs have buttons for rotating left/right, zooming in/out and scrolling up/down. My command "register position" has me click the appropriate buttons or window locations and automatically records the individual mouse coordinates into a text file (which is left resident on that computer) and then the nav commands read those mouse coords from that file as needed.
Actually, I use AHK for the grunt work, but I remember seeing an advanced scripting command on this forum for capturing the current mouse coords, so in theory it can be done all w/ the Dragon.
PG
DNS 9.5 Pro Gen-Large Vocab, on various high-speed, lots of memory, AMD and Intel-based PCs (home, office, laptops) and Plantronics CS50-USB mics - using local user in default directory that is imported and exported as needed to USB flash drive for portability.
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
Thread has been edited.
Edit Count :
4 times.
Date Last Edited :
02/08/2010
Time Last Edited :
04:07 PM
PG (haven't seen you around here for a while) et all
You can use the following script to convert any application-specific mouse position command to any resolution. The only caveat is that the resolution of the screen that you want to use the mouse position commands on must have the same aspect ratio (4:3, 16:9 etc) as the screen on which you wrote the original working mouse position commands.(Otherwise the commands won't be as accurate) All you need to do is change the green numbers to the screen resolution on which the original mouse position commands work and change your original working mouse position coordinates in the red lines and numbers in the code below.
So in the example below the original working mouse position commands were initially written for a screen resolution depicted by the green numbers(1024X760). The original script (in red) will now work for all resolutions with the same aspect ratio e.g. 800x600, 1024x760, 1600x1200. The example mouse coordinates in the example below (512, 384) would be the middle of the screen for a maximised application.
Please don't anyone tell me it's overkill because I know:-) but it's a simple script to implement.
Declare Function GetSystemMetrics Lib "user32.dll" (ByVal nIndex As Long) As Long Const SM_CXSCREEN = 0 Const SM_CYSCREEN = 1 Const WRK_WIDTH = 1024 Const WRK_HEIGHT = 768 Dim widthAspect,heightAspect As Double Sub Main width = GetSystemMetrics(SM_CXSCREEN) height = GetSystemMetrics(SM_CYSCREEN) widthAspect= width/WRK_WIDTH heightAspect = height/WRK_HEIGHT If width = WRK_WIDTH Then SetMousePosition 1,512,384 Else SetMousePosition 1,Int(widthAspect*512) , Int(heightAspect*384) End If Wait 0.1 ButtonClick 1,1 End Sub
Group : Top-tier Member
Active : 03/29/2010 @ 02:22 PM
Posts : 224
Yes that's very cool: multiplying the specified mouse coords by the ratio of the current screen res to the then screen res (from when the coords were first determined). I will implement that in my (only a few) commands that use mouse coords especially because I do sometimes change my screen res on one of my systems to handle certain programs that have unreadably small text otherwise.
Problem is, this technique won't help for using the same mouse coord command in different apps (as in my example where I have to click a button for "next page" [CLICK A BUTTON FOR NEXT PAGE?? How can there NOT be a keyboard shortcut for that?? What monster designs such a program -- and in what alternate universe is that supposed to be efficient?? -- but enough editorilizing] and that button is in a totally different location depending on the particular app I am using or if I move the windows around within the app or perhaps even if I change the display from landscape to portrait) or even in the simple case of having different Windows themes and appearances (and therefore different title/menu/button bar/border widths etc.) on different machines (even with the same res). In these cases, "registering" the mouse coords is both fun and useful! Anyway, sure beats using CASE and window titles to keep all the coords for all the situations.
PG
DNS 9.5 Pro Gen-Large Vocab, on various high-speed, lots of memory, AMD and Intel-based PCs (home, office, laptops) and Plantronics CS50-USB mics - using local user in default directory that is imported and exported as needed to USB flash drive for portability.
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
PG, If you did copy the script then copy it again because the previous version had a nasty bug that locked up naturally speaking with certain coordinates. Apologies to anyone that this happened to, I did it quickly and didn't test it properly.
It's a good point you make about certain applications having no accessibility built into them, it's none of my business but I find it remarkable that an IT manager or whoever was responsible for purchasing or commissioning an EMR didn't check these things out first. I guess it comes down to cost but at the end of the day it it is false economy, because doctors who do use accessibility software like DNS, just have to spend more time doing their thing with it. I don't know this is the case with MDH and like I say it's none of my business but it must be very frustrating whatever the reasons.
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
PG LTU,
Here is the script for getting mouse positions that you referred to, that Lindsay so kindly gave me quite some time ago, which has saved my butt as far as being able to navigate an EMR when no hot-keys or radio buttons were available. I shall forever be indebted to him, as it "broke the roadblock" for me, and opened-up so many possibilities for navigational commands.
Option Explicit Type RECT Left As Long Top As Long Right As Long Bottom As Long End Type Type POINTAPI x As Long y As Long End Type ' declare Win32 API functions Declare Function GetCursorPos Lib "user32" _ Alias "GetCursorPos" (lpPoint As POINTAPI) As Long Declare Function GetWindowRect Lib "user32" _ Alias "GetWindowRect" (ByVal hwnd As Long, _ lpRect As RECT) As Long Declare Function GetForegroundWindow& Lib "user32" () Function GetWinCurPos(hwnd As Long) As POINTAPI Dim pt As POINTAPI Dim wRect As RECT Dim result As POINTAPI GetCursorPos pt GetWindowRect hwnd, wRect result.x = pt.x - wRect.Left result.y = pt.y - wRect.Top GetWinCurPos = result End Function Sub Main Dim hwnd As Long Dim result As POINTAPI Dim clip As String hwnd = GetForegroundWindow result = GetWinCurPos(hwnd) clip = "SetMousePosition 1," & result.x & _ "," & result.y ' Clipboard clip ' End Sub
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
Lindsay,
I TOTALLY agree with you regarding choosing an EMR that has NO accessabilty built into it. I am one of 60 hospital-employed physicians. At a meeting to decide whether to upgrade to the next version or look into getting a different EMR, the physicians voted to get RFPs for a different EMR. One of the things that they disliked about our current EMR was how "clicky" it was. Of course I had eliminated 80-90% of the clicks writing custom commands. Even though I handed them these commands, only one other physician was savvy enough with DNS to use any. No one else was interested in learning, just complaining. Most physicians dictated via phone at $1000 per doc per month, and clicked away for navigation.The 25% of docs that used DNS got poor training, and never progressed beyond mediocre use of DNS.
So an EMR committee was formed with some docs, some IT people (only one of which knew anything about DNS and was the one who "trained" us), and business office people. To make a long story short, of the 35 people on the committee, only 4--including myself--voted against this new EMR. It was the most speech recognition unfriendly of the 20 we initially looked at. Obviously, that was of little concern to most, based on how docs used DNS. It was also the most inaccessable for using voice navigational commands--but heck, only 2 docs used that! Most of the docs have not even seen this new EMR, which I have access to in TEST. They are in for a big surprise since it is AT LEAST 5 times MORE CLICKY than our current EMR. But that is the direction that EMRs are going, to be able to collect discrete data to mine, code,bill, and for quality assurance and pay-for-performance info. It is easier to do this with a point-and-click system, with everything being a drop-down choice (not menu accessable) to pick something approaching what the patient may have said. Telling "the story" via free text is to be discouraged, and gets penalized. So who needs speech recognition given this scenario. However, I refuse to capitulate. With great difficulty, I found a way acceptable to the "powers that be" where I can use speech recognition within the structure of the new EMR that would allow for the discrete data input. This means that I will be dictating single sentences rather than a paragraph. So much for telling the patient's story (or using DNS most effectively!)
Thanks again for all of your help. Sorry for venting my frustrations--but you can see that we are on the same page.
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
Thread has been edited.
Edit Count :
1 times.
Date Last Edited :
02/09/2010
Time Last Edited :
07:31 AM
MDH
No problem, I don't blame you. I've just noticed you are running the EMR under Windows 7 on one of your systems and there is an accessibility tool called Inspect, just type inspect in the start menu. In the top corner of Inspect you can choose MSAA or UIAutomation. If you then position your mouse over different controls/widgets in applications like Microsoft Word and Internet Explorer you will find lots of information about the different controls/widgets on the UI. Try it on the EMR with both MSAA and UI Automation and then just position your mouse over various controls e.g. combo boxes, list boxes etc and see what information you get (you specifically want to look at control types). If there are continuous messages about being unable to ascertain control information from the mouse point when it positioned above the different widgets\controls then you're out of luck that way as well.
There is a better version of the mouse position command attached, well it additionally opens the command editor and fills all the information in for you except the name of the command.
Lindsay
I should add that if there is plenty of information about the controls all that means is that it would be very easy to develop an add-on application to control it all by voice which probaly isn't an option for you anyway but it could be easily done by the relevant people.
Group : Top-tier Member
Active : 09/08/2010 @ 12:57 PM
Posts : 682
Lindsay,
Thank you again so very much. You truly amaze me how knowledgeable you are about this stuff, and for your generosity.
I tried putting Inspect into the search under Start, and this did not get any helpful results--only got me to 4 of my documents. Then there was an option for a further search in Libraries, Homegroup, computer, custom, or internet. The computer search lead to several things, but nothing with the MSAA OR UIAutomation. What am I doing wrong? Is this available in XP also?
Group : Top-tier Member
Active : 09/08/2010 @ 05:44 PM
Posts : 881
Thread has been edited.
Edit Count :
1 times.
Date Last Edited :
02/09/2010
Time Last Edited :
02:18 PM
MDH
Inspect is only available with Windows 7 and not XP. It just shows you the UI Automation and MSAA properties of a user interface in terms of each and every control. Of course these are Microsoft's accessibility tools and although you don't need to specifically build these accessibility options into a program (like your EMR) if the EMR was written with a specific programming development environment then these accessibility options will be available automatically which allows easily developing addon applications that make accessibility much easier. One-day all programming environments will build such accessibility into programs automatically but until then...
Thanks for the kind words but I'm just learning myself programming wise in the big scheme of things but because I'm doing stuff specific to DNS hopefully some of it is useful on the forum for users and other programmers. I know it's over technical for some people but if it helps just one person then it's worth posting.
Incidentally it's unlikely when using inspect with Windows 7 that either API (MSAA or UIA) will reveal anything with your EMR if Max's ShowNumbersPlus didn't manage to highlight anything (this is only using MSAA I believe). If not and anyway try it with Microsoft Word and some other applications and you will see what I mean.
Lindsay
I think you can get it to work with XP if you download the Windows 7 SDK (as it is supported by XP SP3) , and if I am correct is installed automatically with Windows 7 and the .net framework, at least with the ultimate version.
however it is a very big download as I think you need to download the.net framework as well (3.5 SP1). The easiest thing to do is to try it with your Windows 7 set up first to see if it reveals anything interesting about your EMR (don't raise your hopes to mutch) .
Group : Top-tier Member
Active : 09/08/2010 @ 03:17 PM
Posts : 2586
Quote:
Inspect is only available with Windows 7 and not XP
Lindsay,
I just did a search on my Windows 7 Ultimate 64-bit for a file called "inspect", on "my computer", nothing turned up. When I tried typing it into the run box, an error was returned. Of course, it might be that the file name is different in the German version of Windows, but not very likely. Would you be so kind to provide the path to the file on your computer?
For some reason I thought this was installed as part of the .net framework which is automatically installed with Windows 7. However I have just looked at my programs and features in the control panel and I did install the Windows 7 SDK at some point. I also gave someone else this information and they found it (I guess they must have installed the Windows 7 SDK as well). Anyway looks like that is not the case and my apologies to anyone who has been trying to find it. I guess you need the Windows 7 SDK.
Posted : Sunday 02/07/2010 09:47 AM 
jpg.JPG)
