The blog:
https://blog.google/technology/ai/google-project-gameface/

and it's available on GitHub:

https://github.com/google/project-gameface

-------------------------

DPI 15.61 | Plantronics D100 | Windows 10 Home | Intel(R) Core(TM) i5-10210U CPU 1.60GHz

"Confidence isn't walking into a room thinking you are better than everyone; it's walking in not having to compare yourself at all."

This is an interesting project and potentially very useful to disabled users. The Facial Gestures work very well. I'm also an Eviacam user so I could compare the smoothness of the pointer movement. Precise control of the pointer in Gameface is slightly harder than in Eviacam, no matter what Cursor Speed settings I was using. 

 

I needed to run the program as administrator in order to get my WebCam to appear. 

 

I think, it would be better if Gameface were a tray application (one less item in the taskbar).

 

However, it's currently not really usable for me because the CPU footprint is way too high:

 

 

-------------------------

https://speechproductivity.eu

Turbocharge your Dragon productivity with 40+ Power Addons

Speech Productivity Forum

I was curious, so I tried this Google Project GameFace mouse.

Unfortunately I wasn't able to get the mouse cursor working at all. It was tracking inside the camera window in the app, but not moving the cursor in the "OS" UI. Probably some permissions problems. Even though I did run as administrator.


The face gestures work pretty well. I was able to use them to control the Dragon microphone, both toggle or PTT. For that matter, mouse clicks also work, although I had to move the actual cursor by my trackball. Moving the mouse cursor to the center of the display also worked. For the record, moving the mouse cursor between displays worked somewhat, but it had the very common problem of getting confused because my displays have dramatically different sizes and scaling. But of course, I can do all of this by speech commands, with the minor exception of microphone control (if I'm not using SpeechStart+) and PTT.

 

Its keyboard bindings are limited -  e.g. it could not bind  a face gesture to ScrollLock, which is my default microphone control key for Dragon. So I had to find another key. But that's something that would be easy to fix since it's open source.

 

Unfortunately, it consumed 50 percent of my CPU. I am jealous of you, @diligence - you have a much faster system than my Surface Book 3.

I was tickled by the thought of raising my eyebrows as PTT but that didn't work all that well. Microphone toggle might be useful.

Makes me think about the possibility of a hybrid interface. Not just speech and face mouse, but possibly using the face mouse for coarse control combined with a trackball for very fine control.

 

Nice that it's open source. In theory, in my copious spare time,  I might be able to fix the mouse/cursor movement myself.

I'll give this another try if anyone can tell me how to get the mouse actually working. But that would be just for fun, not worth 50 percent of my CPU.

-------------------------

DPG15.6 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, MS Surface Book 3, Intel Core i7-1065G7 CPU @ 1.3/1.5GHz (4 cores, 8 logical, GPU=NVIDIA Quadro RTX 3000 with Max-Q Design.

Just my two cents worth; Nothing beats SmartNav, NaturalPoint and USB foot pedals programmed to click the mouse.

-------------------------

Thanks Mark

 

Dragon Professional Advanced Scripting/KnowBrainer Scripts
Video Examples of Coding by Voice

 

Couldn't agree more, we have a number of SmartNavs currently available, reconditioned and new but they will not be here for long. Contact us via the website on the signature below if you are interested.

-------------------------

I also just tried Gameface. Had the same result with the cursor as did Ag - the cursor didn't move on my screen, even though within the app it appears to be picking it up.

Eyebrow gestures didn't work so well with my glasses on. Worked much better when I took them off, but then I can't see clearly : ( Mouth gestures work okay, as long as I look directly at the camera. But since I use dual screens, when I look to each side, the program can't pick up what is away from the camera.

I had hopes I could at least use this for mouse clicking. I am happy as pie moving the cursor with SmartNav. I would like to find a better way to click. There is too much latency clicking with Dragon, and my eyes get tired holding the cursor in position with the SmartNav. A good thing about Gameface is that the clicks are nearly instantaneous. Very little delay. Too bad there isn't more I can use.