stevehorsfield / poweroverlay Goto Github PK

Implementation in progress.

Should allow a user to specify an image and then search for that image in a target area.

Target area can be relative to enter desktop, specific monitor, application window or cursor. Clipping to desktop, screen or window as selected.

Search should allow for perceptual colour matching to combat rendering adjustments made for colour profiles.

Native search could be computationally costly, especially in terms of cache alignment. User should be able to specify a target rectangle side aligned relative to a selected edge center or vertex.

Cursor should be moved to the found image.

Where more than one match is found, selection should support nearest to search anchor, nearest to cursor and highest colour accuracy matching modes.

The Win32 APIs provide a fairly broken experience for synthetic user input. Pointers and keyboard devices are unaware of the active software but the Windows APIs require various hacks and don't work uniformly across applications.

A better design is to have a synthetic HID driver. This would never be signed by Microsoft and would therefore require enabling installation of test drivers. Furthermore, it would open an attack vector for a malicious agent. Nevertheless, it is the only way to overcome the various limitations of the Win32 API that have been observed, and the only way to provide a truly authentic user experience that mirrors the user's experience and expectations.

The Windows platform does allow for custom software drivers and has particular support for forwarding HID events to the native keyboard and mouse class drivers. So this is entirely feasible.

Such as only if app not running

Mute speaker/microphone, volume etc.

Open window on active monitor, if option enabled.

Useful to see how an action executed and where it ended it it didn't complete.

Delays are needed to prevent unintended double clicks but also to allow windows etc to appear.

Nicer UI to allow delays directly within key/mouse sequences.

Extend PowerOverlay to include a note taking feature where sets of notes can be linked to a menu and (optionally) shown as floating topmost windows. Each menu could support an unlimited number of simple now with different font choices.

Notes would be auto-persisted when changed.

Such as context Pass/Fail/Error. Per action options on context.

User controls would be easier to manage and adapt.

It could be useful to be able to switch to an application, do something and then switch back to the original application.

Current UI is functional but ugly and clunky.

Add support for mouse cursor movement and button/wheel function using SendInput API.