Invoke Virtual Keyboard on Desktop (Apple)?

[richmond62]

CAN OXT LITE invoke this without the machine-user having already turned this on via system settings.

Tip: The Accessibility Keyboard is used for the macOS Keyboard Viewer, which lets you type in different languages when you change input sources. See Use the Keyboard Viewer.

https://forums.developer.apple.com/forums/thread/744835

This would be extremely useful for disabled end-users.

My primary interest in this respect is with MacOS, but it would also be a good thing to know if this is possible re Windows and Linux.

[tperry2x]

for linux, you could just run:

Code: Select all

get shell("onboard")

if you wanted to do it programmatically. If it's not installed, it's easy to get the IDE to ask for that.

You could even 'roll-your-own' onscreen keyboard, built into the IDE by utilising xdotool and synthesizing keystrokes.

[richmond62]

That presupposes your system has Onboard installed . . .

[tperry2x]

That presupposes your system has Onboard installed . . .

If it's not installed, it's easy to get the IDE to ask for that.

[richmond62]

If it's not installed, it's easy to get the IDE to ask for that.

Може би?

[richmond62]

https://support.apple.com/guide/mac-hel ... 0/mac/11.0

[OpenXTalkPaul]

This could probably be done for macOS using XTBuilder FFI -> ObjectiveC.

That system Font Picker I had it bringing up basically can do the same thing, I was using it's character chooser (macOS system) palettes on-screen 'type' in unicode characters (Emojis) into the OXT Message Box.

I also wrote my own 'on-screen' keyboard as a canvas drawn (NOT-native and therefore cross-platform) widget, had it scripted to working to input text into a field.

[richmond62]

As MacOS has a built-in 'Accessability' keyboard there should be a way to send virtual key-commands to the operating system to make this visible.

[richmond62]

Here I am at work, and the first thing that hits me is this:
-

Screen Shot 2024-09-10 at 10.34.28 am.png (17.19 KiB) Viewed 1549 times

-
So, as predictable as Spring coming after Winter, my question is . . .

How can one send multiple modifier keyDowns all at once in such a fashion that the Operating System understands them?

Oh, and if you're wondering (which 99% you are not) these keyDowns (OPTION-COMMAND-F5)does NOT bring up the Accessibility Anything on MacOS 10.7.5 (that's the machine I am typing this on): but that does not surprise me: I am targeting MacOS 11 upwards.

[richmond62]

This:

Code: Select all

if application "KeyboardViewer" is running then
      quit application "KeyboardViewer"
  else
      activate application "KeyboardViewer"
  end if

Should work in AppleScript . . .

But ONLY pre MacOS Catalina as the 'KeyBoardViewer' application was removed at MacOS Catalina, nad the
Accessibility KeyBoard does the same job subsequently.

[OpenXTalkPaul]

Here I am at work, and the first thing that hits me is this:
-
Screen Shot 2024-09-10 at 10.34.28 am.png
-
So, as predictable as Spring coming after Winter, my question is . . .

How can one send multiple modifier keyDowns all at once in such a fashion that the Operating System understands them?

Oh, and if you're wondering (which 99% you are not) these keyDowns (OPTION-COMMAND-F5)does NOT bring up the Accessibility Anything on MacOS 10.7.5 (that's the machine I am typing this on): but that does not surprise me: I am targeting MacOS 11 upwards.

I don't think you can send keysDown out to the OS, at least not directly. Also you can no longer capture keyboard input when the Engine is not the focusedforegrund application (you could in earlier LC / macOS, but this should've always been considered a security no-no). You could do this with AppleScript by using 'do tScript as AppleScript', I think accessibility and UI scripting have to be enabled for that to work. I've used something similar to control GarageBand Record/Stop (which ironically has no built-in AppleScript support), I'm sure there's a few other ways to do it.

[OpenXTalkPaul]

Interesting article on the topic:
https://nshipster.com/accessibility-keyboard/

I wonder if it couldn't just be launched like any other app. It may need to first be enabled in the accessibility control panel.

[OpenXTalkPaul]

In response to a (4 month old) comment on YouTube about Sonoma compatibility for HIDAPI, I updated the Mac 64bit lib binary, which is like an instant update since the API does not appear to have changed much.
https://github.com/PaulMcClernan/OpenXT ... x86_64-mac

Interestingly it worked fine on Mohave and BigSur with the older version I had in there (0.10.0), but when I load the extension with HIDAPI new version (0.14.0), when I load the extension OXT asked for permission to read keystrokes from other applications, which is a sandbox/security Entitlement required for many apps such as Emulators like DOSBox to work properly It also affected some Logitec software, there's a lengthy article on their website about the issues this security measure can cause.
The bottom line is it looks like on macOS Catalina+ this library needs this entitlement to be user authorized (although, like I said the old version worked without asking on macOS 11).

Screen Shot 2024-09-13 at 12.58.30 AM.png (71.22 KiB) Viewed 1242 times

Screen Shot 2024-09-13 at 1.20.59 AM.png (211.52 KiB) Viewed 1242 times

[richmond62]

OK: naive "cabbage" asks:
-

Screenshot 2024-09-13 at 14.42.13.png (72.93 KiB) Viewed 1064 times

-
What do I do next?

[OpenXTalkPaul]

I wonder if it couldn't just be launched like any other app. It may need to first be enabled in the accessibility control panel.

Apparently not, I found the app inside a framework bundle but launching the app from there does nothing.
I'm pretty sure your app would need the appropriate entitlements/permissions granted by end user to do anything like this on newer versions of macOS.

Of course you could just bring it up from the keyboard Status Menu in the system menubar if you enable that (labeled "show input menu in menubar" in the keyboard control panel),

[tperry2x]

Може би?

Like this:

like-this.png (112.01 KiB) Viewed 979 times

onboard check linux.oxtstack

(1.63 KiB) Downloaded 26 times

I'll put a windows version together when I get a moment.
Edit: was super easy.

onscreen keyboard win.oxtstack

(956 Bytes) Downloaded 28 times

On MacOS, you might have to 'roll your own' in a new stack window, to ensure it works across all versions of MacOS. (shame you need to support MacOS really)

[richmond62]

Clever.

BUT: Mac-less.

(shame you need to support MacOS really)

Hmm: Mac has a larger desktop market share than Linux.

Shame I have to clean the toilet every morning: but it would be far more shameful if I didn't.

[tperry2x]

Shame I have to clean the toilet every morning: but it would be far more shameful if I didn't.

That's a good analogy. Using the mac is increasingly about as unpleasant as cleaning a toilet. At least cleaning a toilet, you get results - rather than needing to recompile your toilet
Okay, I'll see if there's a way to script showing the built in MacOS keyboard viewer from within OXT...

edit: only this so far, which I think assumes you already have the keyboard layouts shown in your menubar:
(applescript)

Code: Select all

tell application "System Events"
    tell process "TextInputMenuAgent"
        tell menu bar item 1 of menu bar 2
            click
            tell menu 1
                if exists menu item "Show Keyboard Viewer" then
                    click menu item "Show Keyboard Viewer"
                else
                    click menu item "Hide Keyboard Viewer"
                end if
            end tell
        end tell
    end tell
end tell

We could of course run that applescript in OXT, but there's probably a way to make MacOS show the keyboard icon in the menu through a command too...

[tperry2x]

I'm not at a mac at the moment, but I wonder if you can get it to run with this applescript:

Code: Select all

tell application "Finder"
if application "KeyboardViewer" is running then
      quit application "KeyboardViewer"
  else
      activate application "KeyboardViewer"
  end if
end tell

That should toggle it. If that works, we can make a stack that just runs the applescript. (unless MacOS blocks it of course )

Fun fact by the way, if like me, your MacBook Air that you rely on to produce the mac version of OXT bites the dust - as in, the battery swells and goes pop, the OS powers off your mac and won't let it boot. Removing the battery completely and keeping it as a static desk machine is still possible and then it'll boot right up. The only thing is you lose your battery icon in the menu bar, of course (and you pray for no powercuts )

[richmond62]

Fun fact . . .

About 18 years ago I bought a Mac laptop for my wife, and after 8 years the hard drive went 'wonk' and I could not install a new hard disk into the hole in the laptop: so I superglued an external hard drive case to its underside and she continued to use it booting from the external drive for a further 5 years.

Personally I would not touch a Mac laptop with a 10 foot pole: talk about fitting a gallon into a pint pot.

I have 2 laptops: one 32-bit Toshiba, and one 64-bit 'something' which my father left behind when he died: useful for watching movies connected to a TV in our bedroom: useful for when I visit Mother: the 32 bit runs Debian + XFCE, the 64 bit runs Xubuntu: neither a STARIN on the machines,