Page 1 of 2
Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 06, 2024 8:41 am
by richmond62
CAN OXT LITE invoke this without the machine-user having already turned this on via system settings.
Tip: The Accessibility Keyboard is used for the macOS Keyboard Viewer, which lets you type in different languages when you change input sources. See Use the Keyboard Viewer.
https://forums.developer.apple.com/forums/thread/744835
This would be extremely useful for disabled end-users.
My primary interest in this respect is with MacOS, but it would also be a good thing to know if this is possible re Windows and Linux.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 06, 2024 8:44 am
by tperry2x
for linux, you could just run:
if you wanted to do it programmatically. If it's not installed, it's easy to get the IDE to ask for that.
You could even 'roll-your-own' onscreen keyboard, built into the IDE by utilising xdotool and synthesizing keystrokes.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 06, 2024 8:48 am
by richmond62
That presupposes your system has Onboard installed . . .
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 06, 2024 8:49 am
by tperry2x
richmond62 wrote: ↑Fri Sep 06, 2024 8:48 am
That presupposes your system has Onboard installed . . .
tperry2x wrote: ↑Fri Sep 06, 2024 8:44 am
If it's not installed, it's easy to get the IDE to ask for that.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 06, 2024 8:50 am
by richmond62
If it's not installed, it's easy to get the IDE to ask for that.
Може би?
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 06, 2024 10:12 am
by richmond62
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Tue Sep 10, 2024 3:02 am
by OpenXTalkPaul
This could probably be done for macOS using XTBuilder FFI -> ObjectiveC.
That system Font Picker I had it bringing up basically can do the same thing, I was using it's character chooser (macOS system) palettes on-screen 'type' in unicode characters (Emojis) into the OXT Message Box.
I also wrote my own 'on-screen' keyboard as a canvas drawn (NOT-native and therefore cross-platform) widget, had it scripted to working to input text into a field.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Tue Sep 10, 2024 6:14 am
by richmond62
As MacOS has a built-in 'Accessability' keyboard there should be a way to send virtual key-commands to the operating system to make this visible.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Tue Sep 10, 2024 7:40 am
by richmond62
Here I am at work, and the first thing that hits me is this:
-
- Screen Shot 2024-09-10 at 10.34.28 am.png (17.19 KiB) Viewed 1551 times
-
So, as predictable as Spring coming after Winter, my question is . . .
How can one send multiple modifier keyDowns all at once in such a fashion that the Operating System understands them?
Oh, and if you're wondering (which 99% you are not) these keyDowns (OPTION-COMMAND-F5)does NOT bring up the Accessibility Anything on MacOS 10.7.5 (that's the machine I am typing this on): but that does not surprise me: I am targeting MacOS 11 upwards.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Tue Sep 10, 2024 7:44 am
by richmond62
This:
Code: Select all
if application "KeyboardViewer" is running then
quit application "KeyboardViewer"
else
activate application "KeyboardViewer"
end if
Should work in AppleScript . . .
But ONLY pre MacOS Catalina as the 'KeyBoardViewer' application was removed at MacOS Catalina, nad the
Accessibility KeyBoard does the same job subsequently.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 2:46 am
by OpenXTalkPaul
richmond62 wrote: ↑Tue Sep 10, 2024 7:40 am
Here I am at work, and the first thing that hits me is this:
-
Screen Shot 2024-09-10 at 10.34.28 am.png
-
So, as predictable as Spring coming after Winter, my question is . . .
How can one send multiple modifier keyDowns all at once in such a fashion that the Operating System understands them?
Oh, and if you're wondering (which 99% you are not) these keyDowns (OPTION-COMMAND-F5)does NOT bring up the Accessibility Anything on MacOS 10.7.5 (that's the machine I am typing this on): but that does not surprise me: I am targeting MacOS 11 upwards.
I don't think you can send keysDown
out to the OS, at least
not directly. Also you can no longer
capture keyboard input when the Engine is not the focusedforegrund application (you could in earlier LC / macOS, but this should've always been considered a security no-no). You could do this with AppleScript by using 'do tScript as AppleScript', I think accessibility and UI scripting have to be enabled for that to work. I've used something similar to control GarageBand Record/Stop (which ironically has no built-in AppleScript support), I'm sure there's a few other ways to do it.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 3:02 am
by OpenXTalkPaul
Interesting article on the topic:
https://nshipster.com/accessibility-keyboard/
I wonder if it couldn't just be launched like any other app. It may need to first be enabled in the accessibility control panel.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 6:02 am
by OpenXTalkPaul
In response to a (4 month old) comment on YouTube about Sonoma compatibility for HIDAPI, I updated the Mac 64bit lib binary, which is like an instant update since the API does not appear to have changed much.
https://github.com/PaulMcClernan/OpenXT ... x86_64-mac
Interestingly it worked fine on Mohave and BigSur with the older version I had in there (0.10.0), but when I load the extension with HIDAPI new version (0.14.0), when I load the extension OXT asked for permission to read keystrokes from other applications, which is a sandbox/security Entitlement required for many apps such as Emulators like DOSBox to work properly It also affected some Logitec software, there's a lengthy article on their website about the issues this security measure can cause.
The bottom line is it looks like on macOS Catalina+ this library needs this entitlement to be user authorized (although, like I said the old version worked without asking on macOS 11).
- Screen Shot 2024-09-13 at 12.58.30 AM.png (71.22 KiB) Viewed 1244 times
- Screen Shot 2024-09-13 at 1.20.59 AM.png (211.52 KiB) Viewed 1244 times
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 11:43 am
by richmond62
OK: naive "cabbage" asks:
-
- Screenshot 2024-09-13 at 14.42.13.png (72.93 KiB) Viewed 1066 times
-
What do I do next?
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 2:13 pm
by OpenXTalkPaul
OpenXTalkPaul wrote: ↑Fri Sep 13, 2024 3:02 am
I wonder if it couldn't just be launched like any other app. It may need to first be enabled in the accessibility control panel.
Apparently not, I found the app inside a framework bundle but launching the app from there does nothing.
I'm pretty sure your app would need the appropriate entitlements/permissions granted by end user to do anything like this on newer versions of macOS.
Of course you could just bring it up from the keyboard Status Menu in the system menubar if you enable that (labeled "show input menu in menubar" in the keyboard control panel),
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 4:30 pm
by tperry2x
Like this:
- like-this.png (112.01 KiB) Viewed 981 times
I'll put a windows version together when I get a moment.
Edit: was super easy.
On MacOS, you might have to 'roll your own' in a new stack window, to ensure it works across all versions of MacOS. (shame you need to support MacOS really)
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 6:48 pm
by richmond62
Clever.
BUT: Mac-less.
(shame you need to support MacOS really)
Hmm: Mac has a larger desktop market share than Linux.
Shame I have to clean the toilet every morning: but it would be far more shameful if I didn't.
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 6:53 pm
by tperry2x
richmond62 wrote: ↑Fri Sep 13, 2024 6:48 pm
Shame I have to clean the toilet every morning: but it would be far more shameful if I didn't.
That's a good analogy. Using the mac is increasingly about as unpleasant as cleaning a toilet. At least cleaning a toilet, you get results - rather than needing to recompile your toilet
Okay, I'll see if there's a way to script showing the built in MacOS keyboard viewer from within OXT...
edit: only this so far, which I think assumes you already have the keyboard layouts shown in your menubar:
(applescript)
Code: Select all
tell application "System Events"
tell process "TextInputMenuAgent"
tell menu bar item 1 of menu bar 2
click
tell menu 1
if exists menu item "Show Keyboard Viewer" then
click menu item "Show Keyboard Viewer"
else
click menu item "Hide Keyboard Viewer"
end if
end tell
end tell
end tell
end tell
We could of course run that applescript in OXT, but there's probably a way to make MacOS show the keyboard icon in the menu through a command too...
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 7:12 pm
by tperry2x
I'm not at a mac at the moment, but I wonder if you can get it to run with this applescript:
Code: Select all
tell application "Finder"
if application "KeyboardViewer" is running then
quit application "KeyboardViewer"
else
activate application "KeyboardViewer"
end if
end tell
That should toggle it. If that works, we can make a stack that just runs the applescript. (unless MacOS blocks it of course
)
Fun fact by the way, if like me, your MacBook Air that you rely on to produce the mac version of OXT bites the dust - as in, the battery swells and goes pop, the OS powers off your mac and won't let it boot. Removing the battery completely and keeping it as a static desk machine is still possible and then it'll boot right up. The only thing is you lose your battery icon in the menu bar, of course (and you pray for no powercuts
)
Re: Invoke Virtual Keyboard on Desktop (Apple)?
Posted: Fri Sep 13, 2024 7:25 pm
by richmond62
Fun fact . . .
About 18 years ago I bought a Mac laptop for my wife, and after 8 years the hard drive went 'wonk' and I could not install a new hard disk into the hole in the laptop: so I superglued an external hard drive case to its underside and she continued to use it booting from the external drive for a further 5 years.
Personally I would not touch a Mac laptop with a 10 foot pole: talk about fitting a gallon into a pint pot.
I have 2 laptops: one 32-bit Toshiba, and one 64-bit 'something' which my father left behind when he died: useful for watching movies connected to a TV in our bedroom: useful for when I visit Mother: the 32 bit runs Debian + XFCE, the 64 bit runs Xubuntu: neither a STARIN on the machines,