OXT Cloud IDE v0.00000001

[OpenXTalkPaul]

OXTWebIDE_v000001.jpg (186.58 KiB) Viewed 8471 times

Feeling inspired by HyperCard Simulator, I played around with our Emscripten Engine for a little bit this weekend and copied some components of "FreeGUI" into the OXT Web Playground stack.

[OpenXTalkPaul]

I haven't uploaded it to the repo yet, need to add some more IDE before I bother with that.

The Emscripten Engine is missing some features.
Most importantly it does not have a functional "wait".

It's also lacking a window manager due to the fact that 'the platform" is not a platform at all and is a web browser. Thus Stacks have NO window frames at all. So we need library that is a purely OXT based psudo-Window manager that creates and applies Window frames and 'decorations along with their functionality (close, grab, expand, minimize, etc.). I've been thinking a widget could be created specifically for this purpose, which would eliminate or at least reduce the amount of window management scripting a stack author would need to do to create a faux-Mac or faux-Windows window control for a stack. Such a widget would need several appearances of the frames / decorations (and preferably the ability to set a custom look of decorations) for different window types (like palettes).

[richmond62]

I always wondered what the Emscripten engine, with its cryptic name, and no proper documentation I could find, was for.

[FourthWorld]

I always wondered what the Emscripten engine, with its cryptic name, and no proper documentation I could find, was for.

With Google and DuckDuckGo I find tons of info when searching "Emscripten".

[FourthWorld]

Another possibility: an OXT-specific browser.

It doesn't replace web browsers. It compliments them.

Just as web browsers run HTML apps, an OXT browser could run stackfile apps.

No, it wouldn't be as popular. That would be the point, a specific network riding on top of the web but only accessible to OXT users.

Not everything needs to be for everyone.

Think groupware over publicware.

[OpenXTalkPaul]

Another possibility: an OXT-specific browser.

It doesn't replace web browsers. It compliments them.

Just as web browsers run HTML apps, an OXT browser could run stackfile apps.

No, it wouldn't be as popular. That would be the point, a specific network riding on top of the web but only accessible to OXT users.

Not everything needs to be for everyone.

Think groupware over publicware.

I like this idea, could be a bit remenecant of like AOL HC Forums or Hotine /IP BBSes. Even more cool if it had like a GopherVR style interface for browsing piles of stacks, ha!
https://en.wikipedia.org/wiki/GopherVR#/

Gophervr.png (68.08 KiB) Viewed 8462 times

To some degree browsing all of the Pantechnicon stacks via a card on HyperCard Simulator's Home Stack feels a bit like an 'xTalk-net', or AOL HC forum's files section at least.

Pantechnicon.jpg (150.84 KiB) Viewed 8460 times

[OpenXTalkPaul]

I want to post about this so I don't forget (About to do the graduations party weekend here)
One of the biggest problems I needed to figure out in order to do an xTalk IDE 'in the cloud' was that the 'Save" command doesn't work to save a newly created stacks in the Web Stack 'sandbox' environment with its inherent limitations.

The Emscripten Engine attaches a virtual file system similar to mobile standalone deployments, but it can also use 'temporary storage', which I believe gets stored in the browsers cache.

So the solution is to include a template stack file that can be read with open file; read from file pattern, then write it back out to the temporary file storage where it CAN be opened, modified, saved back to. When you want to ACTUALLY save your new stack to disk it can read that file from temp storage byte for byte into a variable and base64Encode it (if needed) in order to pass it to JS/HTML5 file managing API(s) that can save it to your downloads folder or whatever.

[OpenXTalkPaul]

If you don't want a server involved in saving data, and you just need a way to persistently store some of stack user's data in a way that can be read back in later, that sort of data can be saved to the web-standlones temporary virtual filesystem, but it disappears that when you leave the page and come back later.

This should be doable without adding security concerns of a server. The problem here is the way stacks work inside the web sandbox. What is needed is a browser 'cookie' or similar that could be used store what's in the temporary virtual-files-system data across sessions. Otherwise the only way AFAICT, is to ask web-stack user to download their data and then 'upload' it to the stack again when they want to use that data with the stack again.

In other words.... in a OXT WebStack the specialFolderPath("Temporary") gets you the path to the web-stack's temporary virtual-filesystem, which is a writeable directory (tested with OXT Playground webStack) but it is not persistent across sessions. The best thing would be for this temp-virtual-filesystem to be like 'browser cookie' data that's associated with a particular webstack's URL.

[FourthWorld]

Browser local storage from HTML 5 forward offers at least 5MB of persistent space:
https://www.w3schools.com/html/html5_webstorage.asp

[OpenXTalkPaul]

Browser local storage from HTML 5 forward offers at least 5MB of persistent space:
https://www.w3schools.com/html/html5_webstorage.asp

Yes, the more persistent storage the better, that may come in handy as well, but Emscripten does its own thing.
I'm currently investigating the way things work with Emscriptens virtual File System API, which are created with/by our Emscripten engine deployment.

https://emscripten.org/docs/porting/fil ... rview.html
https://emscripten.org/docs/api_referen ... m-API.html
https://emscripten.org/docs/porting/fil ... Usage.html
https://emscripten.org/docs/api_referen ... system-api

From the first few lines of the JS source it looks like Emscripten's APIs may behave differently depending if it's running on Node.js which is a good thing if you're running it in a web->desktop app wrapper like Electron or similar.
https://github.com/emscripten-core/emsc ... rary_fs.js

In the runtime folder for Emscripten engine there should also be a folder auxiliary_files which gets auto-included with any web-deployment, useful for including additional resources for ALL web stacks. Additionally, any Extensions modules used in the stack are packed into the virtual file system, which can also include their own 'resources' folder.

This morning I did some more elaborate testing of the theory I only briefly tested with the OXT Web Playground.
1) create a blank stack on disk, read the bytes of that stack back in and place the data into a custom property, so a stack embedded in our test stack. Create standalone "web stack" with that blank stack embedded as binary data.
2) now running in a browser, a Button writes embedded stack binary data to file @ specialFolderPath("temporary") & "/test.oxtstack", open the file as a stack, set defaultStack to our new temporary stack, set templateButton's script to close the stack, create a new Button "Close Stack", save our new stack with new button in our temporary storage area, close stack.
3) Reopen this stack from this temp file and test. Check that "Close Stack" button is functional and it is!
4) Just to be sure, another button opens the stack from temp storage, adds "Hello World" button, save, close, reopen. "Hello World" button still there!
5) I also made a very basic file/folder browser that can read data from files on the virtual filesystem directories. It encodes file data to base64 and then the 'put' command writes it to STDOUT, which is wired to the "console" div (built into my HTML file). Using this I can see that the temporarily saved file is growing in the amount of data it's storing. SUCESS! This means it should be easy to pass that data to an HTML5 API in order to save edited stacks to disk from our Emscripten Engine!

My next tests will be importing a stack from my SSD into the virtual file system, where it can be read from and written to from our Emscripten Engine in the same an xT script would do on a Desktop platform. The only difference would be in the I/O mechanisms for opening and saving stacks. I'm very encouraged but the results so far, I'm convinced that this could be built into to a highly portable 'OXT Cloud IDE'.

One unfortunate thing about Web deployed stacks is that by default the stacks looks a lot like they're running on OSes in Metacard 1990s (the emulated UIs that are still available with 'the lookAndFeel' syntax).

[FourthWorld]

Browser local storage from HTML 5 forward offers at least 5MB of persistent space:
https://www.w3schools.com/html/html5_webstorage.asp

Yes, the more persistent storage the better, that may come in handy as well, but Emscripten does its own thing.

...in so many ways.

Ten years in, I'm thinking there's a reason so few (any?) others are attempting to use Emscripten to deploy a rich-media GUI app dev tool in a browser.

Native web tech is quite good, and possibly simpler to translate to.

[OpenXTalkPaul]

Browser local storage from HTML 5 forward offers at least 5MB of persistent space:
https://www.w3schools.com/html/html5_webstorage.asp

Yes, the more persistent storage the better, that may come in handy as well, but Emscripten does its own thing.

...in so many ways.

Ten years in, I'm thinking there's a reason so few (any?) others are attempting to use Emscripten to deploy a rich-media GUI app dev tool in a browser.

Native web tech is quite good, and possibly simpler to translate to.

I didn't even realize that Emscripten was that old but I guess it is (from Wikipedia: "In December 2014, the Internet Archive launched a DOSBox emulator compiled in Emscripten to provide browser-based access to thousands of archived MS-DOS and PC programs.")
Emscripten is a "complete Open Source compiler toolchain" to WebAssembly (and to 'asm.js' before WASM happened).
WASM has certainly been widely used in a lot of different (and pretty darn cool) ways including a .wasm port of media power-house app FFMPEG.
https://ffmpegwasm.netlify.app
Not to mention for entire virtual machines of classic OS(s) running inside browsers!
https://infinitemac.org/1994/NeXTSTEP%203.3
There's even a port of open-source Quake2 to WASM: https://quake2.m-h.org.uk
OprnFrameworks is really cool too https://openframeworks.cc/examples/
I beleive it can only get better.

As far as data storage goes, I'll try to use whatever API is 'least expensive" or most compatible for the task at hand.
I'm also not against incorporating any OTHER 'native' web tech (WebAssembly IS 'native' web tech now). The OXT "WebPlayground" is already using the Web Speech API to do TTS (since revSpeak doesn't work) and also using web/CSS API for changing the cursor (since it seems 'set cursor' doesn't work either).

[OpenXTalkPaul]

Emscripten was indeed around back in 2013 (compiling to asm.js https://en.wikipedia.org/wiki/Asm.js because WebAssembly 1.0 wasn't introduced until 2017), From Unreal Wikipedia Page:
"In 2013, Epic teamed-up with Mozilla to bring Unreal Engine 3 to the web; using the asm.js sublanguage and Emscripten compiler, they were able to port the engine in four days."

It should probably also be mention that .wasm bitcode modules can be used off line outside of a browser.
And also that nowadays it seems that Emscripten is not the only game in town to compile to .wasm
http://schellcode.github.io/webassembly ... emscripten

I've become a fan of WASM and the whole idea of compiling to platform agnostic byte code.

One more link (saving for myself later)...
This site lists a bunch of stuff that was made portable to web via Emscripten and/or WASM:
https://madewithwebassembly.com

[FourthWorld]

(WebAssembly IS 'native' web tech now).

Always has been. It's in the name.

But xTalk engines aren't.

They're all about events and objects, the two things WASM doesn't do.

But even C++ to JS seems overkill for xTalks, an engine running inside of an engine next to another engine just to run some scripts on objects.

Browsers natively have scriptable objects.

Direct translation, xTalk to JS, would likely take less than ten years.

Moving xTalk layouts to HTML/CSS doesn't take long at all.

[OpenXTalkPaul]

Direct translation, xTalk to JS, would likely take less than ten years.

Moving xTalk layouts to HTML/CSS doesn't take long at all.

I agree that an xTalk engine for running on the web really doesn't need to have any UI kit of its own. We really only need an xTalk interpreter that speaks Document Object Model. There is a bidirectional bridge in the Emscripten engine so you can use whatever JS APIs you might want, manipulate DOM objects, etc. but yeah it is riding a-top of another engine (the JS engine), and it certainly has it's failings (could be more complete, faster, smaller, less JS more .wasm etc.), but I do want to know all I can about what the OXT emscripten engine can and can't do anyway since it's has been left to the community as-is, and it is the only version of OXT Engine that should run on ANY platform that has a modern web engine.

I'm not at all against the idea of working on some sort of JS<>xTalk transpiler project as well as the OXT IDE, sign me up!
I really don't think it would take as long as 10 years either, but if it did, well I got ten years left in me (hopefully).

In fact, I think it could be built on some existing project that's already geared that way, like HyperCard Sim.
I was able to easily add to that 'engine' some of the missing syntax from our OXT dialect. I even started making sort-of a mini-IDE for creating a 'html standalones' which packs HCS's JS 'engine' + SimScript along with the 'stack'/page.

Have you ever checked out ///_HyperScript + HTMX ? That's another xTalk project that I find very interesting.
I think that is basically the thing you're talking about.

A python <> xTalk transpiler would be great too.
I would absolutely love it if xTalk could rise to the level of the Python ecosystem.

[OpenXTalkPaul]

Here's HyperWordle built with _HyperScript:
https://arhamjain.com/hyperwordle/

[FourthWorld]

Have you ever checked out ///_HyperScript + HTMX ?

Briefly. The problem is I don't mind JavaScript, and I haven't been able to figure out a business case for an xTalk-to-web solution, so I just keep on doing what I'm doing and watch others do what they do.

[OpenXTalkPaul]

Have you ever checked out ///_HyperScript + HTMX ?

Briefly. The problem is I don't mind JavaScript, and I haven't been able to figure out a business case for an xTalk-to-web solution, so I just keep on doing what I'm doing and watch others do what they do.

Figure out a business case, I guess that's always been the problem hasn't it?
xTalk is a "solution in search of a problem" (-- IIRC, Bill Atkison said that back in the late 80s).

Personally I do mind JS (and CSS). I've never liked it. I'd rather use Lua or Python (both of which have multiple ports to WASM), than JS. But the reality is that despite being thrown together in five days, a quarter of a century later JS is ubiquitous and not just on the web anymore, so I've wound up having to use it a bit, like or not. But there's never been a programming language I've enjoyed using like I do with xTalk script (not even the somewhat related AppleScript, which I used a lot)

In general, I guess use cases would be the same as any other language that's been written for or ported to web (and there seems to be a helluva lot of them lately)

The use cases I'm interested in (besides an xTalk IDE that runs on web-browser engines), are games, multimedia, and sound & music applications in particular, mostly traditionally "client-side" offline/desktop app sorts of things, but thanks to WASM we're starting to really be able to do desktop-style apps in a browser at acceptable speeds.

There are several stacks, virtual bass guitar, step-writing MIDI composing things. I'd really like to have online. runable from inside web browsers virually anywhere. I don't really care if I use our OXT Emscripten Engine to do it and if I have to load some additional JS libraries to do it (I will), then so be it.

I was also thinking I should deploy Meme-maker as web stack that uses my "GIFArray" library for doing things with animated GIFs. That's not exactly a business use-case. lol.

But that's the thing, I'm into this for fun, learning, keeping my brain active. It's also a productive way to distract myself from doom-scrolling the news feed.

[tperry2x]

It's also a productive way to distract myself from doom-scrolling the news feed.

Yeah, Sorry about that. I'll stick to trying to externalise functions and working on other things.

[OpenXTalkPaul]

It's also a productive way to distract myself from doom-scrolling the news feed.

Yeah, Sorry about that. I'll stick to trying to externalise functions and working on other things.

Huh? You've got nothing to be sorry about, quite the opposite, your participltion has been inspiring!

...
Back to the topic:
Unfortunately it seems I spoke too soon about being able to save stacks.

The 'save stack as' command DOES work in Emscripten as long as you save to writeable /tmp/ directory, however it only really saves the changes 'in-memory" (violtile memory), but does NOT save the bytes to the virtual disk's /tmp/ folder, when running in browser. The stack file does show up in the directory's listing but when I read it back byte for byte it is the stack is same as before I made changes. The changes only live on in the copy of the stack that is still in memory, but not in the copy on the virtual disk. If I make a brand-new stack (not from 'template' stack data), I can indeed 'save as' to /tmp/ and it shows in a files listing, but when I try to read it back as a byte stream 'with read from file for binary update', I get zero bytes.

What does work as expected is the open file/write to file/close file pattern, however I have no way to get the bytes of the copy of the stack with the changes I made that exists only in memory in order to pass it to 'write to file'. This is extremely frustrating. It really makes me wish we had that system of 'script-only' for GUI stacks, or an XML-based UI mark-up that I've talked about in the past, because it's no problem at all to import/export plain old text.

It does appears that Emscripten may be set-up to use Node.js File APIs (which can do direct access to a real file-system) when it's running in that environment, but I haven't tested that yet. I'm much more interested in running in a web browser (since we have desktop IDEs already).

I need to do more tests using URL syntax methods, but I think that will yield similar results.

The Emscripten JS that our OXT Engine is encapsulated inside of needs to be updated to newer Emscripten, and it may also need to be modified (use other browser-based virtual file system?) in order to fix this issue, otherwise there's not really any way to do a proper Web IDE with this engine because such a thing would be limited to working on 'script-only' stacks, no binary stack format.