Linux Flavours that DO!

[richmond62]

I was venting about a personal conflict where I shouldn't have.

Join the club.

[richmond62]

But Stallman I think is correct on a lot of what he's said as far as Free (as in Freedom) Open Source Software (and I generally agree with his even more political opinions on his personal blog). However, I can also respect the sedated-socialist benevolent dictatorship of Linus too, that's the pragmatic attitude to take that doesn't scare business people away.

Having seen and met Richard Stallman (2005 People's Palace of Culture, Sofia, Bulgaria): I can honestly say that WHAT he says does, on the whole, make sense: but the WAY he says it is guaranteed to put a lot of people's backs up.

I am most definitely NOT a socialist: in fact I'd describe myself as fairly right-wing. BUT I also believe in giving as many people as many tools for free as possible: where I diverge from the socialist point of view is that I am NOT prepared to bail out people who are NOT prepared to make an effort to use those tools.

Over the last 3 years I have had a number of Ukrainian refugee children attend my language school for nothing: I had NO qualms about kicking one of them out, when, after 6 months they had done NO Homework, never brought their notebook, or their textbooks to class, and had NOT contributed to anything in the classroom.

Mind you: I do believe that any state that would have us believe it is vaguely civilised should provide some semi-decent level of both education and healthcare that is free at the point of delivery (and if that makes me a socialist: so be it).

Linus Thorvalds is interesting; but most definitely a positive influence.

[OpenXTalkPaul]

I gathered that you are a 'conservative' (in the European sense of the word). Personally I think societies need a good balance between socialism and capitalism. But I know some knuckle-dragging people here in 'Murica that nowadays would call you a straight up 'Commie' just for those last bits about civilized nations, or even just if you've been vaccinated against COVID! I wish I was joking about that. I suspect that's part of why Stallman has the attitude he has.

But getting away from politics as much as possible....

I can't find the thread where i posted the stack, but I've added a handler to GeneralLinuxTools.oxtstack that removes ANSI ESC color codes from command-line output that was colorized using them, leaving only the plain text.

Put this in a button and it should return the distro's info text from your Linux Distro (which is sometimes colorized, as is the case with MX Linux)

Code: Select all

on mouseDown pButtonNumber
   -- get shell("cat /etc/issue")
   put "/etc/issue" into tFile
   open file tFile for read
   read from file tFile until eof
   close file tFile 
   put it into tDistroInfo
   -- get the plainText of tDistroInfo:
   Answer stripANSIEscapeColorCodes(tDistroInfo)
end mouseDown

function stripANSIEscapeColorCodes theData
   put 0 into iterator
   repeat for (the number of bytes in theData)
      add 1 to iterator
      if byteToNum( byte iterator of theData) =27 then 
         delete byte iterator to iterator+5 of  theData
         next repeat
      end if
      if byte iterator+1 of theData is empty then exit repeat
      put byte iterator of theData after tNewText
   end repeat
   return tNewText
   ----  NOTE: \x1b. ANSI escapes always start with \x1b , or \e , or \033 . 
   -- These are all the same thing: they're just various ways of inserting the byte 27 into a string. 
   -- If you look at an ASCII table, 0x1b is literally called ESC , and this is basically why.
   --      Here are the most commonly used terminal color codes:
   --      Black: \u001b[30m.
   --            Red: \u001b[31m.
   --            Green: \u001b[32m.
   --            Yellow: \u001b[33m.
   --            Blue: \u001b[34m.
   --            Magenta: \u001b[35m.
   --            Cyan: \u001b[36m.
   --            White: \u001b[37m.
end stripANSIEscapeColorCodes

[OpenXTalkPaul]

I guess that handler is leaving behind some null bytes if the text is UTF-16 encoded because that script I posted above only starts the removal from the relevant byte's offset (the first byte it finds with the dec. value 27, which is the 'ESC' character ) not taking into account multi-byte 'characters' in anyway. So it should really be modified to then check if preceding byte = 0 (the 'NUL' character), and perhaps two bytes before that for 32bit 'characters', and then if found remove those from the output too. We don't want to leave a lot of unused invisible nulls lying around.
No warranty given, but that script should not do any harm to any unicode characters that may be included in the text, it should (theoretically) only remove those ANSI ESC codes used for colorizing text in a terminal.

[richmond62]

I've been vaccinated against COVID 4 times: and as that makes me a right pinko fellow-traveller . . .

Round these parts (Bulgaria) the knuckle-dragging fraternity are trying to get the country to climb into bed with the Russians.

At least, with any luck, sooner or later, COVID, Monkey-pox, Trumpy-pox, or whatever will kill off a lot of the anti-vaxxers.

[OpenXTalkPaul]

How about 'Engine Versions that do"?

It seems to me that in version LC CE v8.1.10 64bit, the Browser Widget actually works rather well, on the same exact machine with same window manager where version v9.6.3 has strange issues with phantom windows appearing and keyboard focus problems. In v8.1.10 I have both the the Dictionary stack open and a stack with a browser widget pointed at this site, and I was able to switch between them with the engine properly refocusing the keyboard input to various input elements.

Clearly LC CE v8 64bit engine with Browser Widget works much better on Linux than the newer version 9.x with included newer versions of Chromium Embedded Framework. Something definitely changed for the worse at some point between v8 and v9. So next thing I want to see if v8 Browser Widget and CEF from v8 can be transplanted into v9 IDE to benefit from the same quality of functionality.

I wouldn't recommend using either for something like online banking, but as a way to use JavaScript libraries or use web APIs to play with SVG or CSS animations, or to view the Syntax dictionary, V8 Browser Widget seems to works OK. It's handy to have an HTML renderer you can drop down on a card and then intermix xTalk with JavaScript.

[tperry2x]

How about 'Engine Versions that do"?

Certainly, LCC 7 (64-bit) on Linux is an absolute trooper. Before unicode though which I know is a deal breaker for some. So so so fast though.

[OpenXTalkPaul]

How about 'Engine Versions that do"?

Certainly, LCC 7 (64-bit) on Linux is an absolute trooper. Before unicode though which I know is a deal breaker for some. So so so fast though.

As I understood there was some Unicode support in v7 and prior to that even (there's deprecated syntax in the dictionary left-over from pre- full unicode support). I do know that there was a Unicode induced text processing performance hit in v7 that got a bit better in v8 and v9. My biggest issues with v7 on Linux is there seems to be no working embedded browser support when running on 64bit kernels, but wasn't it supported on 32bit? I vaguely recall that HH did some experiments on RPi that used JS ( that's ARM 32bit CPU).

V7 does load up the IDE extremely quickly as you've said, but there's a bunch of features I've become used to that are missing, like Widgets and SVG support.

But anyway I'm talking about v8 Engine/Browser Widget here, and the fact that it behaves much better than it's v9 equivalent in OXT.

[tperry2x]

Not that I really want to, but are you wanting to downgrade your OXT engine to v8 in that case?

The thing with doing that is you may (will) need an entirely different set of prebuilts and extra dependencies to get the engine to compile. (which we again, don't have).

The other issue is version 8 of the engine handles quite a lot of things differently. As experienced in my Retro edition of OXT Lite, if you try and take the "revTools" or "application browser" from v9, they just don't work because of syntax differences. You kind of need to reinvent bits and find tricks to get around certain limitations. [2], [3], [4], [progress 1], [progress 2]

If you are testing "OXT Lite Retro", to avoid any 'lumpy custard' moments, you might want to delete your prefs file from v9 that gets created if you are using LCC9 or OXT on the same machine. There's also stuff that gets chucked into the "Application Support" folder in MacOS too if anyone is really bothered.

Not trying to teach you what you already know, this is also for the benefit of anyone reading this of course.

if anyone is really bothered

I'll just eat my own words, as I just noticed this!:

but-then.png (170.21 KiB) Viewed 1950 times

I wonder if it would be a matter of playing swaps: taking all the working bits of the browser script out of version 8 and dropping that into the source for the version 9 engine. Of course, there would probably be a lot of remedial (tidyup) work to do afterwards, but I wonder if the end result would be a working browser? - not that I'd rely on it to do anything other than fetch a directory listing on a remote server - which, come to think of it, I could do via curl / wget anyway.

I do wonder if that amount of effort would be worthwhile. I'm normally not one to shy away from that, but it IS a lot of effort.

As I mentioned back then:

Because there wasn't a lot of people asking me about it, I've not spent any more time on it [OXT Lite "Retro"]. It seems that the demand for a 'retro' mac version is quite low. Everyone else seems to be on Apple's upgrade train.

I got off that particular train, not long after it set off. I'm now on a replacement bus service

[OpenXTalkPaul]

I wonder if it would be a matter of playing swaps: taking all the working bits of the browser script out of version 8 and dropping that into the source for the version 9 engine. Of course, there would probably be a lot of remedial (tidyup) work to do afterwards, but I wonder if the end result would be a working browser? - not that I'd rely on it to do anything other than fetch a directory listing on a remote server - which, come to think of it, I could do via curl / wget anyway.

That's more what I had in mind, pull in the Browser Widget, CEF, et all from v8, using it to replace the corresponding bits in v9 with it.

I went about trying to do the transplant this morning. At a minimum I knew the Widget module would need to be recompiled because LC changed the module format between v8 and v9, but the XB language did not change much so it should be forward compatible and compile OK with Ext.Builder in v9. I've updated a few Widgets that were written with Builder early on, mostly minor changes were needed in those situations.

The problem is that the Browser Widget is NOT like other widgets. It's a 'Native Layer' widget, but also it uses FFI (v8 included the beginnings of FFI) to bind to the 'Browser Factory' internal Engine routines which are what actually create the Browser instances. Which could still work if those internal engine functions hadn't changed between v8 and v9 but they did, at least the initializer's function's symbol (function ID / name ) did.

So here's an interesting thing. The Browser Widget use (a keyword equivalent to a C++ #include) a separate Extension Builder library module which contains the FFI binding strings with handlers for using the Browser Factory routines. The .lcb source is included in the Engine repo in either /engine/src/ or /libbrowser/src/ . OK great, so WHERE is the compiled .lcm in the IDE??? It's not there! Neither is the other language modules like the modules for things like using Java or Objective C objects. Additionally that folder contains C some C++ source code / header files which also appear to be more like 'bindings', exposing the bits of the engine to Builder with code appropriate for a given platform (we're interested in the Linux files which may include .lnx in the names). So that's more pieces that make up the whole. I suspect Build scripts are using lc-compile to build the library module to lcm bytcode and then it merges that with the main Engine binary as part of the 'linker' pat of build process, just like the way a prebuilt C++ .a library gets embedded into the main executable. So it seems that parts of the Engine ARE indeed being 'bootstrapped' (built in itself) to a small degree (and it's debatable wether .lcb counts as an 'xTalk', it's close enough for me).

The bottom line is it's all tied to these other parts, so it's not a straight-forward replacement like changing other Widget/Libraries usually is. I'm actually think it a may be easier to build a separate Browser Widget (or a Browser Window) to interact with a web engine. It would use OUR namespace (like org.openxtalk.widget.browser) to make it a separate kind from the legacy LC Browser Widget. Perhaps we do something completely different, like make binding strings to use a cross platform library such as FCP (FreePascalCompiler), or Lua communities libraries for embedding Browser / CEF instances inside apps. The process wild be similar to binding to the existing internal routines (should be able to use FCP libraries just as if they're C libraries). This would be more of a 'packaged' thing, the idea being that if the web engine (CEF currently) gets an update, as they often do, ideally we would just need to drop in the new compiled binaries downloaded from Spotify's servers (or whereever) and that should just work, and if it doesn't the binding strings could be updated to accommodate changes in the new web framework version without recompiling the core xTalk Engine(s)!

I got off that particular train, not long after it set off. I'm now on a replacement bus service

Yeah I've been trying to tolerate that bus service, but I feel like I'm no where near as productive using any Linux distro, and I find myself constantly tweaking or changing the system instead of coding or anything else productive. Of course lately a lot of that has more to do with testing OXT with various Linux components. However I've been using macOS for around 40 years so there's things I've come to expect, for decades, that simply are not built-in and if they are they aren't cohesive, like visual inconsistencies in theming across apps that are using different UI toolkits for example (which I know is just cosmetic but a lot of people, me included, like consistency), just off the top of my head. But rather than stay in Apple's walled garden or 'sandbox', I'd much rather find a nice starting point Linux to fiddle with, really work it until it's my own (Mac-like) tailored 'daily driver' OS.

[tperry2x]

That makes perfect sense, I had a feeling that it was being merged / folded back into the main application binary - rather than existing as a separate file, which - as you rightly say, means a recompile.

I know we are going back over old ground here, but I would absolutely agree that building an entirely new widget is probably the better way to go in this instance - you aren't having to deal with the inconsistencies that way, and you can start afresh - meaning that it *should* work the same across all platforms.

So, I then wondered - can we use NeoSurf (based on Netsurf, but with privacy and security improvements). You'll see this is already included as a standalone (on linux version of OXT Lite) in /opt/openxtalk/openxtalk_x86_64/Externals/neo/webEngine.bin
I modified the appimage binary so it looks in //tmp/unavailable.html -- you can redirect that with my oxtstack I made, but this was a temporary solution.

If the NeoSurf engine (which is really small at 6MB) can be integrated into a new widget, that would be excellent.

Edit: Actually, just pondering something else - does the CEF engine also get folded back into a standalone in the same way if the original browser object is used? - No, it doesn't. (answered my own question)...

cef-gumph.png (30.9 KiB) Viewed 1809 times

However, rather than generate a separate CEF folder with 13 items inside, which has to live in the same place as the standalone - and an additional CEF-related executable (which seems very messy)...

If it's possible to instead bundle all this up into a single binary so your new widget is embedded inside it, then I'm all for that approach too.

[OpenXTalkPaul]

Yes I have looked at NeoSurf (and related projects). It is nice and small and from what I can tell it does support modern things like HTML5, and wow the parent project Netsurf even has browser builds for ATARI systems! However, looking at it's repo it does seem to rely on a few dependencies (which is probably unavoidable), but more troubling to me is concerns about the compatibility of it's web-engine: https://www.reddit.com/r/linuxquestions ... inux_with/
Also is that available as a library or would it have to be IAC / socket communications with the browser as a separate executable?

Here's what I'm looking for in the way of a Browser widget / CEF replacement:
1) The web-engine it uses should have support for HTML5 APIs, most of them anyway, because I recognize that is a lot of APIs to support.
2) It has to have some mechanism for bidirectional communication between xTalk script and a browser instance's JavaScript core. Otherwise what would even be the point? With out that you would not be able to make use of the HTML5 APIs or do much of anything beyond rendering a web page. Along with this comes Browser instance being able to send messages back to xTalk script engine, so that we can know things like when the web engine has finished downloading a page and rendering it or when it has some errors.
3) I would prefer that as a whole it would be smaller than the whopping 150+ megabytes of space that the current Chromium Embedded framework consumes.
4) I would prefer that it is super-easy to update without having to recompile the xTalk engine. Builder Extension package format is set up well for doing exactly that. For example 'updated' the mac version of the HIDAPI extension (wrapper) to one that was macOS 12+ compatible simply by dropping in the compiled HIDAPI library binary (obtained via 'Homebrew') into the Extension's 'code' for macOS, and it worked without a hitch.

I would prefer that the 'WebView' remains 'in-line' within the stack/card rendering, like the browser widget currently does, but that's definitely NOT a deal breaker requirement for me. Particularly since 'Native Layer' Widgets have issues with a lot of widow managers on Linuxes. It could be a library extension instead of a widget, where the library creates it's own 'Native' window to render web content into, presumably avoiding any window compositing / focusing problems.

I've been looking for a cross-platform C++ wrapper library that would create a GTK Webkit view on Linux and Windows ('Edge' and 'Blink' engine are WebKit). On macOS the Browser widget already does not use Chromium Embedded (since v8 it uses Native/Safari Webkit on mac). Webkit is now the preferred Engine used by GNOME. I'd like to switch to Webkit on all platforms, and not just because LC said they were going to do that (which I think they still haven't done). Webkit is developed with support from Apple and other large corporations with magnitudes more resources than small-community projects like NetSurf/NeoSurf, and so compatibility and prompt security updates would be better.

[tperry2x]

On macOS the Browser widget already does not use Chromium Embedded (since v8 it uses Native/Safari Webkit on mac). Webkit is now the preferred Engine used by GNOME. I'd like to switch to Webkit on all platforms, and not just because LC said they were going to do that (which I think they still haven't done). Webkit is developed with support from Apple and other large corporations with magnitudes more resources than small-community projects like NetSurf/NeoSurf, and so compatibility and prompt security updates would be better.

That is very true, Webkit seems to be the way all things are going - however that also makes it the main target for any attacks of course. Having said that, how worried am I about it? I mean, anything is going to be more secure than what we have at the moment.

I'd love to have some way of incorporating it as a widget to replace the existing "browser" completely. What I'm keen for it NOT to have is a folder full of loads and loads of required files when you build a standalone. Only from a user experience point of view, seems really messy for want of a better word. But that's on my wish list. The best thing initially would be a working browser instance in the IDE based on webkit, which is no small ask.

[OpenXTalkPaul]

I'd love to have some way of incorporating it as a widget to replace the existing "browser" completely. What I'm keen for it NOT to have is a folder full of loads and loads of required files when you build a standalone. Only from a user experience point of view, seems really messy for want of a better word. But that's on my wish list. The best thing initially would be a working browser instance in the IDE based on webkit, which is no small ask.

prefer that as a whole it would be smaller than the whopping 150+ megabytes of space that the current Chromium Embedded framework consumes.

Yes, the current Browser Widget is not only broken and outdated, but also quite lard and is made up of several executables and various libraries, it's basically including a whole web browser and some dependencies 'embedded' (but not really embedded) inside our IDE and any standalone built that use it. Of course there are some apps that are built around that very thing (that is what enabled Electron things 'cross-platform desktop apps with JavaScript' to come about). But it's no longer makes sense to include a full web browser engine, when there is likely going to be one found included with the operating system on just about any modern platform that people actually use. Additionally that engine would be getting security updates for as long as the OS still being supported by its maker. But the big thing that interests me is we wouldn't need to include an embedded browser. Just look-for and link-to one that's already on the system *like everything else this is a bit tricky on Linuxes because you can't be 100% certain of anything in Unix-like / Linux world..

I do like your idea to include a very small browser executable, but maybe as a more of a fall-back option, or maybe to use it with OpenXION if it could use that as GUI, or maybe just to preview .html or .md (markdown) files that it might have generated).

What I would not want to do is embed large chunks of a browser or more of it's dependencies (that's why we have libSkia in there) into the main core engine executable(s), making an even larger monolithic xT VM 'kernel', when maybe all you're building is a very simple UI for some shell scripts you may never use again. it doesn't make sense to have a > 30 mb + your stacks size, for an App binary such a use-case. I don't like lots of large parts or dependency files cluttering up the IDE either. But remember, we do want to separate the Engine from extraneous things like 'a control that is also a full blown web browser' certainly is. So with WKWebkit (Mac/iOS), WebKit2 (MS Edge), and WebKitGTK (GNOME), we should not need to include the engine. Linux that do not could install from their package manager and that should do. We would simply have have a hypothetically low-maintenance 'wrapper' and no worries about including an outdated insecure web engine.

I've look at it a bit in the GTK docs, I'm not so sure it would be 'no easy task' for the essential bits, the API seems to be designed to be rather 'high-level'. With WebkitGTK that's is all written in C, and built onto of infrastructure of GNOME, GTK, Glib, GObjects, etc., and not a lot of binding to do to use it, and there's hooks for evaluating JS (and using JavaScriptCore directly.) I'm more concerned about our Engine's 'Native Layer' capabilities on Linux.

https://webkitgtk.org/reference/webkitg ... archy.html
https://webkitgtk.org/reference/webkitg ... bView.html
https://webkitgtk.org/reference/webkit2 ... bView.html
https://webkitgtk.org/reference/webkitg ... wBase.html
https://webkitgtk.org/reference/webkitg ... w.new.html
https://webkitgtk.org/reference/webkitg ... ction.html
https://webkitgtk.org/reference/webkitg ... cript.html


That reminds me, Trevor Devore did some wrapping some of Apple WebKit stuff in LCB:
https://github.com/trevordevore/lc-maco ... webkit.lcb