Could there be a parallel effort on the Community Edition...

by **OpenXTalkPaul** » Mon Oct 11, 2021 8:54 pm

Hmmm, looks like that docs building script is outdated or something, the API.sqlite file it made was smaller by more then a megabyte, and entries for a bunch of things like most of LCB were missing. So I'll have to take a closer look at that script, in the meantime I tried to come at it a different way. Editing the existing API.sqlite file...

It seems that that db file isn't actually what the Dictionary uses? It seems it actually uses JSON .js files mainly, but still makes use of a db file too? I used an SQL editor to delete some commercial-only entries from the API.sqlite file, and then cleared the Documentation cache folder. When I relaunched the IDE, it rebuilt the dictionary cache and those deleted entries reappeared!?!?! I took the API.sqlite out completely, cleared the cache, relaunched and it rebuilt the cache again, still with the deleted entries, but with a smaller API.sqlite file in the cache folder that it must have built from the JSON files?

Fortunately my eyes aren't bleeding, at least not yet, but it certainly is a confusing setup.

by **OpenXTalkPaul** » Mon Oct 11, 2021 1:25 pm

Got it, the docs builder scripts are run by a stack called builder.rev in the same folder as docs_builder.livecodescript
had to comment out a couple of things (ensureTitlecase, which was only relevant for certain two word editions such as coumminutyPlus) to get the script to run all the way through. Anyway It built the api.sqlite file.

by **axwald** » Mon Oct 11, 2021 7:47 am

Hi,

OpenXTalkPaul wrote: ↑Sun Oct 10, 2021 3:47 pm [...] I think that this script is called by something [...]

Is it just me who's eyes start bleeding looking at this code?

Have fun?

by **OpenXTalkPaul** » Sun Oct 10, 2021 3:47 pm

OK so I just did a bit more searching around, I think that this script is called by something and used to build the DB file that ships with the IDE:
https://github.com/livecode/livecode/bl ... codescript

by **OpenXTalkPaul** » Sun Oct 10, 2021 3:27 pm

susan wrote: ↑Sat Oct 09, 2021 11:00 pm
susan wrote: ↑Sat Oct 09, 2021 9:35 pm
OpenXTalkPaul wrote: ↑Sat Oct 09, 2021 6:20 pm If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary packed into a SQLite API db file that is what is actually included with the IDE
Am I correct in understanding that there is a sqlite file somewhere, and you are hoping to retrieve the entries in that file that are from individual .lcdoc files?

Is this because the .lcdocs are not part of the github, and so need to be extracted to be able to work with them?
Oh, okay. I re-read your question. I had it back-to-front didn't I!

You want to figure out how to turn the individual .lcdoc files into the sqlite file, right?

Coming to the problem side ways, can the database entries not be packed/encrypted, and instead the OpenXTalk API viewer be altered to take the plain text .lcdoc format, so not need a decoding step?

Or am I still misunderstanding?

You got it the second time, I was looking to convert the loose .lcdocs TO a SQL db.
I believe there are already existing routines in the IDE or maybe in the engine repo, for doing this, I just haven't had time to work out exactly where or how.

I get what you're saying about not using a DB file at all, but I think that there's two issues with that...

For one that could possibly probably be a lot of work, but maybe not if we went back to the previous version of the dictionary, prior to the invention of this .lcdoc markdown, but then the entries would have to be gone through and made up to date. The IDE DOES actually already have the ability to insert new entries from lcdocs or inline documentation (in an .lcb file for example), which happens when you install a new Extension.

And secondly, having all of those loose .lcdoc markdown files would increase the size of the IDE on disk because each would take a block of disk space, which these days means usually at the least 32Kbytes, even though some dictionary entries are more like 32bytes! Personally I think the IDE is already larger, disk space wise, then it needs to be.

axwald wrote: ↑Sun Oct 10, 2021 8:00 am Did you have a look at "TinyDictionary" from Bernd Niggemann et al.? It may contain a solution.
I guess it's hidden somewhere in the "sample stacks".

Have fun!

Yes, Bernd is awesome, he does some great work and he helped me out a bit when I was working on my Piano Widget!
I have a copy of TinyDictionary, and I will be looking at that when I get to it, but I think that probably pulls data FROM the the API DB, but what I want is to build the API DB itself from the lcdoc markdown files.
I'm sure there already exists an automated method for doing this, perhaps using handlers in the revDocsParser IDE file, I just haven't had a chance to look into that yet.

by **axwald** » Sun Oct 10, 2021 8:00 am

Hi,

OpenXTalkPaul wrote: ↑Sat Oct 09, 2021 6:20 pm If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary [...]

Did you have a look at "TinyDictionary" from Bernd Niggemann et al.? It may contain a solution.
I guess it's hidden somewhere in the "sample stacks".

Have fun!

by **susan** » Sat Oct 09, 2021 11:00 pm

susan wrote: ↑Sat Oct 09, 2021 9:35 pm
OpenXTalkPaul wrote: ↑Sat Oct 09, 2021 6:20 pm If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary packed into a SQLite API db file that is what is actually included with the IDE
Am I correct in understanding that there is a sqlite file somewhere, and you are hoping to retrieve the entries in that file that are from individual .lcdoc files?

Is this because the .lcdocs are not part of the github, and so need to be extracted to be able to work with them?

Oh, okay. I re-read your question. I had it back-to-front didn't I!

You want to figure out how to turn the individual .lcdoc files into the sqlite file, right?

Coming to the problem side ways, can the database entries not be packed/encrypted, and instead the OpenXTalk API viewer be altered to take the plain text .lcdoc format, so not need a decoding step?

Or am I still misunderstanding?

by **susan** » Sat Oct 09, 2021 9:35 pm

OpenXTalkPaul wrote: ↑Sat Oct 09, 2021 6:20 pm If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary packed into a SQLite API db file that is what is actually included with the IDE

Am I correct in understanding that there is a sqlite file somewhere, and you are hoping to retrieve the entries in that file that are from individual .lcdoc files?

Is this because the .lcdocs are not part of the github, and so need to be extracted to be able to work with them?

by **OpenXTalkPaul** » Sat Oct 09, 2021 6:20 pm

If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary packed into a SQLite API db file that is what is actually included with the IDE (and then subsequently extracted and cached in a users directory), that would be helpful.

Seth already did a bunch of editing the docs files to remove brand references, but they're still in the raw lcdoc format.
Additionally, I think we could remove the dictionary entries for commercial only things such as tsNet.

by **OpenXTalkPaul** » Sat Oct 09, 2021 6:02 pm

susan wrote: ↑Sat Oct 09, 2021 11:54 am
richmond62 wrote: ↑Sat Oct 09, 2021 8:57 am ALL the versions of LiveCode Community "work".
Yes, but what I was meaning was could I get the building from the source to work? (I can see that was not clear.)

I understand, that there is no need to rebuild the binary engine, to remove all branding and make a fork.

All I am interested in doing is spending an afternoon, to see if I can build v6.x it from the source as an interesting exercise.

I won't waste time on it, and of course most people here will want a 64 bit version (especially the users on macs that can no longer run 32 bit) with all the extras of 9.x

But on Windows, around the world not everyone has modern machines, and the way it has continued to run such win32 apps is quite useful.

I'm not yet entirely sure that there is no need to rebuild the binary engine to remove all branding. Mainly because they integrated the installer into the main binary in later versions.

Personally, I don't have a problem with co-opting.

I do think it would be cool to back-port v9 to 32bit. Personally I'd even be interested in back porting to PowerPC 32/64 bit. Just that not right now, as Re/De-Branding has to come first.

I'm also still looking at the startup chain a lot. A lot of interesting things there, like a few calls to what seem to be non-Script, internal engine commands/functions, such as revInternal__SetAppIcon.

Particularly interesting is revInternal__StackFiles(), which returns a list of stack files used by the IDE. What is interesting (pain in the ass) about it is that if you try to use that outside of the beginning of the startup process, like from the message box or even later in the start up chain, it completely disables EVERYTHING in the IDE!
I think this is related to this engine function in the C++ file called exec-interface-stack.cpp:

Code: Select all

 MCStack::GetStackFiles(MCExecContext& ctxt, MCStringRef& r_files)

particularly the MCExecContext part, ctxt param needs to be something particular, not yet sure what exactly or how it checks for the 'starting up' execution context.

I was able to work around that revInternal__StackFiles() in the Home script in order to make it so you can now put a stack called "Start Center.oxtstack" in your "My OpenXTalk" folder and if that is present it will load that as your "Start Center" stack instead of the one in inside the IDE directory (which mine is currently a blank stack because it was loaded with branding and links ). Now if you want you want a start up stack that looks exactly like HyperCard, you can do that, or have one that looks like NASA Mission Control if you'd prefer! And this user "Start Center" is opened as a regular editable stack (instead of as a palette)! That IMO is the best part of the Open in OpenXTalk!

by **susan** » Sat Oct 09, 2021 11:54 am

richmond62 wrote: ↑Sat Oct 09, 2021 8:57 am ALL the versions of LiveCode Community "work".

Yes, but what I was meaning was could I get the building from the source to work? (I can see that was not clear.)

I understand, that there is no need to rebuild the binary engine, to remove all branding and make a fork.

All I am interested in doing is spending an afternoon, to see if I can build v6.x it from the source as an interesting exercise.

I won't waste time on it, and of course most people here will want a 64 bit version (especially the users on macs that can no longer run 32 bit) with all the extras of 9.x

But on Windows, around the world not everyone has modern machines, and the way it has continued to run such win32 apps is quite useful.

by **richmond62** » Sat Oct 09, 2021 8:57 am

ALL the versions of LiveCode Community "work".

At present that is not the problem (after all I use both 9.6.3 and 8.1.10 on a daily basis right now).

What IS the problem is how to remove all the references to LiveCode that need to be removed while
keeping things running so that there can be on-going development.

I, for one, have almost NO intention on doing anything beyond taking the originl source code of an LC
version and attempting to remove those references, quite independently of anyone else's effort, mainly
because I am 100% useless at working in a team.

IFF I manage the above, I will post it somewhere so it is available for anyone who wants it,
and who may or may not choose to "muck about with it" further.

by **susan** » Sat Oct 09, 2021 3:47 am

OpenXTalkPaul wrote: ↑Sat Oct 09, 2021 12:24 am it's just don't know what could be added or fixed to those existing older versions 6/7 that would be worth putting in the effort to try to do?

For me any effort is motivated solely by finding out is it possible, and will it work? That is the reward.

I promise not to co-opt anyone from the main goal here!

by **OpenXTalkPaul** » Sat Oct 09, 2021 12:24 am

susan wrote: ↑Fri Oct 08, 2021 5:26 am
OpenXTalkPaul wrote: ↑Fri Oct 08, 2021 1:45 am One thing I just found out that's a deal breaker for me for LCC 6.x base is that it's a 32bit only build!
I have the opposite problem. I own a couple of 32 bit Windows laptops that are aging, but still work fine. One of them runs Windows 10 under 32 bits quite effectively.

So when software moves to a 64 bit only build, I am sad.

Shotcut went 64 bit only which was a shame, and there have been some steam games and software that I wanted to run on those machines, but couldn't because they no longer support Win32.

LCC 8.x on macOS was 63 & 32 bit (but default is 32bit mode), 32 bit only on Windows until 9.x. I think Linux had separate builds for 32bit & 6bit. It looks as though like they removed some externals like revVideoGrabber in 8.x, maybe because they were 32bit only? LCC 8.x does have LCB middleware so that (AFAIR, LCB in 8 have Foreign Function Interface?) might be useful to restore some functionality like that, and a lot more!

I completely understand/respect the desire to not want to toss perfectly good older hardware into a landfill, it's just don't know what could be added or fixed to those existing older versions 6/7 that would be worth putting in the effort to try to do? Other than maybe removing some now-pointless commercial version related things and perhaps trimming the size down a bit? With LCB in 9 there's plenty that could be added, even without ever recompiling the core engine!

by **susan** » Fri Oct 08, 2021 5:26 am

OpenXTalkPaul wrote: ↑Fri Oct 08, 2021 1:45 am One thing I just found out that's a deal breaker for me for LCC 6.x base is that it's a 32bit only build!

I have the opposite problem. I own a couple of 32 bit Windows laptops that are aging, but still work fine. One of them runs Windows 10 under 32 bits quite effectively.

So when software moves to a 64 bit only build, I am sad.

Shotcut went 64 bit only which was a shame, and there have been some steam games and software that I wanted to run on those machines, but couldn't because they no longer support Win32.

by **OpenXTalkPaul** » Fri Oct 08, 2021 1:45 am

susan wrote: ↑Thu Oct 07, 2021 10:37 pm
OpenXTalkPaul wrote: ↑Thu Oct 07, 2021 9:06 pm If you're on macOS I can tell you that it's not going to work out of the box
Thanks for the time saving heads-up. I am primarily a Windows user (am I the only one here? ) so I am going to try that build. I suspect that my experience will be the same as on the mac, but I will still give it a try and let you know what happens.

My biggest fear is getting a cryptic C++ compilation error. And if that happens, it's game over for me. I also have no experience with sophisticated build tools, but if it is a .bat file that would be awesome.

Yeah, I don't have the 6.7.11 executable, so that might be good to get!

Seth is a Windows and Linux User so you're not alone. I'm mostly macOS user (since 1987), but I've done a lot of playing around with various Linux builds too. I currently have both macOS and Windows 10 on my laptop, and I'm about to try for a triple boot setup, adding a Linux.

One thing I just found out that's a deal breaker for me for LCC 6.x base is that it's a 32bit only build! I had thought that it was 32bit and 64 bit, that was wrong. Which means you can't run 6.x on the last two major macOS versions (10.15 or 11).
Though we might still be able to use older pieces of the 6.x IDE since that's all built with scripts/stacks.

by **susan** » Thu Oct 07, 2021 10:37 pm

OpenXTalkPaul wrote: ↑Thu Oct 07, 2021 9:06 pm If you're on macOS I can tell you that it's not going to work out of the box

Thanks for the time saving heads-up. I am primarily a Windows user (am I the only one here?

) so I am going to try that build. I suspect that my experience will be the same as on the mac, but I will still give it a try and let you know what happens.

My biggest fear is getting a cryptic C++ compilation error. And if that happens, it's game over for me. I also have no experience with sophisticated build tools, but if it is a .bat file that would be awesome.

Yeah, I don't have the 6.7.11 executable, so that might be good to get!

by **OpenXTalkPaul** » Thu Oct 07, 2021 9:06 pm

susan wrote: ↑Thu Oct 07, 2021 7:15 am
SethMorrow wrote: ↑Sat Oct 02, 2021 7:06 pm I certainly wouldn't discourage anyone from working on a 6.x fork.
Agree. Not needing unicode, 6.x most of the time I am attracted by the "solidness" of that IDE version and the general zippiness.

Of course, if unicode is essential for someone then that path is a dead-end.

I might download version 6 from the github and see how difficult it is to compile. If it doesn't work "out of the box" I expect to immediately give up as I don't have the skill (or time!) to figure out such things. But it would be interesting to try.

Happy to share my experiences, if there is anything to report.

If you're on macOS I can tell you that it's not going to work out of the box, unless you're still running macOS 10.9 with the XCode version from back then. There is a script around that can symbolic link newer versions of Xcode to older versions of the macOS SDK, and then it should compile. I haven't gotten back to trying to it compile yet, I needed to clear a bunch of space on my laptop first.

I've been thinking of mixing in parts of the older versions of the IDE, like the preferences UI or StartCenter stacks for example, and also I want to take a look to see why is it that HyperCard Stacks can't be imported in v7 and higer versions. I know there's not much use for that these days, but it something that attracted me to LC when I first saw it about 7 years ago. Another thing I've noticed, looking at the installers all in one place, is how much larger, file size wise, that LC has grown since v.6, the Mac version 9.6.3 installer is literally 10 times the file size that v6.0.1 was! From all the digging into the IDE I've been doing I can tell you there's lots of old unused files / images that are still in there, I'm guessing most of that is for backwards compatibility reasons?

If you just want to download an old community pre-compiled versions they're all up on Archive.org
https://archive.org/search.php?query=cr ... 2C+Inc.%22
As well as still available here:
https://community.livecode.com/6_7_11/

by **susan** » Thu Oct 07, 2021 7:15 am

SethMorrow wrote: ↑Sat Oct 02, 2021 7:06 pm I certainly wouldn't discourage anyone from working on a 6.x fork.

Agree. Not needing unicode, 6.x most of the time I am attracted by the "solidness" of that IDE version and the general zippiness.

Of course, if unicode is essential for someone then that path is a dead-end.

I might download version 6 from the github and see how difficult it is to compile. If it doesn't work "out of the box" I expect to immediately give up as I don't have the skill (or time!) to figure out such things. But it would be interesting to try.

Happy to share my experiences, if there is anything to report.

by **OpenXTalkPaul** » Mon Oct 04, 2021 8:48 pm

axwald wrote: ↑Sat Oct 02, 2021 1:24 pm "serious coder" here is the equivalent to the mailing list's "cheese"? Didn't know, apologize!

I don't know what that means, but OK? I've mostly use the use list via This Week In LiveCode and via search. I've never been a fan of mailing lists, I like/need ability to correct/edit my posts too much. My defensiveness in response to "serious coder" is mostly based on attitudes of certain pro-programmer's that I've personally know, and the fact that there weren't really any C++ pros donating code to Community Edition in the past so perhaps not too many of that C++ pro type of contributor has not been attracted to the concepts of xTalks... although it looks like a few independent projects such as StackSmith have gotten fairly far on their own, so maybe there's some other reason they didn't want to be involved with LCC. I'm willing to at least try to learn more C++ and whatever else it takes to keep this thing alive.

axwald wrote: ↑Sat Oct 02, 2021 1:24 pm
For me a "serious coder" is someone that does its coding with love & considerable time & labour, and has fun doing it. Someone who, for instance, spends most of the time coding by choice - no matter if payed for or not.

In contrast to someone that only codes for money, and wouldn't touch an IDE in private.

In contrast to someone that occasionally does some coding, but has many other, more important, things occupying his time.
Too be honest, I felt a bit startled by your reactions. It really seems as if I triggered a reflex.

It did, people have reflexes, no biggie. To be honest I've gotten a little irritated on occasion because, while I do appreciate the diversity of ideas, we've gotten advice on what we should do or shouldn't do but not much help doing any of it.

Your bullet points are conflicting. In the first one I'm a "serious coder" in the last one I'm not, because I have a full time job (in the graphic arts) that doesn't have much to do with coding, and I also have a family that takes priority over everything else... I also have other hobbies (music composition for example) but they've become intertwined with coding.

axwald wrote: ↑Sat Oct 02, 2021 1:24 pm This explained, I understand that we have quite different opinions & goals, and that I don't really suit in here. So my best wishes for your project, may it (and you!) live long & prosper!

I'm agree with the big tent thing. The more goals and more opinions the better... I just think if you're looking for a commercial quality product, commercial level support, and timely updates that may be an unrealistic expectation.

axwald wrote: ↑Sat Oct 02, 2021 1:24 pm Paul: Obviously you have just skimmed the links I provided. Additional hints:
The libURL problem was observed in Win, too - but only randomly!
The "Windows not being good at expanding it's buffers/memory allocation" is noticed as solved in LC 9.6.4
Both Win & Android show unpredictable fonts & sizes, different from any other apps on these devices.
SQLite on newer Linux versions doesn't work - code-created dbs fail with "invalid database type" ...

I did only skim through those, because I'm trying to do 3-4 different things in a short amount of time.
libURL is a Script library if I'm not mistaken? Not sure... but I'm pretty sure I could find other URL libraries on every platform. In my Mac LCB projects I've use NSURL (NS as in NeXTstep/Apple Cocoa/GNUStep), and I've rolled my own in xTalk at times (going back to my Tarantula HTML editor project way back in the 90s). I could wrap a substitute URL library, It would probably be best to keep the amount of foreign libraries down to a minimum if possible, particularly with something like URLs are just strings.

If the buffer/allocate problem is an engine problem that's solved in 9.6.4, then it should also be in 9.6.3 because my understanding is that they basically admitted that there wasn't really any changes between those two versions beyond the new licensing schemes.

"Win & Android show unpredictable fonts & sizes, different from any other apps on these devices."
This is a mysterious problem, obviously font files don't change (unless they have, which sometimes Font vendors do)... so it must be changes in the rendering engine. I can't imagine why it would be unpredictable? Are you saying fonts look different from one launch to the next with no modifications to the stack?

SQLLite on Linux probably needs a newer build of that library as a driver for the revDB external.
Personally I've become very interested in Linux over the past few years, and will be installing some Debian distro to test/build OXT on.

Could there be a parallel effort on the Community Edition...

Post a reply

Expand view Topic review: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...

Re: Could there be a parallel effort on the Community Edition...