xTalk EVERYWHERE! Lets Make An "XTML"(?) Interchange STACK Format Standard!

[OpenXTalkPaul]

By the way, I did find a doc about the 1.0 spec for HIFF format, circa 1989 and based on HC 1.2...
I posted it in another thread.

[FourthWorld]

I understand you have a lot real-world experience with converting from one xTalk environment to another, it would be great if you could post any relevant information, like how to get a complete list of every function, command, property name, expression names, keywords, reserved words, etc. via script (I already have a script that does most, but not all of it).

I've already shared the most valuable relevant information I have.

On this specific detail of language elements, I don't believe there's an engine-based way to obtain all of those in LC. And with a converter one would need to do the same for every target platform, most of which don't even provide as much engine-based support for that as LC.

And if they did, to what end? Object model differences will require a scope of change difficult to appreciate until you have the callouses on your fingertips from having dealt with it.

Conversion from one file format to another is something I too have done a significant amount of throughout my career (as well as in the course of my hobbies ). In fact I'll currently be dealing with that task at the day job for the foreseeable future due to a major change by an essential proprietary software package. But that's good right? At a minimum it keeps people employed, creates jobs, careers even, as evidenced by us!

Conversion can extend the life cycle of some software.

Automated conversion creates expectations that cannot be met.

Attempts at automated conversion often yield the appearance of a seductively promising first pass, but still require extensive work often on par with fully-manual conversion. And that's before we deal with the technical debt of being bound to ways of doing things no longer relevant in the target platform, and often not even the modern business context in terms of either current workflow needs or user experience expectations.

If you've found substantial employment using an automated converter, it's not that automated is it?

[OpenXTalkPaul]

I understand you have a lot real-world experience with converting from one xTalk environment to another, it would be great if you could post any relevant information, like how to get a complete list of every function, command, property name, expression names, keywords, reserved words, etc. via script (I already have a script that does most, but not all of it).

I've already shared the most valuable relevant information I have.

OK, thanks, I've read everything on your site (and a lot of others) I could find, and examined some of your excellent stacks. In particular that properties palette thing (the one that doesn't support widget properties, I mentioned that to you on the LC forum a while back)

On this specific detail of language elements, I don't believe there's an engine-based way to obtain all of those in LC.

And if they did, to what end? Object model differences will require a scope of change difficult to appreciate until you have the callouses on your fingertips from having dealt with it.

Probably not, but I was able to use the IDE to generate a list of a LOT of it, and if you go through the IDE there's some files in there that can be used to get more of the keywords.

To what end? Please re-read some of my comments in these forums about this matter. I've already listed a few 'Ends' and I've been imagining quite a few more, possibly very useful, 'Ends' daily. This ISN'T a "Solution in search of a problem", it's an idea being formulated as a result of trying to solve several existing problems.

Is the object model is really all that extensive once you take out all of the keywords and alias/synonyms that are in there for compatibility (many of those don't do anything, at least not anymore, a Stack's HCStackAddressing for example). It's certainly no where near as extensive as system API objects that they often mirror.

And with a converter one would need to do the same for every target platform, most of which don't even provide as much engine-based support for that as LC.

Doesn't apply, I'm NOT looking to make a 1:1 LC Stack to whatever converter.

Look at the document for HIFF spec that I posted, the approach for the importer side is same as with other markup specs like HTML are parse. The 'importer' can ignore tags that it doesn't recognize. I'll give you a real world example: if the importer (such as LCC < v7.0) doesn't have a Card image property, that card property could can be ignored or it could be imported as an image and then placed as a group with an image object and a backgroundBehavior set instead.

There aren't too many xTalk environments that are still around to convert to or from, and most have a much smaller scripting vocabularly to making 1:1 syntax AND object conversion virtually impossible to achieve, I wouldn't spend my time on that level of detail (at least not currently).

Conversion can extend the life cycle of some software.

Automated conversion creates expectations that cannot be met.

Attempts at automated conversion often yield the appearance of a seductively promising first pass, but still require extensive work often on par with fully-manual conversion. And that's before we deal with the technical debt of being bound to ways of doing things no longer relevant in the target platform, and often not even the modern business context in terms of either current workflow needs or user experience expectations.

If you've found substantial employment using an automated converter, it's not that automated is it?

I'm not sure what you're getting at here... I didn't say that I've found substantial employment using an automated converter, but I have used automated format converters substantially in the course of my employment, and those converters have often gotten me 95% to 100% of the way to what I needed. The other 0-5% mostly just required a human brain.

I've automated 'conversions' like this pseudo code:
* Parse HTML doc from job ticket web server, extracting layout properties (size, web pinfeeds, stepping, impositions, etc.) NOTE that ONLY the layout specs (not interested in press settings, shipping info or whatever else) are parsed.
* Then script the building of that layout in page layout app
* insert rule-up guide lineBoxes on a new layer, as printable objects, for indicating layout on proofs
* Add text objects with job specific information (job tracking#, colors, etc.) position vertical (rotate 90) along the layout sides.
* Print layout to proof or copy print file to RIP server drop folder for further processing.
At one point in my career a large portion of the work 'I' was doing. It really wasn't very much information to parse because I was only interested in a subset of the properties of the object (the object here being 'the Job' with properties parsed from HTML). Thanks to xTalk chunking expressions, it didn't take all that many lines of script to automate 90% of that task. I basically just plugged in the job# and watched the scripts run. It was built with HyperCard + AppleScript + QuarkXpress and ran fine under MacOS X (via classic up to 10.4.11). When it came time to dump Quark XPress, converting those same scripts to build documents with InDesign instead, the scripts often required only minor tweaks because the objects of a page layout application are inherently going to be very similar to other apps with the same goal. A Page is a Page... an imageBox is a pictureBox, or picture XObject, or an <IMAGE>, or a NSImageView... the names are different but they're all going to have some sort of ImageData property associated with them.

Hell, I'd say that more than anything Q2ID ( Quark to InDesign) plugin converter is the #1 reason that Adobe was able to go from 5% of the page layout app market to 99% in under a decade, meanwhile Quark went from 95 to 1%< of the marketshare at the same time. There's also things like LibreOffice, which can open Word DOCs 99.9% line-break accurately as long as you have the correct fonts on the same platform.

[OpenXTalkPaul]

If anyone is interested the current syntax compilation, properties, reserved words, etc. or as many as I could collect, is here:

https://github.com/PaulMcClernan/lc-hun ... .dic#L1662

1645 syntax keywords (I'm sure this isn't all of it), HOWEVER there some duplicates, variations with camelCase and without, some are obsolete (QuickTime for example), some were only ever there for compatibility WITH OTHER XTALKs (they aren't actually function)!

Interestingly, I've read some info that was collecting dust in the corners of the internet from the original MetaCard dev in which he states that MetaTalk was based on SuperCard SuperTalk syntax, which was obviously based on HyperCard/HyperTalk. It seems there is already some informal level of compatibility that's been volantarily baked into most/all xTalk languages over the years.

[FourthWorld]

...that properties palette thing (the one that doesn't support widget properties, I mentioned that to you on the LC forum a while back)

Yeah, the reason hasn't changed: widget design needs to become consistent with the property array syntax for other control types.

When widgets become first-class citizens in LC they'll automatically become first-class citizens in my tool and all others that use standard property syntax.

Look at the document for HIFF spec that I posted...

Read it when it came out. Tried to use it. A lot of us did. Ken Ray and I used to talk about it late into the night before we made our own. I don't know anyone who's successfully used HIFF in a real-world conversion of a commercial work between different xTalks.

The 'importer' can ignore tags that it doesn't recognize.

Think about that. Take your time...

I'll give you a real world example: if the importer (such as LCC < v7.0) doesn't have a Card image property, that card property could can be ignored or it could be imported as an image and then placed as a group with an image object and a backgroundBehavior set instead.

Easier to just open the HC stack in LC where that particular change happens automatically.

Now do SuperCard's backsize property, or window-specific tool modes, or ListMaster...

There aren't too many xTalk environments that are still around to convert to or from...

Exactly.

I've automated 'conversions' like this pseudo code:
* Parse HTML doc...

There are many formats for many things in the world. Many are worth converting.

In this discussion of xTalk, I was only talking about xTalk.

[OpenXTalkPaul]

...that properties palette thing (the one that doesn't support widget properties, I mentioned that to you on the LC forum a while back)

Yeah, the reason hasn't changed: widget design needs to become consistent with the property array syntax for other control types.

When widgets become first-class citizens in LC they'll automatically become first-class citizens in my tool and all others that use standard property syntax.

I'm not sure exactly what you mean by "consistent with the property array syntax"? There's an array of key/value properties, and certain properties, like those of the "classic" controls, can be "shadowed" for consistency for basic things (foregroundColor, backgroundColor, etc.) but the whole point is that it's possible to make any sort of control with any sort of properties of any sort of data-type (because you can add editors for new types) that an author can think of.

Now I have seen mention that they were planning to allow widgets to be built with LCS in addition to LCB, but it's not at all clear what that means. I imagine this would work the way DataGrid or using a menu Stack already works, basically a "Widget" as a subStack rendered to as an instance in a viewport displayed within another stack.

I've been looking at this quite a bit lately because I'm currently rebuilding the Tools palette including the Widgets section (which can be modified on the fly) using a modified version of the "IconPicker" widget. You can definitely get a array of properties of all the widgets that are loaded, but any Widget can have any number of its own unique properties, if the widget author makes it so.

[quote}

Look at the document for HIFF spec that I posted...

Read it when it came out. Tried to use it. A lot of us did. Ken Ray and I used to talk about it late into the night before we made our own. I don't know anyone who's successfully used HIFF in a real-world conversion of a commercial work between different xTalks.
[/quote]
I assumed that since there was a comercial product called ConvertIt! which lead to the HIFF spec being created, that someone did actually used it for something (commercial or not). I know I personally had zero interest in running xTalk on Windows 1.x PC or Unix in 1989 (I was busy most wasting my parents money on Art School at that time).

The 'importer' can ignore tags that it doesn't recognize.

Think about that. Take your time...

? I have thought about it, and the people who designed XML did too since that is the mechanism that allows an ML to be eXtensible. If I'm only interested in SVG data from SVGIcon Widgets on the this card of this stack, why would I care about any other properties or any other object type?

I'll give you a real world example: if the importer (such as LCC < v7.0) doesn't have a Card image property, that card property could can be ignored or it could be imported as an image and then placed as a group with an image object and a backgroundBehavior set instead.

Easier to just open the HC stack in LC where that particular change happens automatically.
[/quote]
Uhg... I should've used my <sarcasm> tag, I knew that is exactly what LC did with it in its conversion process, that was what I meant to point out. The process would be similar to that for any other importing function.

I pointed out pre-v7 or less because it's still unclear to me wether or not LC Ltd. recent bug fix fixed the importing of HyperCard stacks OR did they just make it so that it doesn't crash the engine when you try to.

Now do SuperCard's backsize property,

Not sure what you mean here.
I just looked that keyword up and LCC / OXT engine already does have SuperCard's backSize property.
https://livecode.fandom.com/de/wiki/BackSize
So I imagine that a markup for that would be as a property of a group control tag, something like <group backSize="640,480" ...

or window-specific tool modes,

Not sure what this is, is it a SuperCard window property? Although I do still have a valid license for SuperCard (v4.73), I've barely ever used SC and I've always thought SC's IDE was sort of weird and not at all HyperCard-ish enough. And since I'm on mainly on now and SC is 32bit and uses Carbon still I'm not likely to suddenly start to.
But surely there is Window-type specific tool modes already, you can't use a brush tool on a stack when it's in a window in palette mode or in a modal dialog mode. I'd image you could script a "window-specific tool mode".

or ListMaster...

Not familiar with this either, it sounds like the name of a Fredric Rinaldi Xternal, but I think that was called 'Listoid' (or something like it)

There aren't too many xTalk environments that are still around to convert to or from...

Exactly.

... which is why I'm really NOT looking to make some sort of stack 'converter' right now.
Although, I don't think it would take all that long to get a basic functioning converter to get a basic stack converted into some other xTalk environment like StackSmith, WebCard, or WyldCard, all three of which use JSON as their 'stack' file format.
Obviously until there is a time when there is more available options for xTalk engines, there isn't much reason for having some sort of intermediary format if that is your only use case.

I've automated 'conversions' like this pseudo code:
* Parse HTML doc...

There are many formats for many things in the world. Many are worth converting.

In this discussion of xTalk, I was only talking about xTalk.

Again, in my thinking this would NOT be used for converting from one xTalk IDE to another, at least not exclusively.
I think that's the main issue here, because I'm NOT only talking about xTalk<->xTalk conversion... even with the conversion use-case it could be xTalk Objects to HTML, QML, or maybe even IDML (Adobe InDesign)... but the point is I wasn't even thinking of xTalk<->xTalk conversion as a main use-case for this. The original inspiration was to use as a 'template' markup (for making the ResourceCenter reference stack easily editable and extensible for example) and for use in package manager.

But in fact while working at rebuilding the Tools palette I was thinking it would be so great if there was already such a markup already settled on. I could then feed this markup of the existing Tools palette layout to a markup property of a custom "button grid" widget (perhaps not entirely unlike the IconPicker widget that is already part of the IDE... modification of which is in progress here: https://github.com/OpenXTalk-org/OpenXTalk-PaletteGrid) and have it more OR LESS match the existing tools palette but is rendered with Shapes/SVG paths inside a widget. Replacing buttons imageData with SVG paths and instantly have a resolution independent version. The widget would only need to parse the 'button' tags in it's Markup property. Widget could ignore card and group properties entirely in it's parsing as it is only interested in the RunEdit and ClassicControls groups buttons in this case (the Tool's palette "Widgets" group is already uses SVG-Icon widgets by default but I want those to be in a separate tab or even a separate palette altogether.).
Using markup tags in widget's properties was already sort of implemented by Herman Hoch in his 'iGrid' sag Icon Grid widget, he used a << tag >> as a placeholder for an images path directory which is later swapped out by the widget's code.

[OpenXTalkPaul]

I'm currently looking into this again, after again reading through what is known about the HyperCard File Format
thanks to Becky: http://www.kreativekorp.com/swdownload/ ... 202010.TXT and also here:
https://hypercard.org/hypercard_file_format/ )
There's gaps in the knowledge about some of the less-common specifics of the file format, but it's a rather like a 'RIFF' format (https://www.loc.gov/preservation/digita ... 0025.shtml) with blocks of data labeled with 4-Character codes (commonly used in the classic macOS), but not enough is unknown to prevent most HC stack files from being parsed. I think that I could write an OXT script that reads AND writes them, rebuilds/converts them into other formats.

I think Dan's Hypercard Simulator and similar, that use html-based format that reflects the HC Stack format is brilliant. Most xTalk/xCard have, at least to some degree, been modeled on HC, or were been built with some level of compatibility with HC, so being based on HC (2.4) provides an excellent baseline, and shooting for a Web browser compatible format just makes sense to me.

So I'm going to look again at the stacks I've made with this idea in mind and see if I can make OXT import an HTML-based 'stack' from Dan's HCSIm. OXT can do far more than HyperCard 2.4 could ever do, but this will be a good starting point.
Dan's HCSim engine, or ViperCard, or WebCard, etc. web could then be expanded on (with HTML5 APIs, WASM modules, etc.) in order to gain as much feature parity with OXT as possible, and of course the OXT Emscipten Engine would (theoretically) already have feature parity (it's not 100%).

Dan's format is nice and simple, and mirrors HC using custom web elements tags such as:
<stack-part>
<background-part>
<card-part>
<button-part>
<field-part>
That list is actually mostly all of them (minus any properties or anything)

New or OXT specific 'part's ('part' is a synonym for OXT's 'control' keyword) could be added, like a <player-part> for example would be implemented the same way as the Video card in HyperSim's 'Home' stack, which is simply a <field-part> with an HTML5 video player element inserted into it. I imagine similarly there could be a <pict-part> could allow an SVG element insert into it.

I want to be able to import and export these generic-ish web 'stacks' via our OXT Emscripten engine.
I do want us to have a functional Web browser based xTalk IDE too!

[OpenXTalkPaul]

I think I have a good grasp now on the bulk of the stack 'File Format' (*the file format is actually 'HTML' with custom elements) that Dan Gelder used for his HyperCard Simulator 'engine'. It is indeed pretty much HyperCard's stack format translated to HTML with a little bit of JSON and Base64 encoding for stuff that would've been in the 'Resource Fork' on macOS classic.

Here's a break down:

Code: Select all

<html>

<stack-part 
width=512
height=342
tool=Browse
visualEffect=''
importedICONs='' 
importedICONResources=''
ImportedWAVs=''
importedPLTEs=''
importedPICTs=''
importedCURSs=''
importedColorData=''

WOBA='' -- A 'WRATH OF BILL ATKINSON' formatted bitmap

buttonCSSFont=''
fieldCSSFont=''
>

<background-part 
ID=0 
name='' 
color=''
dontSearch=false
cantDelete=false
showPict=true
bitmap=''
addColorData=''
>
Background Parts:
	<button-part>
	<field-part>

<card-part 
ID=0 
name='' 
color=''
dontSearch=false
cantDelete=false
showPict=true
bitmap=''
addColorData=''
 marked=false
 bkgndID=0
> 
-- Theoretically Additional 'backgrounds' (aka 'groups' in the case of MC/RR/LC/OXT stacks) could follow.

Card Parts:
	<button-part>
	<field-part>

other parts?


<title-bar-menu 
name=''
show=false
contents=''
markChar=''
menuMessage|menuMsg=''
>

<modal-dialog
name=''
visible=true
x=20
y=20
>

<card-window-delegate
scroll=''
rectangle|rect=''
width=''
height=''
>

<color-bevel-picker 
hasColor=true
color='#FFFFFF'
bevel=2
showBevel=true
>

<marching-ants inset=0 visible=true ></marching-ants>
<packaged-answer-and-ask-boxes></packaged-answer-and-ask-boxes>
<window-part>?

<player-part> -- Could be aded to support MC/RR/LC/OXT stacks (possibly other xTalk/Cards too?)

Embedded Imported Movies list -- could be added for the cases of QuickTime or MC/RR/LC/OXT Movies embedded in stacks (possibly other xTalk/Cards too?)

-- Additional stacks (sub-stacks) could theoretically follow (in the case of MC/RR/LC/OXT stacks)

Everything has a 'script='' Attribute


<field-part
script=''
id=#
visible=true
topLeft
width
height
textHeight
textAlign=''
textFont=''
textSize=''
textStyle='plain'
color=''
bevel=0
type = transparent | opaque | rectangle | shadow | scrolling
scroll=0
lockText=false
dontWrap=false 
autoSelect=false
sharedText=false
multipleLines=false
wideMargins=true
fixedLineHeight=false
showLines=false
>


<button-part
script=''
id=#
visible=true
topLeft
width
height
textHeight
textAlign=''
textFont=''
textSize=''
textStyle='plain'
color=''
bevel=0
type= transparent | opaque | standard | default | checkbox | radiobutton | popup | rectangle | roundrect | shadow | oval | picture
hilite | highlight | highlite | hilight = false
autoHilite=true
sharedHilite=true
family=''
showName=true
enabled=true
icon=0 
selectedLine=0
>

I left out the closing tags, but it's HTML so <stack-part ... > has to be ended with </stack-part>
Resources are in JSON wrapped list (ImportedWAVs,importedPLTEs,importedPICTs') and they're base 64 encoded (sounds, pictures, palettes)

'PLTE' are Palettes. HyperCard did not have 'sub-stacks' and palettes were NOT stacks. The 'Palette' command (it was actually an eXternal XCMD that came with HC) in HyperCard instead would make a 'windoid' with buttons based on a 'PLTE' resources and associated graphics resources. Each button could have its own send-message or script to execute on button press.

There may be a few things missing like for example cards, backgounds, etc. can have a 'WOBA=' property, which is a 1-bit bitmap picture format invented by Bill Atkinson ( WOBA stands for 'Wrath Of Bill Atkinson'), usually a background or card image.
There's also <title-bar-menu > (like a MENU resources) and I there was something about Dialogs (DLOG, ALRT) resources in there too. Ah ResEdit was such an awesome tinker tool!
https://eclecticlight.co/2024/07/20/man ... n-resedit/
https://developer.apple.com/library/arc ... erence.pdf
Another cool technology that Apple abandoned along the way, like HyperCard, QuickTime, QuartzComposer and others.