Pull Requests

[mwieder]

In particular this is for submitting pull requests to the engine, but the process applies to any pull requests to github

[mwieder]

Submitting github pull requests
Commands here in a terminal show a $ prompt

#####################################
Prerequisites: Getting set up
#####################################

You have git installed locally
You have already logged into github
You have forked the main repository (https://github.com/mwieder/OpenXTalkEngine) to create your own copy
If not then
log into github
go to https://github.com/mwieder/OpenXTalkEngine
click on the Fork button
end if
Open a terminal window
In that window type

Code: Select all

	$ git clone --recurse-submodules https://github.com/<yourname>/OpenXTalkEngine.git

That will create your local copy of the repository
The submodules option is critical to get everything needed.
Type

Code: Select all

	$ git remote -v

That should show you

Code: Select all

	$ origin	git@github.com:<yourname>/OpenXTalkEngine.git (fetch)
	$ origin	git@github.com:<yourname>/OpenXTalkEngine.git (push)

If not then
type

Code: Select all

	$ git remote add origin git@github.com:<yourname>/OpenXTalkEngine.git

and try the git remote -v command again
end if

#####################################
Making something to submit
#####################################

Now you're ready to create a patch
type

Code: Select all

	$ git checkout develop
	$ git checkout -b <nameofnewpatch>

that will create a new branch named <nameofnewpatch> off your develop branch
and change your position to that branch
you can experiment with code in this branch
if it doesn't work just get rid of it:

Code: Select all

	$ git checkout develop
	$ git branch -d <nameofnewpatch>

#####################################
Pushing the patch to github
#####################################

But if you've got something ready to submit then
(still in the <nameofnewpatch> branch)

Code: Select all

	$ git add .
	$ git commit -m "brief description"
	$ git push

You should see something like

Enumerating objects: 50, done.
Counting objects: 100% (50/50), done.
Delta compression using up to 4 threads
Compressing objects: 100% (40/40), done.
Writing objects: 100% (41/41), 106.54 KiB | 1.78 MiB/s, done.
Total 41 (delta 7), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (7/7), completed with 1 local object.
To github.com:<yourname>/OpenXTalkEngine.git

That will push your branch to github. Now you're ready to submit it to the main repo.

#####################################
Creating the pull request
#####################################

On github
Go to https://github.com/<yourname>/OpenXTalkEngine
You should see something like
This branch is 1 commit ahead of mwieder/OpenXTalkEngine:develop
You should see a Contribute button with a downarrow

ContributeButton.png (7.13 KiB) Viewed 3594 times

Click that button
You should see a green button that says "Open pull request"
You should see green text that says "Able to merge"

ableToMerge.png (10.32 KiB) Viewed 3594 times

Add a title and a description. The description can be detailed or the same as the commit text.
You should see a button that says "Create pull request".

CreatePullRequest.png (7.93 KiB) Viewed 3594 times

Click it. That will submit the pull request for comments/approval.

[tperry2x]

Thanks for that guide Mark, much appreciated.
I guess this makes sense if you do it all the time.

I'm getting compiler errors using that repo (or a fork of it as mentioned).

errors.png (71.11 KiB) Viewed 3587 times

Seems to be header files jpeglib.h and cups.h causing the compiler to fail currently.

[OpenXTalkPaul]

Thanks Mark.

It would be good to have companion walk throughs for doing the same via GitDesktop (or any other app providing a GUI for Git / GitHub). I tend to use the GitDesktop, rather than command-line Git, but mostly just to sync my own changes to my own projects. Ocassinally I'll use the GitHub web interface too. And I'm not very good at using GitHub either, often keep experimental things offline and then never get around to merging into repo.

I also worry about syncing other people's changes because we may have conflicting ideas on how certain things should be (which is why there's now two OXT forks going), plus I want to make sure any changes don't conflict with other changes we've already made before merging in.

I'll try to look at those pull requests later tonight.

[mwieder]

Tom - what linux distro are you developing on? I don't recall having to do anything special with the jpeg dev libraries, so maybe they were already in my linux Mint 20. I do see that I have the libjpeg-dev library installed (version 8c-2ubuntu8).

The cups libraries are to fix linux printing. I think adding the cups-common library will do the trick.
Hmmm... if cups isn't installed by default in some distros may have to think of a secondary approach.

[mwieder]

Paul-

The only gui tool I use for git work is gitk, so someone else will have to pitch in with other tools. I know github has its own client software and there are other third-party tools (SourceTree?) but I don't have any experience with them. I've always been a commandline kind of guy.

I tend to make new branches locally often for experiments. If they don't work out I can just delete the branch. But if I get something that does fix a problem or create a new functionality then I can just merge the branch into the main branch and get on with other work.

[tperry2x]

Tom - what linux distro are you developing on? I don't recall having to do anything special with the jpeg dev libraries, so maybe they were already in my linux Mint 20. I do see that I have the libjpeg-dev library installed (version 8c-2ubuntu8).

The cups libraries are to fix linux printing. I think adding the cups-common library will do the trick.
Hmmm... if cups isn't installed by default in some distros may have to think of a secondary approach.

This is Xubuntu that was missing the libraries. (Nothing too exotic or niche).
As has been mentioned repeatedly on here before, and is always my gripe with anyone who develops software. It should not matter which variant / distro / window manager / desktop session anyone is using. Any software should either be capable of fetching all the libs it needs itself, or should have all required libraries embedded so that all dependencies are satisfied. A few extra MB is nothing these days, compared to not having the thing run.

[mwieder]

Or at least be able to tell you what's missing and what to install to fix the situation.
As I've mentioned elsewhere I'd love to get rid of gyp in favor of gn or whatever to have a more modern build environment. I can see why google dumped gyp back in 2016. At least my build environment is on a ubuntu base, so we should be able to narrow this down a bit.

[tperry2x]

Or at least be able to tell you what's missing and what to install to fix the situation.

Exactly. Many programs on Windows have their DLLs included in the same folder as the program for example. This is to avoid the situation where the user may have an older (or newer) version of the dll installed elsewhere in the system. They do this of course when they depend on a certain function in their DLL that might not exist in an old version - or have been cut from the newer dll. (Same thing with libs in Linux). Perhaps we could have the engine load all the libs it needs from a subdirectory, completely avoiding any 'dependency hell' on Linux - and probably solving the video / mplayer issues if we bundled a known-working good copy with it. Much like the appimage approach.

As I've mentioned elsewhere I'd love to get rid of gyp in favor of gn or whatever to have a more modern build environment. I can see why google dumped gyp back in 2016. At least my build environment is on a ubuntu base, so we should be able to narrow this down a bit.

That would be excellent, and would no doubt go a long way to getting a working compile of the engine (on all platforms?) to be a reality. What I would love to see is gn or whatever is used, prompt the user for a build target platform when compiling. Perhaps with a list of choices: 1. x86-x64, 2. x86-x32, 3. AArch-x64 etc...

Also some updated compilation instructions wouldn't go amiss on that github front page.
Something that pulls this all together, downloads everything it needs in one go. Not relying on third-party locations (I seem to remember seeing things being pulled in from other locations such as openSSL libs - that's great until one day it ceases to exist). At least fall back to a locally installed copy as a mirror if the third-party site isn't available or suddenly makes the file that the compile relies on unavailable.

In fact, as ultimately user-interaction at compile stage comes down to only a few choices, why not have something like this (completely avoiding the need to use visual studio at compile stage):

compiler-idea.png (3.7 KiB) Viewed 3506 times

[mwieder]

That last bit is just simple scripting to give arguments to the Makefile.
Do notice though that osx builds can only be built natively. Win builds apparently can be built on linux but only using wine.

Also some updated compilation instructions wouldn't go amiss on that github front page.

Just looking over my notes from a year ago. I find

Code: Select all

+// NOTE: cups requires sudo apt install libcups2-dev on the build machine

My prebuilt submodule will download recent sources for libCurl, libOpenSSL, libICU, and libCEF rather than trying to download precompiled libraries from a mysterious unreachable runrev server.

Do note that I'm building locally with python 2.7, not python3. Don't know if that makes a difference.

[tperry2x]

Just looking over my notes from a year ago. I find

Code: Select all

+// NOTE: cups requires sudo apt install libcups2-dev on the build machine

My prebuilt submodule will download recent sources for libCurl, libOpenSSL, libICU, and libCEF rather than trying to download precompiled libraries from a mysterious unreachable runrev server.
Do note that I'm building locally with python 2.7, not python3. Don't know if that makes a difference.

It might well do. Some historical functions aren't included in python3, so some stuff relies on python 2 - but I also know some bits use python3 as well - so there's a bit of a mixture.
Thanks for the pointers on cups. I also have jpeglib.h errors, - seems like both aren't included in the engine's repo.

On MacOS - are we still of the opinion that a MacOS 11 machine should be used? What is the best version of xCode to try it with?
In Windows, I can get it to compile, yet it runs incredibly slowly. There are lots of null pointer errors (same as what has been indicated as being an issue on MacOS with menus, and lots of advisory notes that certain functions are depreciated and have been replaced) -- seemingly the code in the engine is lagging behind the way of doing things in Visual Studio 2017 (???)

I'm sure which versions of both xCode and Visual Studio are used to compile the engine matter.
The fact that Linux uses a standard makefile is probably the only reason I've had success with it - because there's not another layer of interference by an IDE thrown into the mix. (But that's just speculation).

[mwieder]

hmmm...

jpeglib.h is in thirdparty/libjpeg/src. Do you know where your error is being generated? Sounds like possibly a path problem?
The only place I see jpeglib.h referenced outside of the jpeg thirdparty library is in generating libskia, where it's just

Code: Select all

#include "jpeglib.h"

So the libskia files are expecting that to be somewhere in your path. I find I have it in /usr/include/jpeglib.h and I see I have the libjpeg-dev package installed. So an apt install of that might solve the problem.


And yes, the build process has lots of warnings... I've been dealing with them piecemeal... mostly harmless, just annoying.

[tperry2x]

Just trying again on a fresh Install of XUbuntu 24
Using your github Repo this time to compile, Mark:

Code: Select all

git clone --recurse-submodules https://github.com/mwieder/OpenXTalkEngine.git

All going well so far, but then I ran into a problem.
libesd0-dev

Code: Select all

Unable to locate package libesd0-dev

https://askubuntu.com/questions/1082722 ... untu-18-04
So, I'm currently seeing if I can add Repos from Xenial, then will retry...

Edit: after switching my sources list over to xenial, I could download libesd0-dev, then revert the sources.list back to 'Noble Numbat'

I also had to turn on "noble noble-updates noble-backports" to get Python2 as it's now depreciated too.

Package python2 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Turns out I have to build it from source, so...

Download:
https://www.python.org/ftp/python/2.7.1 ... 2.7.18.tgz

Code: Select all

tar -xf '/home/user/Downloads/Python-2.7.18.tgz'
cd Python-2.7.18/
sudo apt install build-essential pkg-config \
      libbz2-dev libffi-dev libgdbm-dev liblzma-dev \
      libncurses5-dev libreadline6-dev libsqlite3-dev \
      libssl-dev tk-dev uuid-dev zlib1g-dev

then...

Code: Select all

./configure --enable-optimizations \
              --enable-shared \
              --enable-loadable-sqlite-extensions \
              --prefix /usr/local \
              LDFLAGS=-Wl,-rpath=/usr/local/lib

then...

Code: Select all

make

then (run Python)...

Code: Select all

python

Now set the environment variable from within Python:

Code: Select all

LD_LIBRARY_PATH=. ./python

So, now I have Python2.7 installed, I run into my next issue:

Code: Select all

make config-linux
travis_fold:start:config
CONFIGURE
./config.sh --platform linux-x86_64
/home/user/Desktop/OpenXTalkEngine/./config.py:278: SyntaxWarning: invalid escape sequence '\d'
  sdk, ver = re.match('^([^\d]*)(\d*)', target_sdk).groups()
ERROR: Java SDK not found; set $JAVA_SDK
make: *** [Makefile:101: config-linux-x86_64] Error 1

...so, now off to install and setup a Java SDK (another 312MB download)...

After that, we hit another hurdle (and come to a stop)

Code: Select all

make config-linux
travis_fold:start:config
CONFIGURE
./config.sh --platform linux-x86_64
/home/user/Desktop/OpenXTalkEngine/./config.py:278: SyntaxWarning: invalid escape sequence '\d'
  sdk, ver = re.match('^([^\d]*)(\d*)', target_sdk).groups()
Traceback (most recent call last):
  File "/home/user/Desktop/OpenXTalkEngine/./config.py", line 855, in <module>
    configure(sys.argv[1:])
  File "/home/user/Desktop/OpenXTalkEngine/./config.py", line 834, in configure
    configure_procs[opts['OS']](opts)
  File "/home/user/Desktop/OpenXTalkEngine/./config.py", line 745, in configure_linux
    validate_java_tools(opts)
  File "/home/user/Desktop/OpenXTalkEngine/./config.py", line 373, in validate_java_tools
    sdk = guess_java_home(opts['HOST_PLATFORM'])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Desktop/OpenXTalkEngine/./config.py", line 353, in guess_java_home
    javac_path.endswith(javac_str)):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
make: *** [Makefile:101: config-linux-x86_64] Error 1

[tperry2x]

...currently compiling...
So far, my changes are:
config.h
line 4:

Code: Select all

exec /usr/bin/python2.7 "${base}/config.py" "$@"

We now get a bit further along, but compilation is terminated due to multiple fatal errors.
Too many to post here, so I've attached a text file of the output.

errors.7z

(921 Bytes) Downloaded 88 times

Screenshot_2024-06-24_22-21-17.png (161.65 KiB) Viewed 3343 times

[mwieder]

Hopefully java won't get you into trouble. I use OpenJDK instead and installing it sets the CLASSPATH and the LD_LIBRARY_PATH.

I have the unicode headers in /usr/include/unicode. I don't remember doing anything special to install it.
It's not in my PATH statement, but I see that config.py references /usr/include.

Update edit:

From https://stackoverflow.com/questions/886 ... e-unistr-h this might work

Code: Select all

sudo apt-get install libunistring-dev

[tperry2x]

From https://stackoverflow.com/questions/886 ... e-unistr-h this might work

Code: Select all

sudo apt-get install libunistring-dev

Unfortunately, no joy

libunistring.png (113.56 KiB) Viewed 3316 times

[mwieder]

OK - some path difference between our two machines.
Try changing the include line in libfoundation-unicode.h from

Code: Select all

#include "unicode/unistr.h"

to

Code: Select all

#include "../../prebuilt/include/unicode/unistr.h"

[tperry2x]

OK - some path difference between our two machines.
Try changing the include line in libfoundation-unicode.h from

Code: Select all

#include "unicode/unistr.h"

to

Code: Select all

#include "../../prebuilt/include/unicode/unistr.h"

Found #include "unicode/unistr.h" on line 22 of file "foundation-unicode.h"
Changed as below:

edit.png (176.28 KiB) Viewed 3298 times

Same error, different path:

error-persists.png (68.71 KiB) Viewed 3298 times

I will try and chase down a solution on this later, so it can locate "unistr.h"
Real life is unfortunately calling again.

[mwieder]

I HATE GIT SUBMODULES!

So... somehow the prebuilt directory got unhooked from the prebuilt submodule. The reason you're getting 'no such file' is because some subdirectories (esp. "includes") are missing from the main directory. The actual prebuilt directory is on github at
https://github.com/mwieder/livecode-prebuilt . My attempts at gluing the submodule back in place of the 'fake' on in the main structure haven't been successful in the past.

In which case

try navigating to the livecode/prebuilt subdirectory. Check the "git remote -v" location and make sure it's

Code: Select all

origin	git@github.com:mwieder/livecode-prebuilt.git (fetch)
origin	git@github.com:mwieder/livecode-prebuilt.git (push)

and try a git pull. If you're lucky that will solve the immediate problem.
If not you'll have to try forcing the pull.

and I'll have to see about gluing the submodule pointer in place.

[tperry2x]

I HATE GIT SUBMODULES!

Why stop at just submodules?

Check the "git remote -v" location
origin git@github.com:mwieder/livecode-prebuilt.git (fetch)
origin git@github.com:mwieder/livecode-prebuilt.git (push)
and try a git pull

Sorry. (Whoosh)
That's the sound of that disappearing right over my head!
Was just about to comment and confirm I don't have those directories.
Can you not send me a dropbox / mega link with everything in - possibly z7'd straight from your computer instead?
I'll provide you a location you can upload it to if you'd like - anything to not have to deal with git.

no unistr.h.png (33.14 KiB) Viewed 3288 times

just-these.png (41.51 KiB) Viewed 3288 times