Spawning another process with LCB and C standard library calls

[OpenXTalkPaul]

I was trying to wrap libMikMod (tracker music player) and crashing over and over, it may have to do with pthreads, which is the C POSIX Threads library. Anyway that had me looking into binding to some of these things with FFI again and was re-reading some of Dr. Peter Brett's articles on the topic.
Particularly, Undergrowth (running extension builder modules without the script engine)
And this one:
http://blog.peter-b.co.uk/2015/09/using ... ecode.html

That article provides a way for an Extension to act as shell() without using shell(), but it looks like it's blocking until the shell command exits just like shell() does.
This has me thinking again how a similar c standard library call could make it possible to spawn a process from Extension Builder module, so say I have a command line tracker player openmpt123 (comes with libopenmpt) installed on the system, I could could theoretically spawn a subprocess of that from my builder module to playback a mod file, and then manage it, monitoring it's status occasional from a timer handler that runs until the task is done, basically I'm thinking of an extension module that acts like a non-blocking terminal instance to run a command line tool from those installed on the system or from within it's extension module package's resource folder (which could encapsulate any required things nicely into the package).

Theoretically you could fork(2) could clone the entire IDE process into another separate process, right?
https://man7.org/linux/man-pages/man2/fork.2.html

https://stackoverflow.com/questions/947 ... and-thread

https://stackoverflow.com/questions/171 ... ocess-in-c

http://man.yolinux.com/cgi-bin/man2html ... mand=popen

[tperry2x]

You can just pipe the shell to dev null so that it becomes non blocking.

Huh you say?
Like this:

Code: Select all

put "mkdir ~/testfolder" into tCommand -- this is my command
put tCommand & " > /dev/null 2>/dev/null &" into tNoWaitShellCommand -- add no wait suffix
get shell(tNoWaitShellCommand) -- run the shell non-blocking

[OpenXTalkPaul]

You can just pipe the shell to dev null so that it becomes non blocking.

Huh you say?
Like this:

Code: Select all

put "mkdir ~/testfolder" into tCommand -- this is my command
put tCommand & " > /dev/null 2>/dev/null &" into tNoWaitShellCommand -- add no wait suffix
get shell(tNoWaitShellCommand) -- run the shell non-blocking

That's cool, I've similarly used '&' at the end of a command to make it run in the background.
The problem is I want my scripts to be able to interact with and/or manage the running process(es).
At least so that when a processes running in the background this way finishes its task, I can have my scripts respond to that event. The way I've done that in the past was to use 'open process tNoWaitShellCommand' instead of shell() and then poll the process(es) to see if it's still running, and 'read from process' for retrieving any output messages every so often (like every 10th of a second), but that can be a bit of a pain.

[tperry2x]

Another way you could do it is by running the process non blocking, but having any callback messages operating on a local port which the stack is listening for. This way, you can communicate backwards and forwards to your heart's content, while the non-blocking command is running.

[tperry2x]

Or, if you are only concerned with if something is running - perhaps get the PIDs (process IDs) of all running apps just before you run your command, then get the PIDS afterwards and do a compare - you then have your process ID (which you can filter down by name to refine further). You can send a quit message from the stack to terminate that, and a check for the PID if you are re-running the function - thereby eliminating the need for recursive and repeated checking.

[OpenXTalkPaul]

Or, if you are only concerned with if something is running - perhaps get the PIDs (process IDs) of all running apps just before you run your command, then get the PIDS afterwards and do a compare - you then have your process ID (which you can filter down by name to refine further). You can send a quit message from the stack to terminate that, and a check for the PID if you are re-running the function - thereby eliminating the need for recursive and repeated checking.

For things that I'm only concerned if the process is still running or not, which I've done mostly with 'player' or converter type things (such as ffmpeg or mplayer), that is basically what I do, using the syntax 'the openProcesses' and 'the openProcessIDs' (which is a list of only the PIDs that were launched by the Engine)' to build and maintain a list of records of PIDs + the process (command line) of that PID of all subprocesses that are running.

Not sure about Linuxes, but on macOS the engine doesn't seem to update these lists in 'the openProcesses'/'the 'openProcessIDs' global properties. They do not automatically remove a process once the process is done running, the process is still open as far as the engine is concerned. So when the PID of a process tProcessCommand in that openProcessIDs list has an PID of zero 0 that means that process is no longer running, then you can run 'close processes tProcessCommand' of the process that is the corresponding line in 'the openProcesses' to let the Engine know that process ended so close it. On macOS this is how I've had to work with multiple processes.

It might be good to have a multi-processing purposed library for doing this sort of thing, which could contain general use methods for generating callbacks messages and such, so one wouldn't need to do too much set up just to get going playing around with whatever arbitrary command line tools.

I believe that the way it works with POSIX is that a thing running as a subprocess should have the same privileges, as far as provisioning, filesystem read write permissions, etc., that the main process has (our Engine that launched the subprocess).

[OpenXTalkPaul]

If you could make a demo showing how to pipe a cli stdout/stdin to local sockets and then read from / write to those sockets that would be great. I haven't played around with that sort of thing very much. I' have played around a bit with the 'chat client/server' demo stack that's around, a long time ago.

Alternatively, I suppose something could be set up with FIFO file pipes? I'm not sure how that might work from a xTscripting standpoint, I've never tried it.

I'm not sure if it would be any different building a library to launch processes via XTBuilder + FFI / libC, but my thinking was to use Builder's 'OnTimer' (of the builder 'widget' module) to set up the processes list / check loop (which I believe would be running in the UI thread, and not the script interpreter thread), and it would 'post' a message back to the Script Engine whenever a process finished, so scripts could respond, something like 'processDidEnd'.
So then all one would need to do is set a commands/tasks list and then respond to the callbacks something like:

Code: Select all

on processDidEnd pTheEndedProcess, pTheEndedProcessID
    Answer "The Process " & pTheEndedProcess & " with ID " & pTheEndedProcessID & " ended"
end processDidEnd

[tperry2x]

It might be good to have a multi-processing purposed library for doing this sort of thing, which could contain general use methods for generating callbacks messages and such, so one wouldn't need to do too much set up just to get going playing around with whatever arbitrary command line tools.

Yes, it would be good to have a way we can reliably start and end processes from the IDE that is cross-platform and works.

I believe that the way it works with POSIX is that a thing running as a subprocess should have the same privileges, as far as provisioning, filesystem read write permissions, etc., that the main process has (our Engine that launched the subprocess).

Yes, absolutely - generally on Windows too as well as Linux - the privileges are inherited from whichever process spawned it. Unless you invoked something with sudo (linux) or "open as administrator" - (UAC prompt, windows) of course. Then, any sub-processes launched from these would also have elevated permissions.