Still working out some of the details, but I will try to answer some of your questions.

The suggestion about 'wait' for the long API calls is valid. This would be 'print', compose, and user command , but I will point out that it is inconsistent as far as the API is concerned. All calls are synchronous except...

Files will be sent/received as attachments. This is the only known way of delivering files that can be "streamed" and not eat up memory when the files get large. Any kind of HTTP client can deal with form-attachments and the file auto-magically gets placed onto the file system. It does also support "inline", so if you were in a browser application make XPP REST calls (via jquery or some other client), the file would be streamed directly to the web-browser.

To get the status, you do a http://...../processes/:taskid. It returns the output of the command or a "running" if it is still active. Good idea about the current output being the last line of "stdout". Could be useful and sort of mimic what happens in the background processor. Will have to think about that.

Files created by "print"; ie. psfmtdrv or divpdf will reside on the file system. You name it with arguments to the "print" and it must be retrieved with a 'getfile' after the task is completed. You dont want to be passing files as base64 encodings in JSON returns because of memory usage (as stated above).

All good questions, and glad there is interest. REST is much better than SOAP

Or alternatively it could be that in the 'completed' JSON status object you return also the url to the resulting file.

So that when we do a GET to that url we simply get the resulting PDF or XML or whatever file ...

And talking about files, how do plan to send the content of a file? Inside a JSON object as a base64 encoded string?

And if you send it as a JSON object you could maybe also include the path of the file and its name on the XPP system?

And now on what exactly happens when we check a (task) ID

Can you give us an idea on how you see the communication taking place?

Here are my thoughts on a possible implementation

So a GET .../xpp/docs/first:xyvision:comp:blkmerge:1-blkmerge/processes/compose will run in async mode and return a task ID

Lets say the command to check on this task ID is:

GET ../xpp/status/#TASKID will have to return the status, which could be either 'running', 'completed', 'error'. I guess that this status will be returned as JSON object  { "status": "running"}

It could be nice if this object also contains something like { "status": "running", "progress": "processing page 5"}, so that the caller gets an idea of what is going on and where we are in the process.

When the task is completed the response could be:

{ "status": "completed", "progress": "...the complete compose log ...", "exitcode": 0}

The JSON object returned by XPP could contain other information that you think could be useful. Implementations not interested in the extra information can just ignore the parts that they are not interested in.

The same could be true for all commands that just get executed (compose, toxsf,..)

But I wonder how things will be done for commands that generate a resulting file like divpdf, xychange,..

Will it be something like:

GET  .../xpp/docs/first:xyvision:comp:blkmerge:1-blkmerge/processes/divpdf -> returns a 200 + TASKID (in a JSON object?)

Then we do a number of GET../xpp/status/#TASKID till xpp returns a status of completed with also all console output generated by the divpdf process.

Then do we get access to the resulting PDF file?

Could it be as simple as doing a GET ../xpp/result/#TASKID ?

Because like that we do not have to care about where xpp will generate the pdf file or even how it is named by xpp...

Further on the async/sync thing.

Glad to see that you make a distinction between short and long run calls.

And yes the short run calls can return their result immediately.

But I would suggest that all long run calls by default are executed in async mode and just return a (task) id.

Running these commands in an sync way is just too risky and sooner or later will result in a time out error.

So I would like to suggest to let go on the -nowait option and turn things around.

->Long run commands (like compose) run in an async way by default and XPP just returns a task id.

If you do not want this, you add the -wait option...

Up to engineering to define what calls are executed as 'long'/nowait/async and what calls get executed as short/wait/sync.

I strongly believe that this will result in a lot less 'bad' implementations.

Continuing the discussion of sync/async. From a direct socket perspective, the new rest API would maintain the socket so time out could be a problem (regardless of whether the client is using a callback to be notified when the data is ready). This is not a problem for the 'short term' functions you might want to run (like set the active style in the job ticket, or check with a division exists).

But for long term processes (like compose, print), I can see the need for a launch without waiting. The "nowait" would start the process in the background, return an ID and provide a way to check on the status of the command based on ID. If the command finished, you would get the status code and the stdout/stderr messages.

So minimally, a "nowait" option should be provided for compose, print, and "usercommand".

This type of connection cannot be tied to a callback (unless the client writes some kind of timer), because in this scenario the XPP rest server cannot send data back to the client via HTTP because the connection is gone. It has to be a query using the ID to get the status.