(More) Job Tools not running external shell script

Good morning,

 

I might be setting myself up for huge embarrassment but I am at my wits end.   I'm modifying a legacy inherited korn shell script that executes at the job level (/procs/jsc).  For some reason it just will not run.  I've tried restarting the XPP service, opening up all permissions, the gamut and it just WILL NOT execute : )  

 

To debug, I created a test script (for brevity's sake I've not included all iterations)...

 

#!/bin/bash

echo "0:$0"   > /tmp/mylog;
echo "1:$1" >> /tmp/mylog;
echo "2:$2" >> /tmp/mylog;


exit;

... to see if XPP is even passing something valuable and I get no output.  I've run this from the command line to make sure it writes to the log and all is fine.  I tried ksh, and bash.   I've tried it with and without . /var/www/xpp/xz/etcxyvision/xyv.shpaths, and a whole bunch of variations in between, nothing works.

 

Can anyone help me understand how XPP executes external commands like this and what is the best way to debug problems?  There must be something I'm missing.  Any help would be appreciated.

 

Thanks - Tod

 

Just to add, I'm executing from XPPathfinder, right click on job tool, more tools, and selecting the script from the list.  The 'same' program in Perl works!

Parents
  • Hi Tod,

    What release of XPP are you running (e.g. 9.2.0, 9.2.1, etc.)?

    What platform?

    How the script is executed from PathFinder definitely depends on the platform (Windows versus Unix/Linux) and there might be variations by XPP release.

  • Hi Jonathan,

    We are running XPP-9.2.1.0, Linux version 4.14.19-1.el7.ib.x86_64. I did notice today that both the jsc and dsc directories are populated with Perl programs that work. The single shell script I'm working with is the one I'm having a problem with. Is there possibly something configuration wise that needs to be set to get shell scripts to work from the Job Tools dialogue? Maybe SEL settings? If not, is there a way; since the job is running in the background, to trace or debug it to get some clues?


    Thanks - Tod
  • Tod,
    Did you start from scratch on your script or modify an existing one? The reason I ask is, that since it is a shell script, the starting line at the top of the PERL file will dictate the program to run.
    If you look at the top of the ones that are working, you should see something like:
    #! /etc/xyvision/common/perl/bin/perl

    So please make sure that is at the top of your file. Also, since it is Linux/Unix the shell script needs to be executable via the "-x-" attribute on the file itself.
    A 'ls -l file.pl' will show you the attributes. It must be set like the others.
    If you don't see the 'x', then a "chmod a+x file.pl" will set it to be like the other files.

    Hopefully it will then run.

    Steve Piercey
    SDL
  • Hi Tod,

    Sorry, there's currently no way to "trace" or "debug" the process of running a Job Tool - any tracing or debugging would have to happen within the tool once it got started - but of course the problem you're having is not being able to get it started.

    Like Steve Piercey, I strongly suspect that your problem is that you do not have the "execute" (---x--x--x) bits set on your script which is a "necessity" on the Unix/Linux platforms for scripts.

    I think Steve missed that you said that doing your own Perl script did work, while trying to do a korn or bash script did not.

    And assuming that your problem is actually that you have neglected to add the "execute" bits on your script, I can explain (and reproduce) that.

    I set up my own test bash script in $XYV_EXECS/procs/jsc, but without any "execuite" bits on the script file, and when I ran it from More Tools nothing happened:
    > ls -l /xz/procs/jsc/mytest
    -rw-rw-rw-   1 username group      40 Mar 21 15:47 /xz/procs/jsc/mytest
    > cat /xz/procs/jsc/mytest
    #!/bin/bash
    echo "$0 $1" > /tmp/mytest

    Then I added the "execute" bits on the script file, and then when I ran it from More Tools it worked:
    > chmod +x /xz/procs/jsc/mytest
    > ls -l /xz/procs/jsc/mytest
    -rwxrwxrwx   1 username group      40 Mar 21 15:47 /xz/procs/jsc/mytest
    > cat /tmp/mytest
    /xz/procs/jsc/mytest /xpp/alljobz/CLS_xyvision/GRP_comp/JOB_blkmerge

    I'm assuming that you also did not have the "execute" bits set on your xxxxx.pl Perl script that you tested and it did run anyway.

    Why?

    Because the way that PathFinder runs a Job Tool is as follows:

    1. It checks if filename of tools ends with ".pl"
    2. If so, it prepends "/etc/xyvision/common/perl/bin/perl " (or on Windows just "perl ") to the command it runs.
      So that's why having a xxxxx.pl Perl script, even without the "execute" bits set, works because the command being run actually is just running "perl" and then Perl reads the script itself (and that does not require the "execute" bits).
    3. On Unix/Linux, if the tool filename does not end with ".pl" then PathFinder prepends "/bin/csh -bcf " to the command it runs.
      Apparently doing it with these csh options the csh program requires the specified script have the "execute" bits set.

    If you were to take your test Perl script that you tried and renamed it without the ".pl" extension and tried it again (without the "execute" bits set), you would find that it no longer worked.

    Hopefully, that's explaining what is happening to you.

    In a nutshell, the "fix" is most likely to just make sure your scripts (of any flavor) always have the "execute" bits set on them.

Reply
  • Hi Tod,

    Sorry, there's currently no way to "trace" or "debug" the process of running a Job Tool - any tracing or debugging would have to happen within the tool once it got started - but of course the problem you're having is not being able to get it started.

    Like Steve Piercey, I strongly suspect that your problem is that you do not have the "execute" (---x--x--x) bits set on your script which is a "necessity" on the Unix/Linux platforms for scripts.

    I think Steve missed that you said that doing your own Perl script did work, while trying to do a korn or bash script did not.

    And assuming that your problem is actually that you have neglected to add the "execute" bits on your script, I can explain (and reproduce) that.

    I set up my own test bash script in $XYV_EXECS/procs/jsc, but without any "execuite" bits on the script file, and when I ran it from More Tools nothing happened:
    > ls -l /xz/procs/jsc/mytest
    -rw-rw-rw-   1 username group      40 Mar 21 15:47 /xz/procs/jsc/mytest
    > cat /xz/procs/jsc/mytest
    #!/bin/bash
    echo "$0 $1" > /tmp/mytest

    Then I added the "execute" bits on the script file, and then when I ran it from More Tools it worked:
    > chmod +x /xz/procs/jsc/mytest
    > ls -l /xz/procs/jsc/mytest
    -rwxrwxrwx   1 username group      40 Mar 21 15:47 /xz/procs/jsc/mytest
    > cat /tmp/mytest
    /xz/procs/jsc/mytest /xpp/alljobz/CLS_xyvision/GRP_comp/JOB_blkmerge

    I'm assuming that you also did not have the "execute" bits set on your xxxxx.pl Perl script that you tested and it did run anyway.

    Why?

    Because the way that PathFinder runs a Job Tool is as follows:

    1. It checks if filename of tools ends with ".pl"
    2. If so, it prepends "/etc/xyvision/common/perl/bin/perl " (or on Windows just "perl ") to the command it runs.
      So that's why having a xxxxx.pl Perl script, even without the "execute" bits set, works because the command being run actually is just running "perl" and then Perl reads the script itself (and that does not require the "execute" bits).
    3. On Unix/Linux, if the tool filename does not end with ".pl" then PathFinder prepends "/bin/csh -bcf " to the command it runs.
      Apparently doing it with these csh options the csh program requires the specified script have the "execute" bits set.

    If you were to take your test Perl script that you tried and renamed it without the ".pl" extension and tried it again (without the "execute" bits set), you would find that it no longer worked.

    Hopefully, that's explaining what is happening to you.

    In a nutshell, the "fix" is most likely to just make sure your scripts (of any flavor) always have the "execute" bits set on them.

Children
  • Jonathon,

    Wait, what? XPP is executing shell scripts using the C shell?:

    "The C shell is a Unix shell created by Bill Joy while he was a graduate student at University of California, Berkeley in the late 1970s."


    Not to take Bill Joy's name in vain, but isn't CSH kind of a dinosaur? I'm so used to working with Linux and using the bourne (Dinosaur Jr. ?) or bash shell I routinely remove csh from my new Linux builds. Bad move this time. Re-installing did the trick.

    My guess is as a software vendor you need to use the shell version that will work for your needs across the multiple environments you support while keeping maintenance costs in check, so no disrespect. I'm just happy that you added that bit of information in your response. As usual, thank you all for your help.


    - Tod
  • Tod,

    I previously said: Sorry, there's currently no way to "trace" or "debug" the process of running a Job Tool - any tracing or debugging would have to happen within the tool once it got started ...

    Yesterday I had looked at the code for any "debug" mode, but missed it the first time.

    I chanced upon it this time that I looked.

    If you type Ctrl+Shift+F9 in PathFinder, it turns on a "debug" mode that  then pops up a dialog (every time) showing the command that is going to be executed (and prompts for whether to run the command or not). Doing Ctrl+Shift+F9 again turns off that "debug" mode (PathFinder always starts up with this debug mode off).

    Doing that would have showed you that PathFinder was trying to run a /bin/csh command.