More and more, we're being asked to integrate our RPG code with software that runs in Unix environments. PHP scripts, OpenSSH tools (including SFTP), commands for zipping and encrypting files, and QShell utilities for working with the IFS are becoming a part of our everyday toolbox.
The only problem is, it's a hassle to integrate them with our RPG code. Because they're designed for a Unix environment, they expect to receive input from the standard input data stream, and they write output and errors to the standard output and standard error data streams, respectively. That's not very intuitive to the average RPG programmer! We tend to work around this problem by calling a CL program or using the QCMDEXC API to run our Unix command and redirect the output to a file. Then we have to read the file to get the output. A workable solution! But it's a little cumbersome.
I've decided to add a new open-source (free of charge) tool to my website to make things easier. It uses RPG's SPECIAL file support to communicate with Unix commands. You open a Unix command as if you were opening a program-described file in RPG. Then you write data to its standard input using the standard WRITE or UPDATE opcodes. You read back the command's output by using the READ opcode.
RPG can declare a file on an F-spec as a SPECIAL file. A SPECIAL file isn't actually a file at all but rather is a program that looks like a file. When you open a SPECIAL file, it doesn't actually open a file on disk, but instead it calls a separate program and tells that program that you want it to open. When you read from a SPECIAL file, it tells that program to perform a read, and so forth. It's up to the called program to calculate the records returned to you.
The tool I'm introducing today is implemented by a program named UNIXCMD. At the end of this article, I give you a link to where can download the source code for UNIXCMD and install it on your system. To use it, you set it up using an F-spec like this one:
FUNIX IF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
This states that the file named UNIX is a SPECIAL file and therefore is not really a file at all. Whenever you try to perform an operation on the UNIX file, RPG will actually call a program named UNIXCMD and pass the parameter list named UNIXPARM to it.
To start with a simple example, I demonstrate reading a list of files in an IFS directory by using the ls command from QShell. The ls command simply outputs a list of files in the current directory. So if I want to find out what files currently exist in the /tmp directory of the IFS, I could run the following command from QShell:
cd /tmp; ls
Using my new tool, it'd be easy to do this from an RPG program as well. Here's the code to run the preceding command, read its output (the list of files), and then (just for the sake of a simple example) print each filename to the spool.
FUNIX IF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
FQSYSPRT O F 132 PRINTER
D cmd s 5000a
D record s 1000a
D outrec s 132a
IUNIX NS
I 1 1000 Record
C UNIXPARM PLIST
C PARM CMD
/free
cmd = 'cd /tmp; ls';
open UNIX;
read UNIX;
dow not %eof(UNIX);
outrec = record;
except;
read UNIX;
enddo;
*inlr = *on;
/end-free
OQSYSPRT E
O outrec 132
When you use the OPEN opcode to open the UNIX command, it calls the UNIXCMD program, tells it that you want to open the file, and passes along the UNIXPARM parameter list. This parameter list contains the command to be executed, UNIXCMD (which is my tool) runs the UNIXCMD as a QShell command, and sets up the output to go back to your program in the form of a program-described file. When you read from the UNIX file in the RPG program, it reads back the output of the command. When you close the file (in this example, it's closed implicitly when the program ends with LR turned on) the command is completed.
Under the covers there are no actual files used. When the UNIXCMD program runs your QShell command, it does not write the output to a file. Instead, it connects to the QShell command using a pipe, and it reads the contents of that pipe for you. But, since all this is done under the covers, you don't have to worry about it. You simply read it as if you were reading a file.
You'll notice that a SPECIAL file is a program-described file, so I coded input and output specs for the file and used those when reading it. Likewise, if I wanted to write data to the Unix command, I could use output specs. For example, the cat QShell utility reads the data sent to its standard input and can write all that data to an IFS file. So this example will write some data to the IFS:
FUNIX O F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
D cmd s 5000a
C UNIXPARM PLIST
C PARM CMD
/free
cmd = 'cat - > /tmp/myfile.txt';
open UNIX;
except writeToIfs;
close UNIX;
*inlr = *on;
/end-free
OUNIX E writeToIfs
O 'This is some silly -
O text to put in the IFS'
O E writeToIfs
O 'And this is line 2.'
One frustrating limitation of SPECIAL files is that they have to be opened as either input or output. You can either read records from a special file or write records to a special file, but you can't do both in the same program. Ugh! Had this been a disk file, I could've specified "input with add" by using the following syntax:
FMYFILE IF A F 1000 DISK
But for some reason you can't do that with a SPECIAL file:
FUNIX IF A F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
*RNF2028 The File-Addition entry is not valid for this device; defaults to blank.
Ouch. It won't let me perform both input and output to the same file! I have to admit that I don't understand why they imposed this limit on SPECIAL files; it really gets in my way because there are Unix commands where it makes sense to both send input and retrieve output all in one command call.
So I decided to work around the problem by using UPDATE. When you UPDATE a record using my UNIXCMD tool, it treats the update the same way it'd treat a WRITE operation. It doesn't really update anything; it just writes data to the pipe connected to the Unix command's standard input. So I can code the following:
FUNIX UF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
D cmd s 5000a
D record s 1000a
D AsciiData c x'48656c6c6f'
IUNIX NS
I 1 1000 record
C UNIXPARM PLIST
C PARM CMD
/free
cmd = 'iconv -f 819 -t 0';
open UNIX;
except writeAscii;
read UNIX;
dsply (%subst(record:1:5));
close UNIX;
*inlr = *on;
/end-free
OUNIX E writeAscii
O AsciiData 5
In the preceding example, I'm invoking QShell's iconv utility. I'm telling it to translate its input from ASCII to EBCDIC and write the output back to me. The data I write to the iconv utility is the word "Hello," but I've used ASCII hex values to make it into an ASCII string. When iconv translates it to EBCDIC, I read the result back and can display an EBCDIC string on the screen.
So, performing both input and output on the same command is possible! It just requires you to perform updates instead of writes.
ILE RPG has a feature that lets you eliminate input and output specs. Anytime you're working with a program-described file, you can specify a data structure in the result field of your opcode, and RPG will perform the I/O to or from that data structure. Note that this feature is not specific to SPECIAL files: It'll work with any sort of program-described file, including DISK and PRINTER files. In fact, I use this feature a lot when working with flat files. But it works nicely with my UNIXCMD utility as well. The only caveat is that the data structure length must match the record length.
For example, I could rewrite my first example (the "ls" example) as follows:
FUNIX IF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
FQSYSPRT O F 132 PRINTER
D cmd s 5000a
D record ds 1000
D outrec ds 132
C UNIXPARM PLIST
C PARM CMD
/free
cmd = 'cd /tmp; ls';
open UNIX;
read UNIX record;
dow not %eof(UNIX);
outrec = record;
write QSYSPRT outrec;
read UNIX record;
enddo;
close UNIX;
*inlr = *on;
/end-free
Personally, I prefer this syntax to using input/output specs. It's just more intuitive to me if I do things inline in my calcs, rather than having to jump up to input specs, or jump down to output specs.
So far, my examples have all been QShell examples. The UNIXCMD utility is not limited to QShell, however! It can use PASE, too. To tell it to use PASE, you need to add a second parameter to the parameter list containing a "P." For example:
FUNIX IF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
FQSYSPRT O F 132 PRINTER
D cmd s 5000a
D mode s 1A inz('P')
D record ds 1000
D outrec ds 132
C UNIXPARM PLIST
C PARM CMD
C PARM MODE
/free
cmd = 'cd /tmp; ls';
open UNIX;
read UNIX record;
dow not %eof(UNIX);
outrec = record;
write QSYSPRT outrec;
read UNIX record;
enddo;
close UNIX;
*inlr = *on;
/end-free
This example will use the cd and ls commands from PASE to accomplish what the QShell commands were previously doing. Adding the extra P parameter makes it use PASE instead of QShell. I know, I know, that's not a very interesting example, since it repeats something I was already doing! But I like to start simple . . .
Here's a more sophisticated example. It uses the ssh tool (provided with IBM licpgm 5733-SC1) to create a secured/encrypted channel over a network and connect to a FreeBSD machine. (In our shop, we use FreeBSD for many things. This could just as easily be a Linux/Unix machine or another IBM i system.) Once connected, it lists the active processes on that FreeBSD machine and singles out any processes that are running a program named "boxrelay," then returns the list to my program.
FUNIX IF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
FQSYSPRT O F 132 PRINTER
D cmd s 5000a
D mode s 1A inz('P')
D record ds 1000
D outrec ds 132
C UNIXPARM PLIST
C PARM CMD
C PARM MODE
/free
cmd = 'ssh bcserv3.klements.com ps axww "|" +
grep boxrelay "|" grep -v grep';
open UNIX;
read UNIX record;
dow not %eof(UNIX);
outrec = record;
write QSYSPRT outrec;
read UNIX record;
enddo;
close UNIX;
*inlr = *on;
/end-free
Once again, this example simply prints the list of processes to the spool. In my real application that this was taken from, however, I use this to determine if the boxrelay program is running. If it's not running, I issue a new command to start the program.
The preceding example will work only if you have the digital keys and other requirements of SSH configured properly.
If I substitute an SFTP or SCP command for the SSH one, above, I can run an SFTP batch script and get any diagnostic messages back into my program by reading the UNIX file. I can write those messages to a log, just as I've written the output from SSH to a spool. I hope you see how this adds flexibility to your PASE environment!
With the rise of PHP on IBM i, I've seen several discussions about how to call PHP scripts from RPG scripts and pass input/output between them. When you call a PHP script, the parameters you pass to the script are input-only parameters. PHP cannot modify them to return a result to your RPG program. So how do you call a PHP script, pass it data, and get back a response?
The answer is you use the standard input and standard output data streams that are standard in Unix environments. You ask the PHP script to read from its standard input and write the response back to its standard output. Your RPG can use my utility to call PHP and use the UPDATE (as a substitute for WRITE) and READ opcodes to communicate with the PHP script.
For example, maybe I want to invoke the Geocoder.us [2] web service from PHP. I could use the following script to do that:
<?php $address = fgets(STDIN); $wsdl = 'http://geocoder.us/dist/eg/clients/GeoCoderPHP.wsdl'; $client = new SoapClient($wsdl); $result = $client->geocode($address); echo $result[0]->lat . "\n"; echo $result[0]->long . "\n"; ?>
The first highlighted line (the fgets() line) of this PHP script reads a line of text from the standard input stream and puts it in a variable named $address. The script then proceeds to call the geocoder web service and ask it to geocode that address. Finally, the last two lines that I've highlighted print the latitude and longitude that it received from the web service.
If you wanted to invoke this script directly from PASE, you'd CALL QP2TERM to get an interactive PASE shell, and then you'd type the following:
echo "1600 Pennsylvania Av, Washington DC" | php /www/zendcore/htdocs/geocode.php
The output on your screen would look something like this:
$ > echo "1600 Pennsylvania Av, Washington DC" | php /www/zendcore/htdocs/geocode.php 38.898748 -77.037684 $
To invoke the same thing from an RPG program using my utility, I can invoke it as follows:
FUNIX UF F 1000 SPECIAL PGMNAME('UNIXCMD')
F PLIST(UNIXPARM) USROPN
D cmd s 5000a
D record ds 1000
D lat s 11p 7
D lon s 11p 7
C UNIXPARM PLIST
C PARM CMD
/free
cmd = 'PATH=$PATH:/usr/local/Zend/Core/bin && +
iconv -f 37 -t 819 | +
php /www/zendcore/htdocs/geocode.php';
open UNIX;
record = '1600 W Pennsylvania Av, Washington DC';
update UNIX record;
read UNIX record;
lat = %dec(record: 11: 7);
read UNIX record;
lon = %dec(record: 11: 7);
close UNIX;
dsply ('lat=' + %char(lat));
dsply ('lon=' + %char(lon));
*inlr = *on;
/end-free
Note: At this time, it appears that the PASE support is not translating my RPG program's EBCDIC to ASCII, and I haven't yet had time to find out why! So for the time being, I've worked around the problem by invoking PHP through QShell and using QShell's iconv utility to perform the translation, as shown above. Hopefully I'll soon discover the problem so that this workaround isn't needed.
The preceding program writes the address (using RPG's UPDATE opcode) to the PHP script and then reads back the latitude and longitude, demonstrating how an RPG program can interact with a PHP script by both sending data to it and receiving the response.
You can download my new UNIXCMD open-source project from my website. [3] If you have problems with UNIXCMD, please report them to the RPG forum of the System iNetwork Forums [4]. I hope it's useful!
Links:
[1] http://systeminetwork.com/author/scott-klement
[2] http://geocoder.us
[3] http://www.scottklement.com/unixcmd/
[4] http://forums.systeminetwork.com