Tim in Indianapolis sent me a note in late December about his "wishlist" for My i-.NET topics. One of his wishes was for a way to "download job logs for quicker processing/searching than what is offered by iSeries Navigator." To provide a flexible solution, the application that I developed extracts data from a job log and writes it to a Comma-Separated Variable (CSV) file. CSV files can be readily imported by many other applications, including Microsoft Excel, databases, and other analysis tools.

Some of the options to work with job logs in the iSeries Navigator are shown in Figure 1 and Figure 2. Figure 1 shows the Navigator's output queues display. Using that display, you can right-click on a job log file and open it in a viewer (shown in Figure 2). The viewer is easier to work with than the equivalent green-screen spool file display because you can see the entire width of the job log spool file.

The starting point for this month’s program is also shown in Figure 1, which is the Export menu option. Using that option, you can export from the spool file to a text file on your PC or on a network drive. An example of an exported spool file is shown in Figure 3. This file is opened in Notepad, and you can see that it has retained all of the characteristics of the spool file that is displayed in Figure 2. The only difference is that blank lines that are shown on the spool file display (Figure 2) are omitted when the job log is exported to the text file (Figure 3). My application does not require the blank lines because I am only interested in parsing the data from the job log.

Figure 4 shows the CSV file that is produced by the application. The data in Figure 4 was extracted from the job log text file shown in Figure 3. Finally, Figure 5 shows the CSV file after it was imported into Microsoft Excel.

The ProcessJobLog Application

The ProcessJobLog application includes these three C# source code files:

• ProcessJobLog.cs –- This is the main class for the application. It runs as a Console application.
• JobLog.cs –- This class contains private member variables and properties for job log header data.
• JogLogMessage.cs –- This class contains private member variables and properties for a message in the job log.

To help you understand the application, I’ll start by describing the JobLog and JobLogMessage classes. When the program reads through the job log text file, it processes one line of text at a time. As you can see on the job log in its original format (Figure 2 or Figure 3), the information that you need may be contained on different lines of the job log. So as you read a line, you need to save the data as you parse it.

The JobLog class contains the header data for the job log. The header data is located at the top of the job log in the second and third lines in Figure 3. The data that will be parsed, stored, and ultimately included in the CSV file includes the job name, user, number, job description, and job description library. It is trivial to extract that data when reading the job log text file; the slight wrinkle that you need to keep in mind is that the header data repeats in the job log text file at each point where a page break occurs in the original job log spool file. So the job log header data is parsed only the first time it is encountered when reading the job log text file; the parsed values are stored in the JobLog class.

The JobLogMessage class contains data for each message that is in the job log. You can see this data in Figure 2 and Figure 3 where it is identified by the column headings and by the labels that appear below each of the messages. Some of the data that is parsed for each message includes the 7-character message ID, the message type (Information, Diagnostic, and so on), message severity, date and time, from program, from library, from instruction, and to program, library, and instruction.

The message line itself is not overly complicated to parse although I did need to make allowance for the occasional omission of the to-library value (for example, see the first message CPF1124 in Figure 3).

After parsing a message line, the program then looks for the actual message text itself, identified by the "Message . . :" label (the label is longer on the listing). The message might continue over multiple lines. In this application, I simply parsed the first line of the message.

Additional message data fields that can be parsed but are not currently parsed in this application include other labeled fields: cause, from module, from procedure, statement, to module, to procedure, and others. I did not find a definitive reference to i5/OS job logs that describes all the possible labels that can appear in a job log, so I coded the application based upon what I found in several example job logs.

Compiling and Running the Application

To use the application, you can start a new C# Console project in Visual Studio 2005 or 2008. You can also use Visual C# Express 2005 or Visual C# Express 2008. After creating the project, right-click the project name in the Visual Studio Solution Explorer and import the three C# classes that I described above. You can then build the application in Visual Studio and run it from there. It will create a stand-alone executable file (.exe) that you can run from a Windows command prompt.

Before running the program you need to export one or more job log spool files from your System i to a drive that is accessible from your PC. The easiest technique to develop and test the application is to use the Export menu option in the iSeries Navigator, shown in Figure 1. You can also select more than one job log spool file in the iSeries Navigator and use the Export option. You are prompted for a path name and file name for each file that is exported from iSeries Navigator. If you decide to use this application for realtime processing of job logs, you’ll probably want to use the i5/OS Copy Spool File (CPYSPLF) and Copy to Import File (CPYTOIMPF) commands or an equivalent technique to get the job log spool files into text files that the application can work with.

When you run the program, it expects two input parameters. The first parameter is the fully qualified drive and path name of the input job log text file. The second parameter is the fully qualified drive and path name of the CSV file that will be the output of the application. If the input file is not found, the application displays a message and ends. If the output file already exists, it is replaced.

After the application is in a CSV file, you can use it in any application that works with a CSV file. For example, Figure 5 shows the job log text file imported into Microsoft Excel. You can use an application such as Excel to quickly examine a job log.

Application Code Notes

As you might expect, the ProcessJobLog application makes extensive use of string-handling techniques. The application starts processing in the Main method, which first verifies that you have provided two parameters for the input and output file names.

In Main, a read-loop is used to read through all the lines of the input text file. As each line is read, it is parsed to determine what type of content is on the line. The job log header is identified and processed, and each line that contains the start of a message is identified.

The message ID is examined in the CheckForMessageId method. This method uses a regular expression to check for the 7-character i5/OS message ID. The first character of a message ID can be any alphabetic (A-Z) character, followed by two characters that can be alphabetic or numeric. The final four characters of a message ID must be a hexadecimal digit (0-9, A-F). If the message ID is encountered at the beginning of the input line, the line is considered to be the start of a message.

Most of the other methods in the application are simple string-handling methods. The CSV header is built in the MakeCsvHeader method. If you decide to add or remove message fields from the CSV file, change the CSV header in this method.

After parsing the job log header and a message, the CreateHeaderFields and CreateMessageFields methods are called to create the comma-delimited line of text that contains the parsed data. Figure 4 shows an example of what the comma-delimited text looks like. If you decide that you don’t need some of the header or message fields, you can simply comment out the sb.Append statements in the CreateHeaderFields or CreateMessageFields methods.

If you want to capture some of the other labeled data (for example, "Cause"), you can use the CheckForKeyword and ExtractKeywordValue methods to look for and parse the value. You can see an example of using those methods in the Main method where the value for the "Message" label is extracted.

Log Analysis

Obviously there are many ways to parse a spool file and get at the data you want. For example, you can parse spool files using an RPG or Java program, in addition to the example shown here.

Feel free to let me know if you develop a log analysis program using the application shown in this article.

Craig Pelkie has worked as a programmer with IBM midrange computers for many years. He has also written and lectured extensively on AS/400 and System i technologies, including client/server programming, Client Access, Java WebSphere, .NET applications for the System i, and web development. You can reach him at cpelkie@systeminetwork.com.