Loading from a z/OS Batch Job using SSH

Navigation:  Table of Contents > Using EasySMF > Loading SMF Data >

Loading from a z/OS Batch Job using SSH

Previous pageReturn to chapter overviewNext page

With the help of Cygwin, SSH and Dovetailed Technologies Co:Z Launcher and Co:Z Dataset Pipes you can use a z/OS batch job to run the whole process of loading data into EasySMF. All messages from EasySMFLoad appear in the job output on z/OS, and the step completion code is the exit code from EasySMFLoad.

 

Requirements

 

Windows

 

Cygwin, including the OpenSSH package from the Cygwin Net group.

Co:Z Microsoft Windows Target System Toolkit from Dovetailed Technologies (32 or 64 bit to match the Cygwin version).

 

z/OS

 

OpenSSH from the IBM Ported Tools. For these batch jobs only the SSH client is required - the SSH server does not need to be set up.

Co:Z Toolkit from Dovetailed Technologies

 

Installation Notes

 

Cygwin

 

The Cygwin installer and installation instructions are at the Cygwin web site.

 

When installing Cygwin, expand the Net group to find the OpenSSH package. Cygwin can install OpenSSH as a service so that it starts automatically with Windows.

 

If you will be using a SSH keypair or RACF digital certificate for authentication rather than passwords, you can set:

 

PasswordAuthentication no

 

in the Cygwin /etc/sshd_config file. This means that users cannot use a password to log in using SSH, and prevents password guessing attacks on your Windows system - at least over SSH.

 

Co:Z Microsoft Windows Target System Toolkit

 

Installation instructions for the Co:Z Windows Target System Toolkit can be found on the Dovetail Technologies website.

 

IBM Ported Tools OpenSSH

 

Dovetail Technologies also provide a helpful guide to setting up IBM Ported Tools OpenSSH. Note the information about using ICSF and /dev/random. The ssh-rand-helper program is used if /dev/random is not available and adds a lot of overhead to starting a SSH session.

 

Setting up the server SSHD portion of SSH is not required for these batch jobs but may be useful for other purposes. For example, logging in to z/OS Unix using a SSH client like PuTTY will allow you to test SSH connections outbound from z/OS, using the z/OS OpenSSH client. The SSH client cannot be used from an OMVS session.

 

Authentication needs to be set up for the z/OS batch user on the Windows system. This can use a saved password, SSH keypair or RACF digital certificate. SSH keypair or a RACF digital certificate are recommended. Dovetail Technologies provide detailed documentation on setting this up in Client Authentication Mechanisms.

 

The first time you start a SSH session to the Windows PC you might receive a message that the authenticity of the remote host cannot be confirmed. This is because the public key of the remote system is not in the known_hosts file. You can manually add the key to ~/.ssh/known_hosts on z/OS, or if you are using the command line SSH client you can reply "yes" to the message and SSH will add the key to known_hosts for you.

 

Co:Z Toolkit

 

Installation instructions for the Co:Z Toolkit can be found on the Dovetailed Technologies website.

 

Batch Job Options

 

There are 3 basic options you can choose for transmission of the SMF data.

 

Co:Z can create a separate TCP/IP connection for the data transfer. This avoids the overhead of sending it through an encrypted SSH tunnel, but the data is transmitted in the clear.

Co:Z can tunnel the data through the SSH connection. In this case the data is encrypted in transit.

Co:Z can tunnel the data through the SSH connection, and tell SSH to compress the data. This will probably improve transfer speed for data that is not already compressed, but obviously has more overhead (CPU time).

 

Tunneling and compressing the data are selected with the COZCFG DD statement:

 

//COZCFG DD *  

ssh-tunnel=true

ssh-options=-C

 

Sample JCL

 

Sample JCL is provided to load data using a z/OS batch job. The job uses 3 steps to transfer the data to a file, load the data then delete the file. The steps could be combined, but separate steps allow the job to be restarted part way through if a step fails.

 

Note: It should be possible to load the data directly from z/OS using Co:Z Dataset Pipes to pipe the data to EasySMFLoad without an intermediate file, but this doesn't currently work. The reason is under investigation - the current suspect is an incompatibility between how Cygwin and .NET handle Windows pipes.

 

cozlaunch.txt

 

cozlaunch.txt contains JCL to load data into EasySMF from a z/OS batch job.

 

//JCLLIB JCLLIB ORDER=SYS1.COZ.SAMPJCL                               

//*                                                                  

//* The COZPROC steps could be combined into one eliminating the     

//* overhead of establishing multiple ssh sessions, however this     

//* arrangement allows the RC of each operation to be reported       

//* separately and the job to be restarted at an intermediate step.  

//*                                                                  

//* S1: Transfer data over ssh tunnel to a temporary file on the PC  

//*                                                                  

//* The following COZCFG options specify that the data will be       

//* transferred through the ssh tunnel (instead of a separate        

//* connection) and that the ssh data should be compressed.          

//* If you don't want to compress the data in transit eg. you have a 

//* fast network or the data is already compressed remove the        

//* ssh-options parameter. The data will still be encrypted via the  

//* ssh tunnel. If you don't require encryption or compression (and  

//* firewalls permit), you can remove both COZCFG parameters and data

//* will be transmitted over a separate connection in the clear,     

//* avoiding the encryption overhead.                                

//*                                                                  

//* There is no benefit in specifying ssh-options=-C without         

//* ssh-tunnel=true because the data will be carried on a separate   

//* connection and will not be compressed.                           

//*                                                                  

//S1  EXEC COZPROC,ARGS='windowsuser@windowshost'                 

//SMFIN     DD DISP=SHR,DSN=smf.dataset                           

//STDIN     DD *                                                  

fromdsn -b -l rdw DD:SMFIN > smf.file                             

//COZCFG DD *                                                     

ssh-tunnel=true                                                   

ssh-options=-C                                                    

//*                                                               

//* S2: Load data from the temporary dataset using command line   

//*     EasySMFLoad                                               

//*                                                               

//* The path to the EasySMFLoad program is in Cygwin/Unix format, 

//* because it is interpreted by the Cygwin shell.                

//* The parameters for the EasySMF repository location and file   

//* name are Windows format because they are interpreted by       

//* EasySMFLoad - a Windows program.                              

//*                                                               

//S2  EXEC COZPROC,ARGS='windowsuser@windowshost'                 

//STDIN     DD *                                                  

/cygdrive/c/Program\ Files\ \(x86\)/EasySMF/EasySMFLoad.exe \     

    "C:\...your-repository" \                                     

    smf.file                                                      

//*                                                               

//* S3: Delete the temporary file                                 

//*                                                               

//S3  EXEC COZPROC,ARGS='windowsuser@windowshost'                 

//STDIN     DD *                                                  

rm smf.file                                                       

cozlaunch.txt