Black Hill Software

  • Home
  • Products
    • EasySMF
      • Online Manual
      • Release Notes and Latest Version
      • License Agreement
    • EasySMF:JE
      • EasySMF:JE Java Quickstart
      • Release Notes and Latest Version
      • Javadoc
      • License Agreement
    • EasySMF:RTI
      • EasySMF:RTI – SMF Real Time Interface
      • Javadoc
    • 30 Day Trial
  • Purchase
    • How to Buy
    • Purchase a License
  • Support
    • Documentation
      • EasySMF Desktop Online Manual
      • EasySMF:JE Javadoc
      • EasySMF RTI Javadoc
      • EasySMF JSON Javadoc
      • z/OS Utilities Javadoc
    • EasySMF Support
    • Get the latest version of EasySMF
    • EasySMF:JE Support
    • Get the latest version of EasySMF:JE
  • News
  • Contact
    • Support
    • Sales

Using zEDC compression for SMF data

April 29, 2024 by Andrew

Should you use zEDC to compress SMF data? Should you use both zEDC and CICS software compression?

There was a recent discussion on the IBM-MAIN email list about exploiting zEDC. One of the topics was SMF data, including whether to turn off the software compression for CICS SMF records.

I ran some experiments to compare the processing of SMF records with and without the various compression options. The tests were run using a dataset containing 6.6 GB of CICS SMF data compressed with CICS software compression. For the tests without software compression, the data was uncompressed and copied to another dataset compressed with zEDC. The uncompressed data size was about 39 GB.

Disclaimer: As always, your mileage may vary. This isn’t intended as a benchmark – it’s just one set of tests run against one dataset. This is a very lightly loaded system running under VM. It might bear no resemblance to a busy production system. CPU times etc. might not be accurately reported.

Data size

CICS software compression124000 tracks
CICS software compression + zEDC18500 tracks
zEDC compression35000 tracks

The data is very compressible. The software compressed data compresses further using zEDC and the final size is smaller than compression by zEDC alone.

Reading data

To test the overhead of zEDC when reading SMF data, I used the IFASMFDP program with the OUTDD pointing to DUMMY. This reads the data and produces a report on the numbers of each SMF record type, showing that individual records were read.

Elapsed (seconds)CPU (seconds)
CICS software compression
(read 6.6GB)
30.10.86
CICS software compression + zEDC
(read 6.6GB, ~1GB compressed)
2.21.2
zEDC compression only
(read 39GB, ~1.9GB compressed)
6.55.6
Chart showing comparative times to read data

This program doesn’t do anything with the CICS data so there is no software decompression. It’s just a indication of the cost of reading SMF records with and without zEDC compression.

I was surprised how slow it was to read the records without zEDC. Is this a result of e.g. running under VM, or is it typical? zEDC was very fast, relative to the uncompressed data – better than 10x speed-up.

Copying Data

I ran the IFASMFDP jobs but this time with OUTDD specifying a real dataset. The difference between the read jobs and the copy jobs should give an indication of the cost of writing the data using zEDC.

Elapsed (seconds)CPU (seconds)
CICS software compression
(read/write 6.6GB)
621.58
CICS software compression + zEDC
(read/write 6.6GB, ~1GB compressed)
8.32.31
zEDC compression only
(read/write 39GB, ~1.9GB compressed)
19.411.88
Chart showing comparative times to copy data

Again zEDC was much faster but unsurprisingly there was a CPU cost (less than 0.2s/GB).

SMF Reporting using Java

I ran the CICS Transaction Summary report from the EasySMF Java samples:

https://github.com/BlackHillSoftware/easysmf-samples/blob/main/sample-reports/src/main/java/com/smfreports/cics/CicsTransactionSummary.java

This report produces a basic summary of CICS transactions grouped by APPLID and transaction name.

Software decompression is performed in Java by EasySMF as required. The program is processing 39GB of data after decompression.

This gives some idea of the overhead of compression on reporting, however this is very dependent on the reporting software you use and your mileage will definitely vary.

Elapsed (seconds)CP (seconds)zIIP (seconds)
CICS software compression
(6.6GB compressed)
64.51.4940.6
CICS software compression + zEDC
(read 6.6GB, ~1GB compressed)
39.51.2545.4
zEDC compression only
(read 39GB, ~1.9GB compressed)
35.02.0238.6
Chart showing comparative times to run a report

One interesting thing here is that zEDC with Java seems to allow more work to move from the CP to the zIIP. The time on CP reduces from 1.49s to 1.25s processing the 6.6GB input. The CP time processing the 39GB compressed with zEDC is only 2 seconds. Reading the same data with IFASMFDP took 5.6 seconds of CPU time. Presumably under Java the rest is included in the zIIP time.

CICS software compression increased elapsed and zIIP times by about 10-15% compared to zEDC compression only, but reduced CP time.

Conclusions

Based on these tests, zEDC works very well with SMF data.

  • Elapsed times were greatly reduced in all cases
  • The CPU time required was fractions of a second per GB
  • CICS software compression appears to be still worthwhile. Reducing the amount of data being processed by zEDC significantly reduces the elapsed and CPU times for SMF copy operations e.g. SMF dump, weekly/monthly processing.

Don’t forget to measure and verify the results on your own systems!

Filed Under: Java, Java SMF

Text message alerts using the z/OS SMF Real Time Interface

April 15, 2024 by Andrew

In this post, I’ll show how you can send SMS text messages from z/OS for failed jobs using Twilio and the z/OS SMF Real Time Interface.

Twilio provides a Java API to use their service, and that can be combined with the EasySMF Real Time Interface to send SMS messages based on real time SMF data.

Sending text messages from z/OS

Twilio is a paid service for production messaging, but you can sign up for a free trial account and send a limited number of messages to a verified phone number.

Twilio provides quickstart documentation here:

https://www.twilio.com/docs/messaging/quickstart/java

You can ignore the stuff about the CLI, all we are going to do is send an outbound message which requires a Twilio account, the Java program and dependencies i.e. the fat jar.

This is the Twilio quickstart Java program with a couple of minor tweaks:

import com.twilio.Twilio;
import com.twilio.rest.api.v2010.account.Message;
import com.twilio.type.PhoneNumber;

public class TwilioTest {
    // Find your Account SID and Auth Token at twilio.com/console
    // and set the environment variables. See http://twil.io/secure
    public static final String ACCOUNT_SID = System.getenv("TWILIO_ACCOUNT_SID");
    public static final String AUTH_TOKEN = System.getenv("TWILIO_AUTH_TOKEN");

    public static void main(String[] args) {
        Twilio.init(ACCOUNT_SID, AUTH_TOKEN);
        Message message = Message.creator(
                new com.twilio.type.PhoneNumber("+14159352345"), // to
                new com.twilio.type.PhoneNumber("+14158141829"), // from
                args[0])
            .create();

        System.out.println(message.getSid());
    }
}

Upload the Twilio fat jar twilio-x.x-jar-with-dependencies.jar to z/OS. It is available from e.g. https://repo1.maven.org/maven2/com/twilio/sdk/twilio/10.1.3/

Java 11 will run single file Java programs without a separate compilation step, so we can just run the program under BPXBATCH:

//ANDREWRG JOB CLASS=A,
//             MSGCLASS=H,
//             NOTIFY=&SYSUID
//*
//BPXBATCH EXEC PGM=BPXBATCH,REGION=512M
//STDPARM  DD *
sh /usr/lpp/java/J11.0_64/bin/java
 /home/andrewr/java/src/TwilioTest.java
 "Hello from z/OS"
//STDENV   DD *
CLASSPATH=/home/andrewr/java/lib/twilio-10.1.3-jar-with-dependencies.jar
TWILIO_ACCOUNT_SID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_AUTH_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
//SYSOUT   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDERR   DD SYSOUT=*

Twilio can also send the messages to WhatsApp using the same functionality.

Sending notifications for failed jobs

Prerequisite: z/OS SMF Real Time Interface

The z/OS SMF Real Time Interface must be active. This requires:

  • SMF running in logstream mode
  • Define an in-memory resource to receive the required records (type 30 in this case)
  • Setup RACF definitions to allow the user running Java program to access the SMF in memory resource

The program

The complete source code for this program can be found here:

https://github.com/BlackHillSoftware/easysmf-samples/tree/main/easysmf-rti/rti-notifications

The following is an overview of the major sections of the program.

Reading the in-memory resource

The first thing to do is set up the connection to the in-memory resource.

public class RtiNotifications
{
    public static void main(String[] args) throws IOException
    {
        try (SmfConnection connection = 
                 SmfConnection.forResourceName("IFASMF.MYRECS")
                     .onMissedData(RtiNotifications::handleMissedData)
                     .disconnectOnStop()
                     .connect();

             // Set up SmfrecordReader to read type 30 subtypes 4 and 5    
             SmfRecordReader reader = 
                 SmfRecordReader.fromByteArrays(connection)
                     .include(30,4)
                     .include(30,5))
        {
            // process data here
        }
    }

    private static void handleMissedData(MissedDataEvent e)
    {
        System.out.println("Missed Data!");
        e.throwException(false);    
    }
}

This code:

  • creates a real time connection to resource name IFASMF.MYRECS
  • sets up an action to be taken if data is written faster than the program can read it and the real time buffer wraps (write a message and suppress the exception)
  • sets up a command handler to disconnect from the resource when the STOP command is received
  • creates a SmfRecordReader to read SMF 30 subtypes 4 and 5 (step and job end) records

The connection and reader will be closed automatically when the program exits the try block.

The connection can be simulated in a development environment by setting environment variables:

SIMULATE_RTI=Y

and

IFASMF_MYRECS=/your/smf/filename

The SmfConnection will read SMF data from the file indicated by the environment variable matching the resource name.

Processing the SMF 30 Records

We want to send the SMS message when the job ends, which is indicated by a subtype 5 record. However, if steps are skipped due to e.g. COND processing or an ABEND, the information in the subtype 5 record might be from a step that didn’t run. These steps have the “flushed” indicator set in the Completion Section.

We need to keep the step end information for steps that did run and check the last executed step when we see the job end record. The Java HashMap provides a convenient way to store information for failed steps.

We can create a class to use as a HashMap key to identify specific jobs. The class has fields for the identifying information (system, jobname, job number, read date and time) which are populated in the constructor. Eclipse can generate the hashCode and equals methods which are required for a HashMap key.

Map<JobKey, Smf30Record> failedSteps = new HashMap<>();

...

private static class JobKey
{
    String system;
    String jobname;
    String jobnumber;
    long readtime;
    int readdate;

    JobKey(Smf30Record r30)
    {
        system = r30.system();
        jobname = r30.identificationSection().smf30jbn();
        jobnumber = r30.identificationSection().smf30jnm();
        readtime = r30.identificationSection().smf30rstRawValue();
        readdate = r30.identificationSection().smf30rsdRawValue();
    }

    // hashCode and equals generated using Eclipse
    @Override
    public int hashCode() {
        return Objects.hash(jobname, jobnumber, readdate, readtime, system);
    }

    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        JobKey other = (JobKey) obj;
        return Objects.equals(jobname, other.jobname) 
                && Objects.equals(jobnumber, other.jobnumber)
                && readdate == other.readdate 
                && readtime == other.readtime 
                && Objects.equals(system, other.system);
    }
}

SMF 30 Subtype 4: Step End

When processing step end records, if the step failed we save the record for later. If it ran successfully, we remove any previous failed step.

if (failed(r30)) 
{  
    // this replaces existing entry if present 
    failedSteps.put(new JobKey(r30), r30);
}
// else it didn't fail, check it wasn't flushed
else if (!r30.completionSection().smf30flh()) 
{
    // If the step wasn't flushed, remove any earlier failed step
    failedSteps.remove(new JobKey(r30));
}

failed is a method which checks whether the step failed:

private static boolean failed(Smf30Record r30) 
{
    return 
            // abended except for S222 (cancelled)
            (r30.completionSection().smf30abd() 
                    && r30.completionSection().smf30scc() != 0x222)
            // or condition code > 8
            || r30.completionSection().smf30scc() > 8
            // or post execution error
            || r30.completionSection().smf30sye();
}

The criteria can be adjusted as required.

SMF 30 Subtype 5: Job End

When the job ends, we check whether we previously saved a failed step.

JobKey key = new JobKey(r30);                        
if (failedSteps.containsKey(key))
{
    // send notification using information from failed step 
    sendNotification(failedSteps.get(key));                            
    failedSteps.remove(key); // finished with this
}
// or if the type 5 record indicates failure
else if(failed(r30))
{
    // send notification using information from job
    sendNotification(r30);                            
}

Sending the SMS message

The job information is extracted from the SMF record, the Twilio information is supplied in environment variables, and sending the SMS message uses the same process as the Twilio quickstart.

private static final String ACCOUNT_SID = System.getenv("TWILIO_ACCOUNT_SID");
private static final String AUTH_TOKEN = System.getenv("TWILIO_AUTH_TOKEN");
private static final String TO_PHONE = System.getenv("TO_PHONE");
private static final String FROM_PHONE = System.getenv("FROM_PHONE");

private static void sendNotification(Smf30Record r30) 
{
    String messagetext = 
            String.format("%s Job failed: %s %s Step: %d %s Program: %s CC: %s",
                    r30.smfDateTime().toString(),
                    r30.identificationSection().smf30jbn(), // job name
                    r30.identificationSection().smf30jnm(), // job number
                    r30.identificationSection().smf30stn(), // step number
                    r30.identificationSection().smf30stm(), // step name
                    r30.identificationSection().smf30pgm(), // program
                    r30.completionSection().completionDescription());
    
    // Send a SMS notification through Twilio
    Twilio.init(ACCOUNT_SID, AUTH_TOKEN);
    Message message = Message.creator(
            new com.twilio.type.PhoneNumber(TO_PHONE),
            new com.twilio.type.PhoneNumber(FROM_PHONE),
            messagetext)
            .create();
    System.out.println("Message Sent: " + message.getSid());
}

What Next?

Browse and build the EasySMF:RTI sample code:

https://github.com/BlackHillSoftware/easysmf-samples/tree/main/easysmf-rti/rti-notifications

The Github sample has some additional functionality not covered here. You can limit the notifications to jobs running in specific job classes. You can use the same functionality to include jobs by job name or create custom failure criteria for specific jobs.

For simplicity, the sample doesn’t include error handling. If sending the message fails, the program will disconnect from the SMF in memory resource and end with an exception. A production version should probably catch certain errors and retry.

Filed Under: Java, Uncategorized

DCOLLECT Reports and DCOLLECT to JSON using Java

October 27, 2023 by Andrew

EasySMF:JE version 2.2.1 added support for DCOLLECT records.

DCOLLECT reports using Java are fast and powerful. Programs can run on z/OS on zIIP processors, or DCOLLECT data can be downloaded to another platform for reporting.

Sample reports are provided for several different scenarios:

  • Space by SMS storage group and volume
  • Space by high level qualifier, on primary volumes and HSM ML1 and ML2
  • Space by time since last reference (7 days, 1 month, 6 months, 1 year, 5 years)
  • Aged datasets – datasets on primary volumes with a last reference data older than a cutoff e.g. 5 years
  • Datasets by migration date (7 days, 1 month, 6 months, 1 year, 5 years)
  • Frequently migrated datasets – datasets that are being migrated and recalled frequently
  • zEDC compression statistics by high level qualifier

Samples are also provided to compare 2 different DCOLLECT runs and find what changed:

  • Change in space by storage group
  • Change in space by the top 50 high level qualifiers
  • Change in space for selected dataset names

Convert DCOLLECT Records to JSON format

DCOLLECT records can also be converted to JSON format, for use with Splunk, Excel or other JSON reporting tools.

See the Samples

Sample code is available on Github:

https://github.com/BlackHillSoftware/easysmf-samples/tree/main/dcollect

30 Day Trial

Get a 30 Day Trial, or request further information:

Loading

Filed Under: Uncategorized

Finding UID 0 work on z/OS using SMF Data

February 17, 2022 by Andrew

Finding work running under UID 0 and understanding what it is is important for security on z/OS.

Some UID 0 processes run for hours and are easy to find with tools like SDSF. Other work might only run for a fraction of a second, and is virtually impossible to catch “in the act”.

SMF data can help. SMF type 30 records have Unix Process sections for z/OS unix work. The Unix Process section includes the UID so you can use it to find UID 0 work.

Here is an example of the EasySMF Unix Work report:

EasySMF Unix Work Report, filtered to show UID 0.

The UID field allows you to filter the report to include only work with UID 0.

The report builds a tree using the parent process information so you can see the relationship between different tasks. Information from SMF about the running program is included, which can help to understand what the work is doing.

The Elapsed column shows how long these tasks were running. Most of these tasks were part of system startup and many ran for less than half a second, so it would be very difficult to catch them in real time.

You can download a 30 day trial to see what the data from your system looks like here:

EasySMF 30 Day Trial

Filed Under: EasySMF News

Apache Log4j CVE-2021-44228 Information

December 14, 2021 by Andrew

Black Hill Software does not use or distribute Apache Log4j in any of our products.

EasySMF:JE does use SLF4J which can be configured by the customer to use Log4j, if the customer provides the Log4j components. Even in this case EasySMF:JE does not log any information from untrusted sources so we do not believe it is vulnerable to this exploit.

However, if customers have configured logging to use Apache Log4j they should upgrade Log4j to a fixed version.

Filed Under: EasySMF News, Java

  • 1
  • 2
  • 3
  • …
  • 7
  • Next Page »

30 Day Trial

EasySMF and EasySMF:JE are available for a free 30 day trial. Download now and start using them immediately.
30 Day Trial

Information

EasySMF:JE Java API for SMF: Quickstart

Java vs C++ : Drag Racing on z/OS

News

  • Using zEDC compression for SMF data
  • Text message alerts using the z/OS SMF Real Time Interface
  • DCOLLECT Reports and DCOLLECT to JSON using Java

Black Hill Software

Suite 10b, 28 University Drive, Mt Helen, VIC 3350, Australia
PO Box 2214, Bakery Hill, VIC 3354, Australia
+61 3 5331 8201
+1 (310) 634 9882
info@blackhillsoftware.com

News

  • Using zEDC compression for SMF data
  • Text message alerts using the z/OS SMF Real Time Interface
  • DCOLLECT Reports and DCOLLECT to JSON using Java

Copyright © 2025 · Enterprise Pro Theme on Genesis Framework · WordPress · Log in