Black Hill Software

  • Home
  • Products
    • EasySMF
      • Release Notes and Latest Version
      • Online Manual
      • License Agreement
    • EasySMF:JE
      • EasySMF:JE Java Quickstart
      • Release Notes and Latest Version
      • Javadoc
      • EasySMF JSON Javadoc
      • License Agreement
    • 30 Day Trial
  • Purchase
    • How to Buy
    • Purchase a License
  • Support
    • EasySMF Support
    • Get the latest version of EasySMF
    • EasySMF:JE Support
    • Get the latest version of EasySMF:JE
  • News
  • Contact
    • Support
    • Sales

Java mapping for CICS SMF records

August 3, 2017 by Andrew

The EasySMF Java API now has experimental support for CICS records.

Experimental, because I want to get some feedback from CICS users about class names, usage etc. before locking down the design. In particular:

  • Do the class names and organization make sense to a CICS person? Would other names or a different organization make more sense?
  • Are the examples of how to process data clear and useful?
  • Are there areas where terminology is used incorrectly?

The complete Javadoc is here with an overview of the CICS functionality here.

If you have any comments, you can leave feedback in the comments box below, send it to support@blackhillsoftware.com, or give feedback in person at booth 323 at SHARE in Providence, Rhode Island.

You can try out the API using the 30 day trial available here: 30 Day Trial.

Installation information is available here: EasySMF:JE Java Quickstart

Using the API

EasySMF:JE aims to provide a consistent interface across different SMF record types and sections, and converts values to standard Java types for simple programming.

Dates and Times

Dates and times are converted to java.time classes. Java.time can represent dates and times with a precision of 1 nanosecond.

Times representing a duration e.g. CPU or elapsed time are converted to Duration.
Dates and times of day are converted to LocalDate, LocalTime, LocalDateTime or ZonedDateTime depending on exactly what information is in the field. Typically, times based on UTC(GMT) are converted to ZonedDateTime with ZoneOffset.UTC. Other dates and times are converted to LocalDate/Times.
Java has time zone rules so it is possible to apply a ZoneId to a LocalDateTime and perform date aware conversions between time zones.

Numeric Values

1, 2 and 3 byte integer values and 4 byte signed integer values are converted to int (32 bit signed) values.
4-7 byte integer values and 8 byte signed values are converted to long (64 bit signed).

8 byte unsigned values are available as both long (64 bit signed) and as a BigInteger. The long value may provide better performance if the value will not exceed the maximum value for a long. If a value does exceed the maximum value (i.e. the high order bit is set) an exception will be thrown. If the field value might exceed the maximum value for a long, use the BigInteger version.

Integer values greater than 8 bytes are converted to BigInteger.

Floating point values are converted to Java double.

String Values

EBCDIC and UTF8 string/character values are converted to String. Java uses Unicode internally – values are converted from EBCDIC or UTF8.

Flags

Flag bits within a byte are converted to a boolean value indicating whether the bit is set.

CICS Statistics

Reading CICS statistics is very done the same way as reading sections from other records using the API. Sections of a specific type are returned in a List<E> of that type. If there are no sections of the type in the record an empty List is returned. This allows you to iterate over the sections without explicitly checking whether the sections exist in the record – an empty list will iterate 0 times.

Example

The following code reads all FileControlStatistics sections from type 110 SMF records from the DD INPUT.

 try (SmfRecordReader reader = 
         SmfRecordReader
             .fromDD("INPUT")
             .include(110, Smf110Record.SMFSTSTY))
 {
     for (SmfRecord record : reader)
     {
         Smf100Record r110 = new Smf110Record(record);
         for (FileControlStatistics fc : 
             r110.fileControlStatistics())
         {
             //...   process FileControlStatistics sections here
         }
     }
 }

CICS Performance Monitoring

Accessing data from CICS monitoring performance records is slightly different to other SMF records because the data needs to be accessed using a Dictionary.

Dictionary records are handled automatically, however you cannot access the data from a record before a related dictionary record has been seen. You can check whether a dictionary record is available using Smf110Record.haveDictionary() or simply concatenate all required dictionary records ahead of the data records in the input data.

Specific fields are defined by name and type. Then Performance records are read from the SMF record, and specific fields accessed using getField(…) methods or variations.

Example

 ByteStringField transactionField = ByteStringField.define("DFHTASK","C001");
 TimestampField startField = TimestampField.define("DFHCICS","T005");
 TimestampField stopField = TimestampField.define("DFHCICS","T006");
 ClockField dispatchField = ClockField.define("DFHTASK","S007");

 try (SmfRecordReader reader = 
         SmfRecordReader
             .fromDD("INPUT")
             .include(110, Smf110Record.SMFMNSTY))
 {
     for (SmfRecord record : reader)
     {
         Smf100Record r110 = new Smf110Record(record); 
         if (r110.haveDictionary())
         {
             for (PerformanceRecord perfdata :
                 r110.performanceRecords())
             {
                 String txName = perfdata.getField(transactionField);
                 ZonedDateTime start = perfdata.getField(startField);
                 ZonedDateTime stop = perfdata.getField(stopField);
                 double dispatch = perfdata.getFieldTimerSeconds(dispatchField);

                 //...  process data
             }
         }
     }
 }

Complete CICS Statistics reporting sample

These samples are designed to show how to use the API, not to suggest items that you should specifically be reporting. However comments about their relevance are welcome.

import java.io.*;
import java.util.*;
import static java.util.Comparator.comparing;

import com.blackhillsoftware.smf.*;
import com.blackhillsoftware.smf.cics.*;
import com.blackhillsoftware.smf.cics.statistics.FileControlStatistics;

public class CicsFileStatistics 
{
    public static void main(String[] args) throws IOException 
    {
        Map<String, Map<String, FileData>> applids = 
                new HashMap<String, Map<String, FileData>>();

        try (SmfRecordReader reader = 
                args.length == 0 ? 
                SmfRecordReader.fromDD("INPUT") :
                SmfRecordReader.fromStream(new FileInputStream(args[0]))) 
        {
            reader.include(110, Smf110Record.SMFSTSTY);
            for (SmfRecord record : reader) 
            {
                Smf110Record r110 = new Smf110Record(record);

                Map<String, FileData> applidFiles = 
                        applids.computeIfAbsent(r110.stProductSection().smfstprn(),
                        files -> new HashMap<String, FileData>());

                for (FileControlStatistics fileStats : r110.fileControlStatistics()) 
                {
                    String entryName = fileStats.a17fnam();
                    applidFiles.computeIfAbsent(entryName, 
                            x -> new FileData(entryName)).add(fileStats);
                }
            }
        }
        writeReport(applids);
    }

    private static void writeReport(Map<String, Map<String, FileData>> applidFiles) 
    {

        applidFiles.entrySet().stream()
            .filter(applid -> !applid.getValue().isEmpty())
            .sorted((a, b) -> a.getKey().compareTo(b.getKey()))
            .forEachOrdered(applid -> 
            {
                // Headings
                System.out.format("%n%-8s", applid.getKey());

                System.out.format("%n%-8s %12s %12s %12s %12s %12s %12s %12s %12s%n%n", 
                        "ID", 
                        "Gets", 
                        "Get Upd",
                        "Browse", 
                        "Adds", 
                        "Updates", 
                        "Deletes", 
                        "Data EXCP", 
                        "Index EXCP");

                applid.getValue().entrySet().stream()
                    .map(x -> x.getValue())
                    .sorted(comparing(FileData::getTotalExcps)
                            .reversed())
                    .forEachOrdered(fileInfo -> 
                    {
                        // write detail line
                        System.out.format("%-8s %12d %12d %12d %12d %12d %12d %12d %12d%n", 
                                fileInfo.getId(),
                                fileInfo.getGets(), 
                                fileInfo.getGetUpd(), 
                                fileInfo.getBrowse(),
                                fileInfo.getAdds(), 
                                fileInfo.getUpdates(), 
                                fileInfo.getDeletes(),
                                fileInfo.getDataExcps(), 
                                fileInfo.getIndexExcps());
                    });
                });

    }

    private static class FileData 
    {
        public FileData(String fileId)
        {
            this.id = fileId;
        }

        public void add(FileControlStatistics fileStatistics) 
        {
            gets += fileStatistics.a17dsrd();
            getupd += fileStatistics.a17dsgu();
            browse += fileStatistics.a17dsbr();
            add = fileStatistics.a17dswra();
            update = fileStatistics.a17dswru();
            delete = fileStatistics.a17dsdel();
            dataexcp = fileStatistics.a17dsxcp();
            indexexcp = fileStatistics.a17dsixp();
            totalexcp += fileStatistics.a17dsxcp() 
                    + fileStatistics.a17dsixp();
        }

        public String getId() 
        {
            return id;
        }

        public long getGets() 
        {
            return gets;
        }

        public long getGetUpd() 
        {
            return getupd;
        }

        public long getBrowse() 
        {
            return browse;
        }

        public long getAdds() 
        {
            return add;
        }

        public long getUpdates() 
        {
            return update;
        }

        public long getDeletes() 
        {
            return delete;
        }

        public long getDataExcps() 
        {
            return dataexcp;
        }

        public long getIndexExcps() 
        {
            return indexexcp;
        }

        public long getTotalExcps() 
        {
            return totalexcp;
        }

        private String id;
        private long gets = 0;
        private long getupd = 0;
        private long browse = 0;
        private long add = 0;
        private long update = 0;
        private long delete = 0;
        private long dataexcp = 0;
        private long indexexcp = 0;
        private long totalexcp = 0;
    }
}

Complete CICS Transaction Monitoring reporting sample

import java.io.*;
import java.time.*;
import java.util.*;
import static java.util.Collections.reverseOrder;
import static java.util.Comparator.comparing;

import com.blackhillsoftware.smf.*;
import com.blackhillsoftware.smf.cics.*;
import com.blackhillsoftware.smf.cics.monitoring.*;
import com.blackhillsoftware.smf.cics.monitoring.fields.*;

public class CicsTransactionSummary 
{

    public static void main(String[] args) throws IOException 
    {
        Map<String, Map<String, TransactionData>> applids = 
                new HashMap<String, Map<String, TransactionData>>();

        ByteStringField transaction = ByteStringField.define("DFHTASK", "C001");

        int noDictionary = 0;

        try (SmfRecordReader reader = 
                args.length == 0 ? 
                SmfRecordReader.fromDD("INPUT") :
                SmfRecordReader.fromStream(new FileInputStream(args[0]))) 
        {     
            reader.include(110, Smf110Record.SMFMNSTY);
            for (SmfRecord record : reader) 
            {
                Smf110Record r110 = new Smf110Record(record);

                if (r110.haveDictionary()) 
                {
                    Map<String, TransactionData> applidTransactions = 
                        applids.computeIfAbsent(
                            r110.mnProductSection().smfmnprn(), 
                            transactions -> new HashMap<String, TransactionData>());

                    for (PerformanceRecord mn : r110.performanceRecords()) 
                    {
                        String txName = mn.getField(transaction);
                        applidTransactions.computeIfAbsent(
                                txName, 
                                x -> new TransactionData(txName)).add(mn);
                    }
                } else 
                {
                    noDictionary++;
                }
            }
        }

        writeReport(applids);

        if (noDictionary > 0) 
        {
            System.out.format(
                    "%n%nSkipped %s records because no applicable dictionary was found.", 
                    noDictionary);
        }

    }

    private static void writeReport(Map<String, Map<String, TransactionData>> transactions) 
    {
        transactions.entrySet().stream()
            .sorted((a, b) -> a.getKey().compareTo(b.getKey()))
            .forEachOrdered(applid -> 
            {
                // Headings
                System.out.format("%n%-8s", applid.getKey());

                System.out.format("%n%-4s %15s %15s %15s %15s %15s %15s %15s %15s %15s%n%n", 
                        "Name", 
                        "Count", 
                        "Elapsed",
                        "Avg Elapsed", 
                        "CPU", 
                        "Avg CPU", 
                        "Dispatch", 
                        "Avg Disp.", 
                        "Disp Wait", ""
                                + "Avg Disp Wait");

                applid.getValue().entrySet().stream()
                    .map(x -> x.getValue())
                    .sorted(comparing(TransactionData::getCpu, reverseOrder())
                            .thenComparing(TransactionData::getCount, reverseOrder()))
                    .forEachOrdered(txInfo -> 
                    {
                        // write detail line
                        System.out.format("%-4s %15d %15f %15f %15f %15f %15f %15f %15f %15f%n", 
                                txInfo.getName(),
                                txInfo.getCount(), 
                                txInfo.getElapsed(), 
                                txInfo.getAvgElapsed(), 
                                txInfo.getCpu(),
                                txInfo.getAvgCpu(), 
                                txInfo.getDispatch(), 
                                txInfo.getAvgDispatch(),
                                txInfo.getDispatchWait(), 
                                txInfo.getAvgDispatchWait());

                    });
            });

    }

    private static class TransactionData 
    {
        public TransactionData(String name) 
        {
            this.name = name;
        }

        public void add(PerformanceRecord perfdata) 
        {
            count++;
            elapsed += Utils.ToSeconds(
                    Duration.between(perfdata.getField(start), perfdata.getField(stop)));
            dispatch += perfdata.getFieldTimerSeconds(dispatchField);
            dispatchWait += perfdata.getFieldTimerSeconds(dispatchWaitField);
            cpu += perfdata.getFieldTimerSeconds(cpuField);
        }

        public String getName() 
        {
            return name;
        }

        public int getCount() 
        {
            return count;
        }

        public double getElapsed() 
        {
            return elapsed;
        }

        public double getDispatch() 
        {
            return dispatch;
        }

        public double getDispatchWait() 
        {
            return dispatchWait;
        }

        public double getCpu() 
        {
            return cpu;
        }

        public Double getAvgElapsed() 
        {
            return count != 0 ? elapsed / count : null;
        }

        public Double getAvgDispatch() 
        {
            return count != 0 ? dispatch / count : null;
        }

        public Double getAvgDispatchWait() 
        {
            return count != 0 ? dispatchWait / count : null;
        }

        public Double getAvgCpu() 
        {
            return count != 0 ? cpu / count : null;
        }

        static TimestampField start = TimestampField.define("DFHCICS", "T005");
        static TimestampField stop = TimestampField.define("DFHCICS", "T006");
        static ClockField dispatchField = ClockField.define("DFHTASK", "S007");
        static ClockField dispatchWaitField = ClockField.define("DFHTASK", "S102");
        static ClockField cpuField = ClockField.define("DFHTASK", "S008");

        private String name;
        private int count = 0;
        private double elapsed = 0;
        private double dispatch = 0;
        private double dispatchWait = 0;
        private double cpu = 0;
    }
}

Filed Under: EasySMF News, Java, Java SMF

Java SMF API Version 1.3.0

September 7, 2016 by Andrew

Verison 1.3.0 of the EasySMF:JE Java API is now available.

Major Changes

EZSMFKEY DD

The program will now look for the key in the EZSMFKEY DD statement as well as the EASYSMFKEY environment variable. The EASYSMFKEY environment variable is no longer required for z/OS batch jobs.

API messages

API messages are now issued via the SLF4J logging framework instead of to STDOUT. This allows the destination to be customized, and avoids mixing them with program output written to STDOUT. The default destination for messages is STDERR.

SMF Section Constructors marked private

The SMF section class constructors were always documented as “intended for internal use only”. Now they are private. This allows more flexibilty in construction, which in turn allows more transparent handling of changes to sections.

Sample JCL changes

The sample JCL procedures have been modified to eliminate the extra step to concatenate user additions and changes to the environment variables. This is now done in the main program execution step.

QWHCTOKN

The DB2 Accounting Token QWHCTOKN in the Qwhc section now has its own class. It was initially unclear what type of data appeared in the QWHCTOKN. While it was frequently readable characters, it also often contained binary data. Since the purpose is presumably to match data by token, a string representation was no good because binary data might result in different tokens converting to the same String. So initially a BigInteger was chosen as a class that could uniquely represent 22 bytes of data, and be compared and used as a key etc.

However, since there is character data there, people might reasonably want to see it as a string. BigInteger doesn’t make for a simple EBCDIC to UTF8 conversion. So I created a specific class for the QWHCTOKN. Now they can be reliably compared and used as a key, and there is a custom toString() implementation that does the EBCDIC to UTF8 conversion on the character portion.

Reminder: the DB2 SMF record support is still in experimental status so there is a greater likelihood of changes to the API. In particular, while the status is experimental, improving the API is likely to take precedence over mainitaining compatibility with existing programs.

Filed Under: Java SMF Tagged With: DB2, Java

New version of Java SMF API available

January 30, 2015 by Andrew

There is a new version of the Java SMF API available. The changes are:

  • Add SMF record types
    • 0 – IPL
    • 2 – Dump Header
    • 3 – Dump Trailer
    • 7 – Data Lost
    • 8 – I/O Configuration

Download it from: http://www.blackhillsoftware.com/downloads/javasmf/java-smf.zip

Record type 6 should be coming soon.

Filed Under: Java SMF

Processing z/OS SMF data using Java

November 4, 2014 by Andrew

Black Hill Software is currently developing an API for processing SMF data using Java.

The API builds on experience gained developing EasySMF to create a powerful and easy to use API for working with SMF data.

The API will run on z/OS and other platforms such as Linux and Windows. The API is 100% Java which makes it zAAP eligible on z/OS.

[Read more…]

Filed Under: Java SMF

EasySMF:JE Installation

November 4, 2014 by Andrew

[Read more…]

Filed Under: Java SMF

  • 1
  • 2
  • Next Page »

30 Day Trial

EasySMF and EasySMF:JE are available for a free 30 day trial. Download now and start using them immediately.
30 Day Trial

Information

EasySMF:JE Java API for SMF Quickstart

EasySMF:JE Sample 1 : SMF Records by type and subtype

Systems Programmer Friendly Java

Sending Email from z/OS using Java

Sign up for EasySMF News

Stay up to date. Enter your email address to receive updates about EasySMF.
unsubscribe from list

Black Hill Software

Suite 10b, 28 University Drive, Mt Helen, VIC 3350, Australia
PO Box 2214, Bakery Hill, VIC 3354, Australia
+61 3 5331 8201
+1 (310) 634 9882
info@blackhillsoftware.com

News

  • Finding UID 0 work on z/OS using SMF Data
  • Apache Log4j CVE-2021-44228 Information
  • Java vs C++ : Drag Racing on z/OS

Twitter

My Tweets

Copyright © 2023 · Enterprise Pro Theme on Genesis Framework · WordPress · Log in