PRODUCT SPOTLIGHT: tcVISION v6 Overview and Updates

tcVISONv6

Several exciting new features and updates are now in tcVISION v6, including new output targets Adabas LUW, DB2 BLU, EXASOL, Hadoop,and MongoDB. Additionally, new input sources include z/OS VSAM Logstream (CICS and Coupling Facility / Shared VSAM),z/OS VSAM Batch Extension, z/OS DBMS to Logstream, CA IDMS v17, CA Datacom CDC, IMS Active Log and SMF data.

Another feature recently announced is the tcVISION “Direct Loader” for BULK_LOAD processing. The function does not require output to a sequential file, and the loader utility for the target DBMS is called via API with data passed directly. Direct loader supports PostGreSQL, Microsoft SQL Server and DB2 LUW / DB2 BLU. The advantage of using the Direct Loader is the elimination of disk access in writing and reading the sequential loader data file. File output is still supported (e.g., where loader data is to be distributed to other machines).

Finally, as mentioned in a previous Treehouse Blog, with tcVISION v6 comes the newly enhanced web statistics functionality and web server. Any standard web browser can access this server (Internet Explorer, Firefox, Opera, Chrome, Safari, etc.)

This valuable feature enables users to view data from the tcVISION Manager Monitor, and statistical and operational information from the tcVISION Manager network.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

3270 Application Modernization Using OpenLegacy, Part 3: Generating the Application Model

by Frank Griffin, Senior Software Developer for Treehouse Software 

This is the third in a series of blog posts concerning the Treehouse partnership with OpenLegacy. As discussed previously, OpenLegacy is a way to wrap legacy mainframe applications for presentation to Java, Web, and Mobile clients.

In the last post, we covered the process of using OpenLegacy run-emulation to monitor 3270 traffic as you interacted with CICS to use the CEMT transaction to obtain information about one of the standard CICS transaction programs. This resulted in a trail file – an XML document which described the interaction in detail. This time, we’ll go through the process of generating the application model, which is a set of Java classes that represent each 3270 screen and provide methods for reading and writing the fields on the screen. These classes will be used later by the service parts of the application which interact with the 3270 screens on behalf of the web client.

If you still have the OpenLegacy IDE open from when you did run-emulation, you’ll need to refresh the CICSWeb project to make it aware of the new trail file, which has been created in src/test/resources/trails. Right-click on the project and select “Refresh”:

OpenLegacy_Screens03_01

OpenLegacy will detect that a new trail has been created, and will ask if you want it analyzed.

OpenLegacy_Screens03_02

We’re going to select “No” at this point, in order to go through the model generation process in detail. This will return you to the usual IDE view of the project. Expand the project and drill down through src, test, resources, and trails to expose the newly-created trail file:

OpenLegacy_Screens03_03

and double-click on the trail file to open it using the Trail Editor:

You’ll notice that the editor opens in Snapshot mode by default, showing the first screen in the trail file. If you want to see the actual XML source, you can click the Source tab at the bottom left of the editor window and toggle back and forth.

In order to see the index of screen thumbnails, maximize the editor window. Then right-click on Screen 1, and select Generate Screen Model:

OpenLegacy_Screens03_04

OpenLegacy_Screens03_05

Selecting Generate Screen Model will get you a prompt:

OpenLegacy_Screens03_06

Uncheck Generate JUnit Test, and click OK. This will open the model generation dialog for Screen 1:

OpenLegacy_Screens03_07

At the top, the Fields table should contain one entry for each field on the screen which can contain dynamic data, that is data that can change from client request to client request. These fields may be unprotected (designating input fields to be populated with input data from the client) or protected (designating output fields to be populated with output data to be sent to the client). In our case, there is only one: the command field on line 24. OpenLegacy will have chosen a default name for the field based on assumed usage or text appearing to the left which might be a label, and if the name has multiple segments, e. g. New Password, the name will appear in camel-case (newPassword) since the name will be used as a field name and as part of method names in the generated Java code.

The Identifiers table to the right will contain OpenLegacy’s best guess as to the fields on the screen which uniquely identify the screen. OpenLegacy can’t really know which protected fields on the screen represent static data that can be used to identify the screen and dynamic fields which may change, so this is one of the areas where you may have to dive in and correct the default guesses. Your screen model must include at least one Identifier; otherwise, OpenLegacy can’t identify a screen. For screen matching, identifiers are additive, so you can specify as many as needed to identify the screen uniquely within the trail file. Conversely, if OpenLegacy has tagged a field as an Identifier but the field can change from use to use, you’ll need to delete it as an Identifier by selecting it and clicking on the X at the top right of the table.

Note that OpenLegacy has chosen a name for the screen (ApplicationDeveloperSystem) based on text it found on the screen as an Identifier. This will be used as part of the Java class name for this screen, so it will need to be unique among the screens in the trail file.

In our case, the guesses for Screen 1 are adequate, so we’ll leave everything as is. Scroll the window down, if necessary, to expose the Cancel/OK buttons:

OpenLegacy_Screens03_08

Once you’ve finished, click OK at the bottom of the window, which will open a properties dialog:

OpenLegacy_Screens03_09

Much of the data on display here is the same stuff you saw on the previous screen, e. g., Name, Display Name, but other data has been taken from the trail file, such as the screen size. If you select the Fields and Identifiers tabs at the bottom, you’ll see the same Fields and Identifiers as well, but with additional trail data. Select the Fields tab, and note that the position of the field on the screen is recorded, as is the fact that the field is unprotected (“editable”):

OpenLegacy_Screens03_10

Going back to the General tab, select Screen Navigation rather than Screen Entity:

OpenLegacy_Screens03_11

Most of these fields are beyond the scope of this discussion, but note particularly the Accessed From field. I mentioned before that OpenLegacy creates a Java class for each screen, with methods that can be called to interact with fields on the screen. For application simplicity, users of these methods do not have to navigate to the desired screen manually. You can simply call the method associated with the field you want, and OpenLegacy will figure out what it has to do in the way of 3270 screen navigation to get there. It does this by using Accessed From and other navigation data here, and correlates this information across all of the screens in the trail. In this case, Accessed From is blank because this screen is obtained simply by connecting to the mainframe.

Lastly, notice the tab in the Screen Editor for this display: it says “ApplicationDeveloperSystem.java”, because you are looking at a graphical rendition not of the trail file, but of actual Java code which OpenLegacy has generated. Remember that Eclipse editors are capable of showing the same file in multiple ways. To illustrate this, select the bottom notebook tab marked “Source (read only)”:

OpenLegacy_Screens03_12

The tokens starting with “@” are called “annotations” in Java, and Java provides language support for reading annotation data in compiled class code without having to decode the class file yourself. Note that all of the data displayed in the other notebook tabs is represented here as attribute fields in one annotation or another. OpenLegacy generated this code from the trail file data for this screen and compiled it when we clicked OK. Then, the Screen Editor filled in the fields on the notebook tabs by retrieving the annotation data from the compiled code. Once we clicked OK, we were done with this screen’s section of the trail file; the reference location for all of the screen’s data from now on is the generated Java code. If we make any changes to this data using the Screen Editor, saving the changes will result in regenerating the Java code with new values in the annotations and recompiling it.

The tab says “(read-only)” because any manual edits you make here won’t be saved. During Generate Model, OpenLegacy is dealing only with automatic code generation. Manual modifications and additions, if needed, are done later using the standard editor.

You’ll note that in the Trail Editor, screens are marked either “(IN)” or “(OUT)” depending on whether they came in from the mainframe or went out to the mainframe from the emulator. Only (IN) screens have a Generate Screen Model option in their right-click menus.

Moving on, let’s close ApplicationDeveloperSystem.java, right-click on Screen 3 back in the Trail Editor, and select Generate Screen Model:

OpenLegacy_Screens03_13

We see considerably more activity here than on Screen 1. OpenLegacy has assigned this screen the name SignonToCics based on the top banner line, and because of the labels and the presence of non-display fields, it has selected a Screen Type of “Login Entity” rather than “General”. As before, various fields and identifiers have been pre-selected for you, and based on labels and field attributes “user” has been given the status of a userid field and “password” has been designated a password field. This means that these fields will be filled in using authentication data supplied by the client. Note that the community version of OpenLegacy supports only userid/password authentication, so no special status has been given to the “groupid” or “newPassword” fields since client protocols, e. g. HTTP, don’t make any provision for them.

Once again, the default choices for Fields and Identifiers look suitable, so we’ll leave them as is. Scroll down and click OK to generate the code and open the Screen Editor, then select the “Source” tab:

OpenLegacy_Screens03_14

This looks pretty much like before, but as we have more fields, we have more @ScreenField annotations.

However, we have some customization to do here as regards navigation. Select the “General” tab again and “Screen Navigation”, where we will need to fill in Accessed From to indicate that we are accessed from ApplicationDeveloperSystem and we will need to indicate that we are accessed from that screen by entering “l cicsts51” in the “command” field of the previous screen.

It’s worth noting that if we had chosen to allow OpenLegacy to do Generate Model on the entire trail file at once, automatically moving from one screen to the next, it would have filled these fields in for us. We need to do it manually here only because we are processing each screen standalone.

To fill in Accessed From, we don’t just type the previous screen name; OpenLegacy needs to know the name of the Java class that represents the previous screen, not just the screen name. Click on the “Browse” button next to Accessed From, and you’ll get a standard Eclipse class browser with nothing displayed. Type “Application” in the top input field, and Eclipse will display all of the known classes beginning with that string:

OpenLegacy_Screens03_15

Double-click on ApplicationDeveloperSystem in the list, and that will be filled in as Accessed From.

Now, it may seem counter-intuitive to be dealing with “l cicsts51” in Screen 3, since in the trail file it was entered on Screen 2. However, it shows up here because the Screen Navigation tab for Screen 3 describes what needs to be done to get to Screen 3, not what happens once you’re there, so this tab is where we stipulate that the fixed string “l cicsts51” has to be entered in “command”. We do this by scrolling down to Assigned Fields and clicking on Add, which will add a blank row to the Assigned Fields table. Then just type a Name of “command” and a Value of “l cicsts51”:

OpenLegacy_Screens03_16

Note that as a result of these activities, the Screen Editor tab for SignonToCics.java now has an asterisk indicating that it has been modified. Change to the Source tab, and you’ll see that it is unchanged from before; then save the changes by pressing CTRL-S, and watch how the source changes, adding a @ScreenNavigation annotation with your changes:

OpenLegacy_Screens03_17

Close the SignonToCics.java Screen Editor tab and return to the Trail Editor. Now proceed to Screen 5, right-click and select Generate Screen Model. Surprise – nothing happens ! This is because Screen 5 has no Fields, and therefore OpenLegacy has nothing with which to work. In this case, we’ll need to define the screen ourselves, using right-click and New Screen Model, which will give us a display with no Fields or Identifiers filled in:

OpenLegacy_Screens03_18

We’ll assign a Screen Name of SignonIsComplete. We need to have at least one Field (whether it will see any use or not), and the unprotected field at the right end of line 24 will do nicely. To add a Field, you must select/trace it on the screen display by left-click and dragging the mouse, and then click on one of the Field Type icons at the top of the Fields table. We’ll select the area and click on the S icon (String):

OpenLegacy_Screens03_19

We’ll also need an Identifier, and the first part of the message will suffice. To add an Identifier, trace it out on the screen and click on the + icon at the top of the Identifier table:

OpenLegacy_Screens03_20

Scroll down and click OK; no customization is needed, so you can just close SignonIsComplete.java. Note that we did not fill in Accessed From or any assignable fields in Screen Navigation, because all data entered on the SignonToCics screen is dynamic (userid and password), and the AID key which gets us here is the default (ENTER).

Screen 7 is another challenge. It results from hitting CLEAR to Screens 5/6, and has neither Fields nor Identifiers that OpenLegacy can recognize, so Generate Screen Model once again has nothing to work with, and we must use New Screen Model, which we’ll name “Empty”. Adding a Field is just like before; we’ll use the first line of the blank screen, and rename the default name of “newField1” to “command”. However, to provide the Identifier, we must use a trick since there is no identifying text present. Since this screen can be differentiated from the others by the fact that it begins with an unprotected field in row 1 column 1, we can define a zero-length identifier there (start (1,1) end (1,1)) and stipulate that the field must be editable. Adding the Field:

OpenLegacy_Screens03_21

Adding the identifier is not so straightforward, as there is no way to trace a zero-length field. We’ll do it after the fact using the Screen Editor.

Scroll down and click OK to open the Screen Editor, and select the Identifiers tab at the bottom to display the empty Identifiers table. Click “Add” to add a blank Identifier row to the table and change the row/column from 0/0 to 1/1. Type “true” in the Value column (this is the value of the attribute, not the Identifier text). Now click on the Attributes column, which will give you a dropdown list of attribute types, and select “Editable”:

OpenLegacy_Screens03_22

There’s one more bit of configuration needed here. The navigation from Screens 5/6 to Screen 7 doesn’t involve any assigned field, but it does involve hitting the CLEAR key (ESCAPE in the browser emulator), which is not the default AID (ENTER) for screen navigation. Thus, we need to go to Screen Navigation, set Accessed From to SignonIsComplete, and Terminal Action to ESCAPE using the dropdown list:

OpenLegacy_Screens03_23

One screen left – Screen 9. Close Empty.java, right-click on Screen 9, and select Generate Screen Model:

OpenLegacy_Screens03_24

While we could return all of this information to the client, for simplicity we’re just going to return the program length and use count. The Fields and Identifiers on this screen are not standard OpenLegacy Fields and Identifiers, because the “command” field must be created dynamically using the program name supplied by the client, and the program length and use count are both parts of a larger 3270 field, e. g. “Leng(0000000624)” and not just “0000000624”. But no matter, this will give us the chance to expand our experience with OpenLegacy configuration.

We don’t have to have a Field, but we’ll leave the “command” field selected by OpenLegacy, even though we have no use for it. Note that it’s an example of a Field which is protected, but whose contents will change dynamically depending on the program name supplied by the client. In fact, none of the other chosen Fields are needed either, so we’ll delete them by highlighting them one by one and clicking on the X at the top right of the Fields table:

OpenLegacy_Screens03_25

Now, we come to the Identifiers. None of them are suitable, because “Pro” hardly uniquely identifies the screen, and the other two will change dynamically depending on the program name supplied. So, we’ll delete them all just as we did the Fields – select and click X – and add a unique Identifier of the fixed portion of the CEMT command by tracing it and clicking + at the top left of the Identifiers table:

OpenLegacy_Screens03_26

Now we need to add two additional Fields for the data we wish to extract. The values for the length and use count are integers, so in each case trace the integer portion of the field (within the parentheses) and click on the I (Integer) icon on top of the Fields table to create the Fields. Select the fieldnames and overtype the defaults with “length” and “useCount” respectively:

OpenLegacy_Screens03_27

Scroll down and click OK to generate the model and open the Screen Editor. No additional configuration is needed here, so it can just be closed.

We’re done generating the Screen Models. But wait, you say, don’t we have to create the CEMT command from the program name ? And doesn’t Screen 9 need an Accessed From of Empty ?

The answer is no in both cases. The job of the Screen Model is just to provide program getter/setter access to screen fields of interest. The classes in the Model will be used later on by the other part of the generated application, called the Service, to provide dynamic data to the 3270 emulator and retrieve output field values.

And Accessed From is only needed if static data (like “l cicsts51”) needs to be entered or if the transition AID to the screen differs from the default of ENTER. In our case, the CEMT command to be entered on Empty will be built dynamically by the Service and the Empty class setter for the “command” field (the “command” on Empty, not the one on ResultsOvertypeToModify) will be used to inject the command into the Empty screen. The transition AID for the Empty to ResultsOvertypeToModify transition is the default of ENTER, so neither condition applies.

That wraps up this post. OpenLegacy has taken our trail file, and with a little GUI-only tweaking from us has generated a complete set of Java classes with simple-to-use methods like setCommand( “CEMT INQUIRE PROG(“ + programname + “)”) and getLength() which will drive the legacy application through the 3270 emulator to supply our client’s input data and retrieve his output data. If you’re so inclined, you can see this Java code by expanding the src/main/java directory tree under the project.

In the next post, I’ll cover generating the Service portion of the application and making the few needed code modifications.

3270 Application Modernization Using OpenLegacy, Part 2: Creating the Trail File

by Frank Griffin, Senior Software Developer for Treehouse Software 

This is the second in a series of blog posts concerning the Treehouse partnership with OpenLegacy (See Part 1 here). As discussed last time, OpenLegacy is a way to wrap legacy mainframe applications for presentation to Java, Web, and Mobile clients.

We’ve already covered the conceptual workings of wrapping 3270 applications in general, so this time we’ll examine the first steps of wrapping an actual 3270 application in detail.

If you haven’t already downloaded OpenLegacy, you’ll need to do so in order to follow along in this Proof of Concept. Go to http://www.openlegacy.com and select Resources → Download. You’ll need to provide your name and an email address, which will get you a userid and password for the download site at ftp://ftp.openlegacy.org .

CICS is one of the bulwarks of legacy 3270 applications. These days, it also supports a variety of other communications protocols, but for 3270 it acts very much like a Windows/DOS/Unix command line. You type a transaction name followed by optional arguments, and hit ENTER. CICS looks up the transaction name in a table to find the application program associated with it, and runs that program. The program writes to the terminal, either in line mode or using formatted 3270 screens, and may request further input from the user and provide additional terminal output.

CICS provides a number of utility transaction names with the product. We’ll be looking at two of them today: CESN (sign on to CICS as a known user), and CEMT (general multi-purpose transaction for querying and controlling aspects of CICS). We’ll construct a demo application by connecting to CICS, signing on with the CESN transaction and issuing the transaction CEMT INQUIRE PROG(xxxxxxxx) to obtain information about a specific CICS transaction program. The input to the application will be a userid, password, and application program name. The output will be some of the data associated with that program.

Open the OpenLegacy IDE, and choose New → OpenLegacy Project. Select “Screens” as the backend solution type, “Integration Web” for the frontend solution type, “Mainframe” for the host type, and supply your host’s name or IP address:

OpenLegacy_Screens01

OpenLegacy_Screens02

Now, we need to run an emulation session to “teach” OpenLegacy how to navigate. Right-click on the project and select OpenLegacy → run-emulation. This will set several things in motion. It will start an internal web server within the IDE on port 1512 and launch an instance of your default browser with a URL of http://localhost:1512. When the browser connects to the web server, it will execute a servlet that will use the open-source s3270 scripting emulator to connect to your mainframe and report the 3270 datastream to the servlet, which will convert the screen to HTML and send it to the browser.

The result is that your browser instance will appear to be a 3270 screen, and you’ll be able to interact with it as if it were a real terminal or terminal emulator. As you do, the servlet is sitting in the middle, logging all terminal activity between you and the mainframe. On our mainframe, the first screen you’ll see is the standard VTAM Logon screen:

OpenLegacy_Screens03

We logon to CICS (the VTAM logon screen and the command you use may differ at your site), and depending on how CICS is configured, you may need to type “CESN” explicitly, or CESN may be started for you automatically:

OpenLegacy_Screens04

Fill in a valid Userid and Password, and hit ENTER:

OpenLegacy_Screens05

Clear the screen (ESC key in the browser emulator), as CICS transactions have to be invoked from a clear screen, type “CEMT INQUIRE PROG(DFHZNEP)” (DFHZNEP is a dummy network error program supplied with CICS), and hit ENTER:

OpenLegacy_Screens06

OpenLegacy_Screens07

The CEMT transaction has displayed all sorts of information about the DFHZNEP program, which is what we wanted.

At this point, we could continue with 3270 navigation to exit and signoff from CICS, but for simplicity in this example, we’ll just cut off the emulation here. This will cause OpenLegacy to disconnect the TN3270 session, which will cut us loose from CICS; if this were TSO, we’d have to follow through or risk leaving disconnected TSO sessions active.

Note that depending on the size of your browser window you may have to scroll down to make the “Logoff | Flip” links visible. Now click on “Logoff”, which will disconnect the terminal, shut down the emulator, and create the trail file which describes everything you’ve done:

OpenLegacy_Screens08

The trail file is an XML document which describes in detail your interactions with the mainframe. It will be the input to the next phase of generating the application, which will be the subject of the next post.

Can OpenLegacy help connect PHP with DB2 on the mainframe?

Recently, a poster to a mainframe technical discussion forum asked the question: How can we connect PHP (a popular server-side scripting language designed for web development) with DB2 on the mainframe? Treehouse Senior Software Developer Frank Griffin replied, describing how OpenLegacy could be the answer.  

You have some options here. If all you want is access to the raw DB2 data, JDBC or ODBC access will work fine for you, although you will have to write either C ODBC or Java JDBC code that can be called from PHP to do the deed.

If you already have mainframe code accessible via the network (CICS/IMSDC/TSO via TN3270, CICS via TCP/IP) that accesses your data and adds business logic to the mix, you can use the FOSS* OpenLegacy project to mate this logic to your PHP app.

The simplest approach involves using OL to navigate through the green screens of (CICS, IMS/DC, TSO, or whatever) to get to the data you want. This is done via the OSS** s3270 scripting 3270 emulator which creates an XML “trail” file which documents the interactions between yourself and the legacy application over the TN3270 connection. Once you’ve navigated through the series of screens that exposes the data you want to find, you trigger a portion of OpenLegacy which analyzes those screens, identifies logon sequences and unprotected fields on those screens which are linked to client-supplied input data, and generates Java classes which can at some later time use the “trail” file to re-drive the emulator to access the legacy application using client-supplied values for some of the input in order to obtain transaction output associated with those input values.

If your mainframe apps are better modularized, i. e. if you have separated the business logic from the display logic, and the business logic can be invoked through a CICS COBOL program which is designed to obtain its input and provide its output via a CICS COMMAREA, OL can analyze the source code and generate Java classes that invoke those programs directly without screen-scraping.

And, if what you really want is JDBC, OL can generate a series of Java classes that do all of the JDBC work for you and provide you with methods that simply provide the legacy data with no hint as to where it came from.

OL can also layer additional access software. Once the fundamental Java classes that access the legacy data are in place, OL can generate Java apps that use those classes on behalf of clients using all sorts of modern APIs, including SOA/SOAP, REST/JSON, and Mobile. All of this happens with the push of a button.

All of this is FOSS. You can download the OL code and start using it immediately, and it can do all that I’ve described, out-of-the-box. OL makes their money from selling an Enterprise Edition that includes support and some security and management pretties.

If the only access path from PHP to Java is direct invocation of Java code, you’d have to write a Java stub to interact with the OL classes, but this s going to be a *lot* simpler than trying to write JDBC or ODBC applications on your own.

*FOSS = Free Open-Source Software

**OSS = Open-Source Software


__OpenLegacy_Logo

OpenLegacy is the world’s first and only light-weight, non-intrusive solution for automated legacy modernization and enterprise application integration. With its standards-based, open-source platform, OpenLegacy enables enterprises to rapidly extend legacy systems to mobile, web and cloud applications; delivering risk-free, high-impact results that solve immediate business needs.

__OpenLegacy_High_Level

OpenLegacy‘s standard tools rapidly extract the services and information from within legacy systems into an editable format that puts the power of integration into the enterprises’ hands without the expensive handcuffs of vendor lock in. Once a business process is exposed — which can be done in minutes — the output can automatically be transformed into stand-alone mobile, web, and cloud applications; and connected with other solutions. Most importantly, no changes are required to the legacy system in order for OpenLegacy to work — the process is risk-free.

Contact Treehouse today for more information!