3270 Application Modernization Using OpenLegacy, Part 2: Creating the Trail File

by Frank Griffin, Senior Software Developer for Treehouse Software 

This is the second in a series of blog posts concerning the Treehouse partnership with OpenLegacy (See Part 1 here). As discussed last time, OpenLegacy is a way to wrap legacy mainframe applications for presentation to Java, Web, and Mobile clients.

We’ve already covered the conceptual workings of wrapping 3270 applications in general, so this time we’ll examine the first steps of wrapping an actual 3270 application in detail.

If you haven’t already downloaded OpenLegacy, you’ll need to do so in order to follow along in this Proof of Concept. Go to http://www.openlegacy.com and select Resources → Download. You’ll need to provide your name and an email address, which will get you a userid and password for the download site at ftp://ftp.openlegacy.org .

CICS is one of the bulwarks of legacy 3270 applications. These days, it also supports a variety of other communications protocols, but for 3270 it acts very much like a Windows/DOS/Unix command line. You type a transaction name followed by optional arguments, and hit ENTER. CICS looks up the transaction name in a table to find the application program associated with it, and runs that program. The program writes to the terminal, either in line mode or using formatted 3270 screens, and may request further input from the user and provide additional terminal output.

CICS provides a number of utility transaction names with the product. We’ll be looking at two of them today: CESN (sign on to CICS as a known user), and CEMT (general multi-purpose transaction for querying and controlling aspects of CICS). We’ll construct a demo application by connecting to CICS, signing on with the CESN transaction and issuing the transaction CEMT INQUIRE PROG(xxxxxxxx) to obtain information about a specific CICS transaction program. The input to the application will be a userid, password, and application program name. The output will be some of the data associated with that program.

Open the OpenLegacy IDE, and choose New → OpenLegacy Project. Select “Screens” as the backend solution type, “Integration Web” for the frontend solution type, “Mainframe” for the host type, and supply your host’s name or IP address:

OpenLegacy_Screens01

OpenLegacy_Screens02

Now, we need to run an emulation session to “teach” OpenLegacy how to navigate. Right-click on the project and select OpenLegacy → run-emulation. This will set several things in motion. It will start an internal web server within the IDE on port 1512 and launch an instance of your default browser with a URL of http://localhost:1512. When the browser connects to the web server, it will execute a servlet that will use the open-source s3270 scripting emulator to connect to your mainframe and report the 3270 datastream to the servlet, which will convert the screen to HTML and send it to the browser.

The result is that your browser instance will appear to be a 3270 screen, and you’ll be able to interact with it as if it were a real terminal or terminal emulator. As you do, the servlet is sitting in the middle, logging all terminal activity between you and the mainframe. On our mainframe, the first screen you’ll see is the standard VTAM Logon screen:

OpenLegacy_Screens03

We logon to CICS (the VTAM logon screen and the command you use may differ at your site), and depending on how CICS is configured, you may need to type “CESN” explicitly, or CESN may be started for you automatically:

OpenLegacy_Screens04

Fill in a valid Userid and Password, and hit ENTER:

OpenLegacy_Screens05

Clear the screen (ESC key in the browser emulator), as CICS transactions have to be invoked from a clear screen, type “CEMT INQUIRE PROG(DFHZNEP)” (DFHZNEP is a dummy network error program supplied with CICS), and hit ENTER:

OpenLegacy_Screens06

OpenLegacy_Screens07

The CEMT transaction has displayed all sorts of information about the DFHZNEP program, which is what we wanted.

At this point, we could continue with 3270 navigation to exit and signoff from CICS, but for simplicity in this example, we’ll just cut off the emulation here. This will cause OpenLegacy to disconnect the TN3270 session, which will cut us loose from CICS; if this were TSO, we’d have to follow through or risk leaving disconnected TSO sessions active.

Note that depending on the size of your browser window you may have to scroll down to make the “Logoff | Flip” links visible. Now click on “Logoff”, which will disconnect the terminal, shut down the emulator, and create the trail file which describes everything you’ve done:

OpenLegacy_Screens08

The trail file is an XML document which describes in detail your interactions with the mainframe. It will be the input to the next phase of generating the application, which will be the subject of the next post.

Treehouse Software Partners with OpenLegacy

Treehouse Software Partners with OpenLegacy

By Wayne Lashley, Chief Business Development Officer for Treehouse Software

Mainframe legacy systems, be they Software AG Adabas/Natural, CA IDMS/ADS, CA Datacom/Ideal or VSAM/COBOL, have several attributes in common:

  • They represent an enormous investment and body of proprietary business knowledge and process that are not easily replaced;
  • They are mission-critical, reliable and well-managed;
  • As originally conceived, they do not lend themselves readily to today’s common technology standards and practices: Java, REST, RPC, Web Services and mobile and Web apps — not to mention whatever is coming next.

The last point is a principal rationale for the rise of the legacy modernization industry in recent years. Various ways and means of “modernizing” legacy applications have emerged, and these are well-known and well-documented in the industry, and even here in the Treehouse Software blog and its predecessor, the Treetimes newsletter. After a few years of shakeout and consolidation, it would seem that there is little new under the sun in terms of legacy modernization practices and practitioners. But that’s not so.

It was back in 2013 when we first encountered OpenLegacy, a new entrant in the field and one with a novel approach. Nobody else in the modernization biz seems to be offering a standards-based, open-source toolkit/platform that can open up any IBM z or i legacy environment to provide Web and mobile interfaces, Web services and APIs—without migrating the legacy application or changing its code. With OpenLegacy, the value of the legacy environment can be fully leveraged in today’s technologies — and in whatever comes next.

We’ve been watching the company and its offerings evolve over the months, and Treehouse Senior Software Developer Frank Griffin participated in a technical evaluation and went through OpenLegacy training. We were all pretty impressed—so much so that we recently inked a partnering agreement with OpenLegacy to help market and deliver their solutions in North America. In the post below, Frank discusses one aspect of how OpenLegacy can be used to open up mainframe 3270 applications. Stay tuned for more posts from Frank as our OpenLegacy journey continues.


__OpenLegacy_Logo

Treehouse Software is now partnering with OpenLegacy to provide access to legacy IBM mainframe applications for a range of non-mainframe devices, including mobile phones and tablets as well as “heavier” clients like Service-Oriented Architecture (SOA) consumers.

Unlike many modernization approaches that require a commitment to migrate the legacy application in some way or at the very least require legacy application code changes, OpenLegacy adds more modern potential client populations to the existing application clients with no required changes to legacy code.

OpenLegacy is open-source, and is built using standards-based protocols and other open-source components wherever possible. The starter version is free to download, so you can get started with your testing immediately.

OpenLegacy can interact with legacy applications in several ways, but for simplicity in this initial post I’ll concentrate on just one: access to 3270-based applications.

We’ll assume that you have a 3270 application which can be made to display data required by a non-legacy client by navigating through one or more screens. The data need not all appear on a single screen, but can be spread over several screens. All you have to know is how to navigate through the application to get the data you want displayed.

OpenLegacy development starts with a developer environment which allows developers to describe:

(a) any input parameters needed to display the desired data

(b) the navigation process to arrive at a screen containing data to be captured

(c) the location on that screen of the data to be captured, as well as a description of that data

The OpenLegacy development suite runs as a plugin for the Eclipse Integrated Development Environment (IDE) or a self-contained IDE installation, which has several advantages. It allows OpenLegacy to piggyback on the rich GUI which Eclipse provides, and provides a familiar IDE environment for customer developers.

The OpenLegacy plugin opens several custom windows within Eclipse:

OpenLegacyScreen001

In the upper right, you can see a window with the current state of the 3270 emulator, which initializes at the standard initial logon screen for VTAM applications. In the upper left, you can see the directory structure which is automatically created for you when you use the OpenLegacy wizard to create a project. In project creation, you specify the target mainframe host, so OpenLegacy can open a TN3270 session to that host.

Navigation directions are entered by having OpenLegacy open a browser-based 3270 emulator which uses a captive copy of the s3270 emulator to interact with your mainframe system under OpenLegacy’s control. You use the browser-based emulator exactly as you would a 3270 terminal to log on, enter requests and input data, and do whatever a 3270 user would do to get to a screen of interest. However, because the 3270 emulator is running under OpenLegacy’s control, OpenLegacy is recording every keystroke needed for the navigation.

You can see the browser-based emulator in the following screenshot:

OpenLegacyScreen002

The browser application captures your keystrokes and feeds them to the 3270 emulator, also capturing the input and output from the emulator. These traces are referred to by OpenLegacy as “trails”, and are saved as XML files. When your project actually executes, this file is used instead of browser keystrokes to provide input to the emulator and scrape data from the emulator output.

When you’ve arrived at a screen containing data of interest, you switch from the browser window back to the OpenLegacy Eclipse GUI, and let OpenLegacy analyze the screen contents, identify the fields on the screen, and allow you to select those of interest and identify them with names and datatypes.

This navigation/selection cycle continues until you’ve identified all of the data you wish captured. At that point, you literally just push a button, and OpenLegacy will generate a complete rich web Java application which will accept whatever input is needed for the navigation through the legacy application, and then use the captive terminal emulator behind the scenes to log into the application, navigate to the screens of interest, and capture the fields of interest.

For Java developers (depending on what you’ve asked for), OpenLegacy also generates front-end web/Java code to use this application to accept input and return output using a number of mobile and web protocols, including SOA, HTTP GET/POST, or REST/JSON, making the legacy application immediately accessible to a wide range of modern devices.

Once compiled and exposed to the client community via a servlet container, these web/mobile applications accept requests in the specified input protocol, extract the input argument data, call the core Java code to drive the legacy application filling in fields and simulating keystrokes as needed, and collecting the desired output from screen fields as previously directed. The collected output is then formatted by the front-end application according to the requirements of the protocol involved, and returned as a response. Your device has no idea that it is interacting with a 3270 application, and your 3270 application has no idea that it is interacting with anything other than a 3270 terminal.

Oh, and did I mention that the Eclipse container includes a servlet container, and (since it was designed to allow developers to write/generate code and immediately compile and test it) a few additional mouse-clicks compile the generated code, create JARs and WARs and deploy them to that servlet container ? The result is that in as little as an hour from starting this process, you can be interfacing to your legacy application from your phone for testing. All you need is someone who knows how to navigate the legacy application and someone who knows how to navigate the OpenLegacy Eclipse GUI–two completely separate skill sets.

Regardless of what you are considering for the future for your legacy applications, OpenLegacy provides an immediate way to vastly increase your client population without touching a line of legacy code.

If this just seems like screen-scraping to you, look more closely. The differentiator here is the transparent creation of those front-end interfaces with no need for you to understand those technologies. Development using OpenLegacy doesn’t require you to know anything other than how to use the existing application.

In future posts, I’ll discuss other ways to access legacy applications using OpenLegacy via mainframe protocols that bypass 3270 emulation.

Contact Treehouse today for more information!

Request a Free, Online Demo of Treehouse Software’s Mainframe Real-Time and Bidirectional Data Replication / Integration Products

Did you know that Treehouse Software offers online demonstrations of the most complete and flexible portfolio of solutions available anywhere for real-time, bidirectional data replication and integration between mainframe and LUW data sources?

You can see how Treehouse Software’s popular tcACCESS and tcVISION products efficiently and cost-effectively use ETL, CDC, SQL, XML, and SOA technologies for data replication / integration, in an interactive demonstration with our skilled technical experts.

Integrate mainframe data and applications with LUW data sources…

tcACCESS is a comprehensive software solution that enables two-way integration between IBM mainframe systems and client/server, Web and SOA technologies — without the need for mainframe knowledge or programming effort. A proven platform that facilitates SQL-based integration of mainframe data sources and programs into LUW applications, tcACCESS uses industry standards such as SQL, ODBC, JDBC, and .NET. SQL queries to access mainframe data can be easily created using drag and drop techniques — no programming required.

tcACCESS_Diagram01

tcACCESS is a modular software solution. It consists of a base system that can either be implemented as a CICS transaction or as a VTAM application. The base system provides its own communication modules. The heart of the system is the tcACCESS SQL Engine which allows access to mainframe data sources using SQL statements. tcACCESS offers Listener components on the mainframe and on the client, as well as scheduling and security functions. Batch processors automate the information exchange processes between distributed applications.

Enable ETL and bi-directional data replication between mainframe and LUW platforms…

tcVISION allows the exchange of data between heterogeneous databases, from legacy non-relational mainframe sources to standard RDBMSs, in batch or real-time, via CDC (change data capture). With tcVISION, complex replication scenarios can be implemented with ease–including bi-directional “master/master” replication requirements.

tcVISION_simple

tcVISION considerably simplifies mainframe data exchange processes. The structure of the existing mainframe data is analyzed by tcVISION processors, then automatically and natively mapped to the target. The data mapping information is presented in a user-friendly and transparent format – even for users with no mainframe knowledge.

See for yourself, right at your desk…

DemoRequest

Tell us about your challenges. If you have a project where our mainframe data replication and integration products could be of assistance, our skilled sales and technical staff would be happy to set up a free, online demo. Simply fill out our short Treehouse Software Demo Request Form.

Treehouse Participates in CA IDMS and CA Datacom Technical Conferences

By Wayne Lashley, Chief Business Development Officer for Treehouse Software

Recently I, and others from Treehouse, had the pleasure of attending the CA IDMS Technical Conference in Framingham, MA, as well as the CA Datacom Technical Conference in Plano, TX. Both of these were free events organized by the IUA and CADRE (the IDMS and Datacom user associations), with event hosting and assistance provided by CA Technologies, to provide customers with an opportunity for training, product briefings and technical discussions with product management, product development and with other customers.

I have to commend the IUA, CADRE and CA for their initiative and efforts in putting on these events. With the changes made to the content and focus of CA World in 2014, IDMS and Datacom customers had limited opportunities to learn about product roadmaps and to interact with product representatives. The technical conferences were well-organized, offered excellent technical content—including a full day of condensed product training for both novices and advanced technicians—and even included some social time.

CA_Datacom_Event

Above: CA Datacom Event

In representing a vendor to these CA customer groups—we have a number of Datacom and IDMS customers using tcACCESS and tcVISION—, I was especially appreciative of the opportunity to get more involved in the community and to have our own technical people benefit from the training and interaction.

I was impressed with the level and quality of resourcing that CA is putting into its development and support teams. Both product groups have adopted Agile methodology for delivering, in a timely way, product capabilities and features that customers “want and will use”. Furthermore, with emerging functionality in IDMS (DB), IDMS/DC, ADS, Datacom and Ideal, there was ample demonstration that CA is embracing modern standards and trends such as SQL, Web services and Big Data.

CA_IDMS_Event

Above: CA IDMS Event

Readers from IDMS and Datacom shops that were unaware of, or unable to attend, the technical conferences should note that the success demonstrated by the recent events is likely to result in similar technical conferences being scheduled for the future. I encourage you to take advantage of them!

Treehouse Software – Your Mainframe / LUW Data Integration Partner

To learn more about tcACCESS and tcVISION, or to request a demonstration, Contact Treehouse Software today.

A Well-Earned (Application) Retirement

Guest blogger Howard Sherrington, CEO of NSC LegacyData Solutions Ltd., developers of DataNovata, discusses how retiring legacy applications can cut IT costs and free up resources – if the right approach is taken to accessing the legacy data.

The one constant in business IT is that yesterday’s new systems will become tomorrow’s legacy.  As organizations evolve and new technologies emerge, IT departments have to deal with the impact of this constant evolution and change on business operations.

But change events are difficult to predict.  Mergers or acquisitions can transform even recent deployments into duplicate or legacy systems, as the IT function struggles to keep up with the changing demands of the business.

Legacy systems represent a drain on IT resources, both in terms of cost and manpower.  Industry analyst Gartner conservatively estimates that businesses spend around 10% – 25% of their IT budget in supporting and managing legacy systems – and I believe that this can rise to as much as 35% in some organizations. This is especially true for organizations that are running old and complex systems and applications.

So, given the current business climate, where capital for new projects is harder to come by and operational expenditure for existing systems under close scrutiny, it’s prudent to look for more efficient ways of dealing with these older applications than to simply keeping them running.

It’s all about the data

In these circumstances, it’s essential to establish why the legacy system is being maintained.  In the vast majority of cases, it’s because of the data held in the system or application.

I frequently hear the statement, “No, we can’t do anything with that service, as we still need it”, even though nine times out of 10 it’s simply access to the data that’s required, rather than the application itself.  This also correlates with analyst Forrester’s estimation that nearly 85% of data in databases is inactive, simply being stored for subsequent access rather than being processed.

This is often the case for financial systems, especially in pensions and investment management, which usually lie dormant – at a considerable cost, as we saw earlier – so the business has access to the legacy data for legal, taxation, due diligence or compliance purposes.

A retirement bonus

So why not retain the vital elements and let go of the redundant parts?  By separating the legacy data that the business needs from the legacy system, and then decommissioning the applications and platforms, a business could make substantial savings in budgets, support and resource commitment.

There’s also the opportunity to increasing the efficiency of operations by giving staff wider, more flexible access to the legacy data.  Let’s take a closer look at how application retirement should be approached and managed, and the benefits it can offer your business.

Migration matters

The initial issue in application retirement is the migration of the data from the legacy application or platform.  Exactly how this is done will depend on several factors, including the type and age of the application and platform, and how the data is stored.  However, there are a couple of key ‘best practice’ points which should be observed in any migration project.

First, reduce the risk of data loss or damage by testing your procedures.  Make sure you have a backup copy of the data before trying a migration, and if possible, pilot the process using a small subset of the data.  Then you can compare the extracted data with the original to ensure the process isn’t changing the data in any way.

Second, ensure the data is migrated into a database format that is accessible by the widest range of applications, and that can run on low-cost, flexible computing platforms – ideally a structured, relational SQL-compliant database.  This helps to ensure flexible, open access to the data for a range of different user types.  Data structuring tools are available to simplify migrations to relational databases.

Access all areas
Once the migration is complete, the focus should be on how users will access the data: on building the applications that will support easy, flexible but robust data access.  The key to this is to use a tool that takes advantage of the open, Web model to run on any hardware and operating system at both the server and client side, delivering customizable data views and queries within a browser-based interface. 

This gives even non-technical users uniform access to data migrated from legacy systems from a familiar point-and-click interface – minimizing the need for user training.  It also helps organizations avoid ongoing licensing, maintenance and hardware costs for access to legacy data, and can give access to data over the Web from any location.

This makes application retirement a more efficient and cost-effective way of dealing with legacy systems, compared to other alternatives such as modernizing the application – which may mean expensive hardware updates, terminal emulation and so on – or transferring to a virtualized environment, which can carry a significant penalty in migration costs and ongoing management.

Considering the benefits

By retiring legacy systems, considerable benefits can be realized.  Firstly, analysts estimate that payback of outlay is often less than 12 months and the total ROI over three years usually exceeds 150%.

What’s more, by decommissioning your legacy systems, your IT team can focus on more strategic tasks than maintenance and support for old platforms.  There are also benefits such as reduced risk due to outages, acceleration of new product initiatives due to fewer integration or support issues, as well as a more streamlined disaster recovery plan.

So with the benefits that application retirement can offer, letting go of your IT department’s past while preserving the business information could make a key impact on your operations.  Applications and platforms will come and go, but data is forever.

TREEHOUSE CUSTOMER UPDATE:

 

 

Image

by Chris Rudolph, Senior Technical Representative for Treehouse Software and Joseph Brady, Marketing and Documentation Manager for Treehouse Software

This is a follow-up to our recent Treehouse Software Blog entry “Treehouse Software is Setting Sights on Many New Data Replication Projects”, in which we described a typical customer visit to implement data replication.

Treehouse representatives were on-site at a state government agency to configure tcVISION and set up bulk transfer and change data capture, as well as train the State employees on using and managing tcVISION. In a subsequent discussion with our contacts at the site, they reported that their deadline for delivering a reporting database in Microsoft SQL Server replicated from 63 ADABAS files has been met. They also happily noted that by using tcVISION, the bulk transfer of 60 million ADABAS records into SQL Server completed in only 20 minutes.

We are very pleased to have yet another satisfied customer benefitting from one of Treehouse Software’s mature and proven enterprise software solutions.

tcVISION provides real-time data replication through change data capture, and allows easy and fast data migration for mainframe application modernization projects. Enterprises looking for a product that enables bi-directional heterogeneous data replication between mainframe, Linux, Unix, and Windows platforms need look no further than to tcVISION from Treehouse Software.

To learn more about tcVISION, or to request a demonstration, contact Treehouse Software today.

Free and Informative Treehouse Software White Papers

Image

Download the free Treehouse Software white paper, “Legacy Data Migration: DIY Might Leave You DOA.”  This informative white paper outlines how failure of the data migration process can cause failure of an entire application migration/renewal project. Additionally, this white paper will demonstrate that given the maturity, wealth of functionality and relative low cost of tools like tcVISION, as compared to the effort, complexity and risk entailed in a “Do-It-Yourself” solution, there is no reason why a legacy renewal project should run aground on data migration.