Treehouse Software Customer Success: Element Fleet Management Corp. Real-time, Bi-directional Synchronization Between IBM Mainframe Datacom and PostgreSQL

Element_Building

As a leading global fleet management company, Element provides a comprehensive range of fleet services that span the total fleet lifecycle, helping customers optimize their strategies at every stage – from acquisition and financing to program management and vehicle remarketing. These services, together with exceptional tools, technologies and consulting, empower customers to realize extraordinary results.

Business Background

Element Fleet Management has over one million vehicles under management, and serves more than 2,800 customers. The company employs over 2,700 staff members across 12 offices. Transaction sizes range from $500,000 to $40 million. 50 countries are served through the Element-Arval Global Alliance. Top industries served by Element Fleet Management are agriculture, business services, chemical, construction, consumer products, education and non-profit, energy, food and beverage, insurance, manufacturing, pharmaceutical and healthcare, professional services, telecommunications, transportation, and utilities.

Business Issue

Much of Element’s mission critical mainframe data is stored in a legacy IBM mainframe database. The cost to maintain these legacy databases is high and they lack the features required for modernizing the data architecture. The data is utilized by an extensive number of interlinked programs dependent on this legacy data. Element required a solution that would allow the legacy environment to continue while having real time replicated data on a modern database/platform. This enables them to utilize modern programing and data tools to be competitive in their market.

Element_Diagram

Technology Solution

Treehouse software and the customer developed a phased plan to prove out and implement tcVISION. This started with a proof-of-concept to prove out the technology’s effectiveness and technical suitability. This was proceeded by architectural phase to define the architectural solution including, configuration management, environments and promotion, high availability, security, disaster recovery, monitoring, operations procedures, script customization, data replication mapping and training plans. After defining the architecture, the production deployment phase started with incremental sprint-like deployments. Additional files were deployed into production every few weeks.

This project enabled the tcVISION product to synchronize mission critical financial IBM mainframe Datacom data to a modern PostgreSQL database. Bi-directional, real-time data synchronization enables changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected on mainframe Datacom). This allows their business to modernize the application on PostgreSQL without disrupting the existing critical work on the legacy system.

NOTE: Many modern tools are using the PostgreSQL database, which greatly enhances business agility (e.g., other divisions within Element are already using Amazon Web Services (AWS), so having tcVISION in place now makes mainframe data delivery to the AWS cloud an easy option). See a step-by-step video of Mainframe-to-PostgreSQL on AWS here. 

Successful, real-time mainframe data replication to an open systems application allows web and mobile access and updates. Element indicated their satisfaction with the results and the solution, especially reduced mainframe MIPS costs, and enhanced business ability to respond to business environment changes.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

tcVISION_Connection_Overview_Web01

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including PostgreSQL, Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, and IBM Informix.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

 

Want to See a Live, Online Demonstration of Mainframe-to-Amazon Web Services (AWS) Data Replication using Treehouse Software’s tcVISION?

tcVISION is a powerful and comprehensive solution for real-time, bi-directional replication of mainframe data to AWS. tcVISION’s GUI modeling and mapping, and ease of migrating data to AWS makes it an ideal choice for modernizing large mainframe environments. Additionally, tcVISION can synchronize mission critical data from a mainframe system on AWS. Real-time, bi-directional data synchronization enables changes on either system to be reflected on the other system (e.g., a change to a PostgreSQL table on AWS is reflected on the mainframe database). This allows businesses to modernize an application on AWS without disrupting the existing critical work on the legacy system.

Contact Treehouse Software today for a live, online session that demonstrates real-time replication of mainframe data to Amazon RDS instances!

Example of a Treehouse Software Mainframe-to-AWS Data Replication Demo Environment…

Slide1

tcVISION Technology

tcVISION‘s support for AWS as a target is fully integrated alongside traditional Linux/Unix/Windows (LUW) targets such as PostgreSQL, Oracle Database, IBM DB2 LUW, Software AG Adabas LUW, IBM Informix, Sybase, Microsoft SQL Server, and ODBC.

tcVISION_Connection_Overview_Web01

tcVISION considerably simplifies mainframe data exchange processes. The structure of the existing mainframe data is analyzed by tcVISION processors, then automatically mapped to a target data mapping. The data mapping information is presented in a user-friendly and transparent format – even for users with no mainframe knowledge.

The mapping information is saved in a meta data repository hosted on a relational database, and can easily be made available to other applications. The Windows-based Control Board of tcVISION provides an easy-to-use facility to administer the data flow. tcVISION provides a variety of interfaces to allow seamless integration with ETL or EAI solutions.


__tsi_logo_400x200

Since 1982, Treehouse Software, Inc. has been serving enterprises worldwide with industry-leading software products and outstanding technical support.  Today, Treehouse Software is a global leader in providing data migration, replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, business intelligence and analytics, and application modernization.

 

 

Treehouse Software Eases the Challenges for Mainframe-to-Modern Platform Transformation Programs for Systems Integrators

Unlock Your Customers’ Business Value Faster with Frequent Production Deliveries, Using Agile Methods

Treehouse Software’s product, tcVISION has proven to be vital for the ease and success of many mainframe modernization initiatives.

A big problem for a Systems Integrator (SI), or any mainframe modernization vendor, is the myriad of dependencies between mainframe and open systems data and components. As part of a mainframe modernization project, these vendors need to determine the ordering of phases or waves of applications to be migrated. There are often shared databases that can make it very difficult to move one set of applications without moving others. This can result in the conclusion that they must go with a “big bang” approach, which can lead to a bid for a program with one large delivery at the end with all of the integrated components in place. This is very unappealing to customers as they are paying out big dollars without the corresponding business value of a production deployment. This approach also significantly increases the project risk.

___Mainframe_To_LUW

With Treehouse Software’s bi-directional replication product, tcVISION, a subset of applications can be migrated, and tcVISION’s data synchronization capabilities can be used to bridge specific shared legacy mainframe databases to the modern database versions on open systems. This allows them to deliver in phases, thus getting modernized modules in production faster. In addition, it decreases risk as issues can be found much faster with the more frequent production deployments. When the last modules are deployed to production, they would no longer need the synchronization.

As an example, a large Treehouse customer is currently involved in gradually modernizing and replacing a set of applications, and is using a similar approach. The customer will be using their mainframe Adabas database synced with Oracle on Linux, while in the process of modernizing the applications. Treehouse Software’s experts and tcVISION product are making sure the customer’s transition goes smoothly and seamlessly.


Enterprise ETL and Real-Time Data Replication Between Virtually Any Source and Target

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

_0_tcvision_connection_overview

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Hadoop/HDFS, MongoDB, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix, PostgreSQL, etc.


__tsi_logo_400x200

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Treehouse Proof of Concept: Bi-directional Replication Between Adabas and SQL Server

Chris Rudolph and Kevin Heimbaugh, Senior Technical Representatives for Treehouse Software, visited a customer site (a large retail and distribution company) to perform a five-day proof of concept (POC) of tcVISION with bi-directional replication between Software AG’s Adabas and Microsoft SQL Server.

Chris and Kevin initially met with the customer team, consisting of the DBA, Applications Manager, and a technical applications person. The agenda for the week was set to:

  • Import metadata from several Adabas files
  • Bulk load the Adabas data into SQL Server
  • Set up replication from Adabas to SQL Server
  • Add the bi-directional replication back to Adabas

Additionally, there were a few other items the customer wanted the Treehouse team to address, including support for date formats; timestamps for bi-directional replication to avoid update conflicts; using Predict views to define multiple SQL Server tables; and support for MUs and PEs. Chris noted that everything on the customer’s list is easily supported, and there are several options for the update scenarios that can be used.

_0_tcVISION_Adabas_To_SQLServer

After the tcVISION components were installed, the POC began by using tcVISION’s Control Board to define a metadata repository database in SQL Server. Once that was set, the teams moved on to import the first Adabas file’s metadata using tcVISON’s Metadata Import Wizard. As part of this process tcVISION generated Adabas to SQL Server schemas and field-to-column links as well as created target tables in SQL Server. Bulk Transfer scripts were created using a wizard to read the Adabas file on the mainframe, and load the data into SQL Server using the SQL Server bulk loader. Chris created a control script to show how tcVISION can concurrently bulk transfer multiple Adabas files into SQL Server This required increasing the tcVISION Manager’s VSE partition size to successfully test multiple load scripts executing in parallel.

The teams moved on to define the real-time change data capture (CDC) scripts necessary to process the Adabas PLOG. The tcVISION scripts use a two-phase approach to queue captured Adabas transaction on the open platform, then transform and apply the transactions to SQL Server. The scripts were set up to automatically generate detailed logs to track the PLOG transactions captured, SQL statements successfully applied to SQL Server, failed SQL statements, and informational items such as auto-corrected data and transactions rejected due to processing rules.

Now that several tables were defined and loaded, the bi-directional process was set up. SQL Server CDC was enabled for each table to be replicated. The team made a change within SQL Server and verified that the change show up in the SQL Server CDC tables. The SQL Server-to-Adabas mappings were defined in the tcVISION metadata repository, including the “back update check” to ensure only non-tcVISION transactions are captured, and the scripts on both Windows and mainframe were defined to create the LUWs from the SQL Server CDC and apply the changes to Adabas.

CDC from SQL Server to Adabas was successfully tested. Chris then showed the ability to create Journal replication where each change can be captured by replication type. The team spent time creating a few more mappings so multiple file / table updates could be tested, in addition to doing updates while the scripts were stopped to simulate a lost connection. This included setting up a new script to process copied PLOG datasets created by the ADARES utility.

The team defined the remainder of their Adabas files to the metadata repository. Some were set them up for bi-directional replication, and others were setup for unidirectional replication and Journal replication. Everything work as expected at the wrap-up meeting where the team provided a live demonstration to management of tcVISION and the items accomplished. The final tcVISION presentation and demo went very well, and everyone was pleased with the progress made during the week.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcvision_connection_overview

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.


__tsi_logo_400x200

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

TREETIP: Integrate Mainframe Data Sources In Your Big Data Initiatives

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

Analysts have observed that perhaps 80 percent of the world’s corporate data still resides on mainframes. So it’s no surprise that Bloor Research (http://www.bloorresearch.com/research/spotlight/big-data-and-the-mainframe/), notes that “it is necessary today to place the mainframe as a ‘first-class player’ in any enterprise Big Data strategy.”

In February 2017 we highlighted tcVISION’s support for replication to the leading NoSQL database MongoDB. MongoDB continues to increase in popularity as a back end for operational applications with real-time requirements.

tcVISION also supports analytics and “mainframe offload” Big Data use cases that generally leverage Hadoop HDFS and/or streaming data transport. With tcVISION, data from a wide variety of IBM mainframe data source can be quickly and easily replicated to Big Data targets, requiring minimal mainframe know-how and having minimal impact on the mainframe.

___tcVISON_Big_Data_001

Boost the return on investment for your Big Data initiatives using tcVISION!


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix, and PostgreSQL.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Can OpenLegacy help connect PHP with DB2 on the mainframe?

Recently, a poster to a mainframe technical discussion forum asked the question: How can we connect PHP (a popular server-side scripting language designed for web development) with DB2 on the mainframe? Treehouse Senior Software Developer Frank Griffin replied, describing how OpenLegacy could be the answer.  

You have some options here. If all you want is access to the raw DB2 data, JDBC or ODBC access will work fine for you, although you will have to write either C ODBC or Java JDBC code that can be called from PHP to do the deed.

If you already have mainframe code accessible via the network (CICS/IMSDC/TSO via TN3270, CICS via TCP/IP) that accesses your data and adds business logic to the mix, you can use the FOSS* OpenLegacy project to mate this logic to your PHP app.

The simplest approach involves using OL to navigate through the green screens of (CICS, IMS/DC, TSO, or whatever) to get to the data you want. This is done via the OSS** s3270 scripting 3270 emulator which creates an XML “trail” file which documents the interactions between yourself and the legacy application over the TN3270 connection. Once you’ve navigated through the series of screens that exposes the data you want to find, you trigger a portion of OpenLegacy which analyzes those screens, identifies logon sequences and unprotected fields on those screens which are linked to client-supplied input data, and generates Java classes which can at some later time use the “trail” file to re-drive the emulator to access the legacy application using client-supplied values for some of the input in order to obtain transaction output associated with those input values.

If your mainframe apps are better modularized, i. e. if you have separated the business logic from the display logic, and the business logic can be invoked through a CICS COBOL program which is designed to obtain its input and provide its output via a CICS COMMAREA, OL can analyze the source code and generate Java classes that invoke those programs directly without screen-scraping.

And, if what you really want is JDBC, OL can generate a series of Java classes that do all of the JDBC work for you and provide you with methods that simply provide the legacy data with no hint as to where it came from.

OL can also layer additional access software. Once the fundamental Java classes that access the legacy data are in place, OL can generate Java apps that use those classes on behalf of clients using all sorts of modern APIs, including SOA/SOAP, REST/JSON, and Mobile. All of this happens with the push of a button.

All of this is FOSS. You can download the OL code and start using it immediately, and it can do all that I’ve described, out-of-the-box. OL makes their money from selling an Enterprise Edition that includes support and some security and management pretties.

If the only access path from PHP to Java is direct invocation of Java code, you’d have to write a Java stub to interact with the OL classes, but this s going to be a *lot* simpler than trying to write JDBC or ODBC applications on your own.

*FOSS = Free Open-Source Software

**OSS = Open-Source Software


__OpenLegacy_Logo

OpenLegacy is the world’s first and only light-weight, non-intrusive solution for automated legacy modernization and enterprise application integration. With its standards-based, open-source platform, OpenLegacy enables enterprises to rapidly extend legacy systems to mobile, web and cloud applications; delivering risk-free, high-impact results that solve immediate business needs.

__OpenLegacy_High_Level

OpenLegacy‘s standard tools rapidly extract the services and information from within legacy systems into an editable format that puts the power of integration into the enterprises’ hands without the expensive handcuffs of vendor lock in. Once a business process is exposed — which can be done in minutes — the output can automatically be transformed into stand-alone mobile, web, and cloud applications; and connected with other solutions. Most importantly, no changes are required to the legacy system in order for OpenLegacy to work — the process is risk-free.

Contact Treehouse today for more information!