A real-world example of how Treehouse Software is helping enterprises with their Mainframe-to-Cloud / LUW / Open Systems data migration and replication initiatives…

by Joseph Brady, Director of Business Development for Treehouse Software

Many medium to large size companies are still investing in new mainframe systems, because they have been relying on them for decades to handle mission critical data transaction processing. For these companies, their mainframes are deeply entrenched, and have been rock-solid reliable over the years, allowing many users and applications to simultaneously access the same data without interfering with each other. Mainframes also perform large-scale and extremely fast transaction processing (thousands of transactions per second). Chances are, most transactions for which you use your debit or credit cards have a mainframe processing the data in the background.

There have been countless predictions of the demise of the mainframe over the years, but they have been proven wrong every time, and there’s no sign of extinction any time soon. Mainframes remain a vital part of IT infrastructures used by companies spanning a variety of industries around the world, including banking, insurance, healthcare, government, aviation, and retail.

Today, most companies that have their enterprise data stored on a mainframe system are discovering the advantages of newer cloud and open systems databases and services, including instantaneous global database deployments (e.g., Amazon Aurora), economics to scale, high availability, elasticity, security, testing at scale, disaster recovery, etc. Consequently, these companies are looking for an automated solution that allows real-time data replication between the two environments, and many are turning to Treehouse Software.


_0_Airline_Manitenance

Customer Scenario: A Look at a Large Airline’s Mainframe Data Replication Project

Treehouse Software has a major airline customer that is a long-time user of Software AG’s Adabas database on their mainframe system. Their desire was to move their data to more modern applications off of the mainframe. However, their mainframe contains vital airline maintenance and parts data that must always be highly available, so rather than performing a complete migration from the mainframe, the customer decided to utilize a data warehouse with a requirement of real-time data replication. This allows the mainframe legacy teams to maintain existing applications, while the modernization team develops applications on the newly created RDBMS. All teams determined that tcVISION real-time mainframe data replication was the perfect fit for meeting their goals.

How tcVISION Addressed the Customer Needs...

The high-level environment for the airline customer using tcVISION for data replication looks like this…

___Airline_tcV_Overview

  • Adabas: Mainframe data source containing business critical information replicated to the RDBMS.
  • Oracle: Open Platform RDBMS chosen by customer as replication target for both data warehouse and modernization project. The tcVISION Manager also uses Oracle as a repository for the metadata (field mappings).
  • tcVISION Manager: Software component deployed on both source and target systems. It is responsible for provisioning resources for:
    • Processing scripts
    • Metadata import
    • Scheduling
  • tcSCRIPT: Software component deployed on both source and target systems. It works in conjunction with the tcVISION Manager to:
    • Perform initial load of Adabas into Oracle
    • Ongoing near realtime CDC (Change Data Capture) and replication from Adabas to Oracle
    • tcSCRIPT processes data from:
      • Direct from the DBMS (initial load) or from DBMS extracts
      • Adabas Protection log (PLOG)
  • tcVISION Control Board: Software component deployed on a Windows machine which provides graphical user interface for a single point of control to administer the tcVISION environment:
    • Metadata import, creates metadata rules governing relationship between mainframe and open platform data structures
    • Replication rule maintenance
    • DDL creation
    • Creation of ETL and replication processes
    • Start, stop, and schedule replication processes

Customer’s Technical Requirement for tcVISION

The airline requires that the replication solution must support online and batch Adabas transactions and provide data replication between Adabas and Oracle. Additionally, their large databases contain millions of rows (e.g., > 50 million) that must be supported, as well as support for database change audit requirements (datetime and type of operation), transaction management, and notification of exception events.

There must also be support for configuration management between development, QA, and production.

Finally, the solution must address hardware and software failures on the mainframe, Linux, and database nodes.

Customer Outcomes

All requirements were met by tcVISION, which led to a successful project implementation.  Here is a look at the customer outcomes and benefits:

Business Outcome Customer Benefit
Data warehouse is always in sync with Adabas using tcVISION data replication Improved timeliness and reliability of reporting data
Reduced usage of mainframe MIPS Reduced cost
All data required for modernization project was successfully replicated to target environment Increased business efficiently and agility

Many more mainframe data migration and replication customer case studies can be read on the Treehouse Software Website.


__TSI_LOGO

Contact Treehouse Software for a Demo Today…

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer. 

_0_Treehouse_tcV_Cloud_OpenSystems

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.


Further reading: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software

TREETIP: Did You Know About tRelational’s Schema Auto-Generation Feature?

by Joseph Brady, Manager of Marketing and Technical Documentation at Treehouse Software, Inc.

tRelational, the data analysis, modeling, and mapping component of Treehouse Software’s Adabas-to-RDBMS product set provides three options for developing RDBMS data models and mapping ADABAS fields to RDBMS columns:

Option 1: Auto-generation

Option 2: Importation of existing RDBMS schema elements

Option 3: Detailed definition and manipulation of schema and mapping elements using tRelational

The Auto-generation function can be an extremely useful productivity aid. By simply invoking this function and specifying an Adabas file structure, a fully-functional corresponding RDBMS schema (Tables, Columns, Primary Keys, Foreign Key relationships and constraints) and appropriate mappings are created virtually instantaneously. The table and column names, datatypes, lengths, and mappings/transformations are all automatically tailored specifically for the RDBMS product, platform, and version–the user need not be an expert in the RDBMS.

tRelational’s schema auto-generation simply requires specification of an Adabas file structure and associated options…

tRe_AutoGen

The auto-generated model can be immediately used to generate both RDBMS DDL and parameters for the Data Propagation System (DPS) component. Within minutes of identifying the desired Adabas file or files to tRelational, the physical RDBMS schema can be implemented on the target platform and DPS can begin materializing and propagating data to load into the tables.

It is important to note that these modeling options complement each other and can be used in combination to meet any requirements. Auto-generated schema elements can be completely customized “after the fact”, as can imported elements. Auto-generation can be used at the file level to generate complete tables and table contents at the field level, making it easy to then manually define and map one or more columns within a table, or even to denormalize MU/PE structures into a set of discrete columns.


About Treehouse Software’s tRelational / DPS Product Set

tReDPSMainDiagram01

tRelational / DPS is a robust product set that provides modeling and data transfer of legacy Adabas data into modern RDBMS-based platforms for Internet/Intranet/Business Intelligence applications. Treehouse Software designed these products to meet the demands of large, complex environments requiring product maturity, productivity, feature-richness, efficiency and high performance.

The tRelational component provides complete analysis, modeling and mapping of Adabas files and data elements to the target RDBMS tables and columns. DPS (Data Propagation System) performs Extract, Transformation, and Load (ETL) functions for the initial bulk RDBMS load and incremental Change Data Capture (CDC) batch processing to synchronize Adabas updates with the target RDBMS.

Visit the Treehouse Software website for more information on tRelational / DPS, or contact us to discuss your needs.

L10n in Heterogeneous Data Replication

by Wayne Lashley, Chief Business Development Officer for Treehouse Software

Most software vendors whose product markets extend beyond their own home country are familiar with the concepts of “i18n” and “L10n”, which are numeronyms for “internationalization” and “localization” respectively. i18n is the process of making a software product capable of adaptation to different languages and cultures, while L10n is the specific adaptation process for a given local market.

These terms take on special significance in the context of data replication software products—such as Treehouse’s DPSync, which provides real-time replication of mainframe ADABAS data to relational database (RDBMS) targets like DB2, Microsoft SQL Server and Oracle on various platforms. The very purpose of these products is to take data from a source and apply appropriate L10n to make it usable at the target, which is generally dissimilar in various aspects of the technical environment.

Perhaps the simplest form of L10n, having nothing to do with language or locale, is to transform database-specific field/column datatypes. Alphanumeric (A) fields in ADABAS are often mapped to CHAR or VARCHAR datatypes in an RDBMS, which are conceptually quite similar. Packed (P) fields may be expressed in an RDBMS as NUMBER, INTEGER, NUMERIC, DECIMAL, etc., depending on the vendor implementation and desired usages.

When it comes to Binary (B) format, things get tricky.  An array of bits in an ADABAS field can’t usually be mapped directly to a binary representation in an RDBMS column, due to the differences in the way data are represented between the platforms.

Decades ago, when I was earning my stripes as a novice mainframe programmer, the rules seemed simple: 8 bits made up a byte, and characters were expressed in single bytes encoded in EBCDIC.

(True story: During a university Assembler class many years ago, one of my classmates was muttering to himself, and the professor queried him about the subject of the “conversation”. The student replied “Just practicing my EBCDIC, sir!”)

Later on, I learned about that ASCII column of the “CODE TRANSLATION TABLE” in my indispensable System/370 Reference Summary GX20-1850-3, and I realized there was a whole world of computers beyond mainframes.

Image

But in fact things can be much more complex than simply EBCDIC and ASCII. L10n of data has to take into account the multitude of code pages and conventions that customers may use—and the customizations and exceptions to these.

Our European Technical Representative, Hans-Peter Will, has had to become somewhat of an expert in this over the past few years as he has worked with various customers in the Middle East on DPSync implementations.

Take the case of the way the Arabic language is handled in the context of applications at one site. Arabic is normally read right-to-left. But depending on system configuration, Arabic characters in a given field may be stored either left-to-right or right-to-left. Certain characters are represented in one byte, others in two. The cursive appearance of certain characters must be altered if they appear in the middle of a word rather than on an end. And in certain of this customer’s applications, the same screen display may show both Arabic and English. Even on screens where all of the words are in Arabic, and displayed right-to-left, there may be embedded numbers (e.g., telephone numbers) that need to be displayed left-to-right.

Now take all these complexities and factor in different database management systems (ADABAS vs. Oracle) running on different platforms (mainframe vs. Unix), each of which have their own configuration settings that affect the way characters are stored and displayed. Add to that the question of endianness (big-endian vs. little-endian) of the processing architecture.

The first time that Hans-Peter visited the customer in question, Treehouse software engineers had to figure out how to handle all these issues to ensure that ADABAS data would be replicated accurately and appropriately for use in Oracle-based applications. Fortunately, the combination of great product maturity (DPSync and its key underlying components tRelational/DPS having been battle-tested at countless sites over many years) and product extensibility (the ability to plug in complex custom transformations) enabled DPSync to be readily configured to accomplish the task at hand.

Having learned from that initial experience, Hans-Peter is now on familiar ground when assisting new Arabic-language sites implementing DPSync. Recently he was back in the Middle East visiting one of these new customers, and only hours after product installation he was able to confirm the accuracy of the SQL Server representation of data materialized (initially loaded via what is commonly called ETL, Extract-Transform-Load) from ADABAS using DPSync. The customer was also impressed with the speed of the process, both in terms of configuring the materialization (taking advantage of the tRelational schema auto-generation feature) and executing it (using an ADASAV backup as source, avoiding any workload on ADABAS). That customer is now in production with real-time ADABAS-to-SQL Server replication.

What’s your L10n challenge? Contact Treehouse and learn how DPSync and our other products are able to meet it.