by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
Software AG’s Adabas is a mainframe database that is still heavily used by government sites throughout the U.S. and the world, and this blog focuses on a current Treehouse Software customer – a U.S. State Government Agency that uses Adabas on their mainframe system.
Business Issue
The Agency’s modernization team was looking for a Change Data Capture (CDC) technology solution that enables them to synchronize their mainframe Adabas data on AWS, particularly an Amazon RDS. As with most Treehouse customers, the State’s mainframe contains vital data that must always be highly available, so rather than attempting a complete migration from the mainframe, the modernization teams decided to implement a multi-year data replication plan. This allows the mainframe legacy teams to maintain existing critical applications, while the modernization team develops new applications on AWS.
After researching various technologies, the Agency discovered tcVISION on the AWS Parter Network Blog and contacted Treehouse Software to discuss their project and to see a demonstration of Mainframe-to-AWS data replication.
Addressing the Uniqueness of Adabas
Having specialized in tools and services complementary to Adabas/Natural applications since 1982, Treehouse Software has successfully encountered and addressed many unique scenarios within the Adabas environment. The Treehouse technical team documented three primary issues with Adabas/Natural that the Agency needed to consider when they began planning data replication on AWS:
- Adabas has no concept of “transaction isolation”, in that a program may read a record that another program has updated, in its updated state, even though the update has not been committed. This means that programmatically reading a live Adabas database—one that is available to update users—will almost inevitably lead to erroneous extraction of data. Record modifications (updates, inserts and deletes) that are extracted, and subsequently backed out, will be represented incorrectly—or not at all—in the target. Because of this, at Treehouse we say “the only safe data source is a static data source”—not the live database.
- Many legacy Adabas applications make use of “record typing”, i.e., multiple logical tables stored in a single Adabas file. Often, each must be extracted to a separate table in the target RDBMS. The classic example is that of the “code-lookup file”. Most shops have a single file containing state codes, employee codes, product-type codes, etc. Records belonging to a given “code table” may be distinguished by the presence of a value in a particular index (descriptor or superdescriptor in ADABAS parlance), or by a range of specific values. Thus, the extraction process must be able to dynamically assign data content from a given record to different target tables depending on the data content itself.
- Adabas is most often used in conjunction with Software AG’s Natural 4GL, and “conveniently” provides for unique datatypes (“D” and “T”) that appear to be merely packed-decimal integers on the surface, but that represent date or date-time values when interpreted using Software AG’s proprietary Natural-oriented algorithm. The most appropriate way to migrate such datatypes is to recognize them and map them to the corresponding native RDBMS datatype (e.g., Oracle DATE) in conjunction with a transformation that decodes the Natural value and formats it to match the target datatype.
The tcVISION Technology Solution...
After technical discussions and a successful proof of concept (POC) that proved out a set of use cases, all teams at the Agency determined that tcVISION real-time mainframe data replication capabilities were the perfect fit for meeting their goals.
tcVISION‘s modeling and mapping facilities are utilized to view and capture logical Adabas structures, as documented in Software AG’s PREDICT data dictionary, as well as physical structures as described in Adabas Field Definition Tables (FDTs). Given that PREDICT is a “passive” data dictionary (there is no requirement that the logical and physical representations agree), it was necessary to scrutinize both to ensure that the source structures were accurately modeled.
Furthermore, tcVISION generates appropriate mappings and transformations for converting Adabas datatypes and structures to corresponding target datatypes and structures, including automatic handling of the proprietary “D” and “T” source datatypes.
The teams examined the three ways that tcVISION can access Adabas data:
- ETL – read the active database nucleus
- ETL – read datasets containing unloaded Adabas files created by the ADAULD utility
- CDC – read the active and archived PLOGs datasets
It was decided to access the data by reading the active and archived PLOGs datasets. The schema, mappings, and transformations from the metadata import were tailored to the customer’s specific requirements. It is also now possible to import an existing RDBMS schema and retrofit it, via drag-and-drop in tcVISION, to the source Adabas elements.
Additionally, the Agency’s teams are very pleased with tcVISION‘s minimal usage of mainframe resources. The product’s “staged processing” methodology accomplishes this, whereby the only processing occurring on the mainframe is the capture of changes from Adabas PLOGs. The bulk of the processing occurs on the AWS side, minimizing tcVISION’s footprint on the mainframe as seen in this diagram:
The user defines on which platform stage their processing should be done. Do as little as possible on the mainframe: Stage 0 – capture data and send data (internal format) to target, and process data in Stages 1 – 3 in AWS.
Customer Outcome
All requirements were met by tcVISION, which led to a successful project implementation.
Contact Treehouse Software for a tcVISION Demo Today…
No matter where you want your mainframe data to go – the Cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.
Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.
Further reading:
Many more mainframe data migration and replication customer case studies can be read on the Treehouse Software Website.
Treehouse Software's Blog 