Treehouse Software – 40 Years and Still Moving Forward (Part 3)

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc. 

Introduction

Many readers know that Treehouse Software has been around since 1983, serving enterprises worldwide with industry-leading software products and outstanding technical support. This blog series has discussed Treehouse Software’s origins and the growth of the software company from the early 1980s up to the present.

Change is in the Air, and in the Clouds…

Parts 1 and 2 of this series illustrated the solid beginnings of Treehouse Software in the 80’s and 90’s.  Several products were developed and introduced.  Marketing representatives were acquired in several countries around the globe.  Also, other companies with valuable products sought out Treehouse Software to sell and support their offerings.

In the late 90’s and the early 2000s, we began to experience certain customers’ needs to have Adabas data moved (migrated/copied/converted/distributed) to other database systems.  We developed the tRelational/DPS product set to analyze their Adabas data and structure, and move this data to other relational database systems (RDBMS) such as Oracle, Db2, etc.  This complicated product was our foray into the mainstream of our customers’ mainframe IT processing.  Millions of records of initial data needed to be materialized (ETL) efficiently, and a Change Data Capture (CDC) capability was imperative.  Mission accomplished.

In the early 2000s, some customers required real-time CDC.  This led to the development of DPSync, so named because it kept Adabas data in Sync with the target RDBMS; but this was limited to uni-directional replication.

Requests began coming in for bi-directional (e.g., moving data back to Adabas from Oracle).  Then it was Oracle-to-Db2.  And vice versa – and more variations.  We investigated companies purporting to do data migration from/to various database systems.  In 2007, we found a company with a product already developed and proven, that could in fact move data from/to practically all known database systems at that time.  We partnered with B.O.S. of Germany for Treehouse to do worldwide sales, marketing, support, demos, POCs, training, etc., for that impressive, growing product set, tcVISION.

tcVISION caught on quickly with some of our existing customers of Adabas, but the significant interest commenced when the Cloud took hold.  The Cloud was not just a remote data center, or a place to archive large amounts of data, but has capabilities and features that would attract our types of customers with mainframes and terabytes of data. Enterprise cutomers needed tools that allowed them the connectivity to take advantage of Cloud-based technologies, such as highly available and scalable databases; advanced analytics and security; machine learning and artificalial intelligence; data warehouses and stores; and the list goes on.  This shaped the future direction of Treehouse Software.  More on this in the next blog.

---

About Treehouse Software

Since 1983, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.

Contact Treehouse Software

Treehouse Software – 40 Years and Still Moving Forward (Part 2)

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc. 

Introduction

Many readers know that Treehouse Software has been around since 1983, serving enterprises worldwide with industry-leading software products and outstanding technical support. This blog series will dig a little deeper into Treehouse Software’s origins and explore how founder and president, George Szakach blazed a trail from being a programmer and manager in the 1960s and 1970s, to creating and growing his own software company from the early 1980s up to the present.

Treehouse First Generations… 1980s – 90s

As mentioned in Part 1 of this series, George’s foundational mainframe experience from 1960 to 1975, combined with skills honed from 1975 to 1982 at Software AG, planted the seeds for the next step, building Treehouse Software. George explains:

“Treehouse Software was started during Christmas of 1982, with a consulting assignment I decided to take in the Kingdom of Saudi Arabia doing ADABAS performance analysis and tuning.  In addition, I was asked to teach the customer how to use ADAMINT (which I get credit for developing from July 1975 through May 1979 – or blame, take your pick).  I thoroughly enjoyed my 10 weeks in the Kingdom, where I became a big fan of pistachio nuts. 

Treehouse Software was incorporated in mid-1984 in Sewickley, PA.  The plan then was to provide consulting and educational services to SAG sites, while developing software products related to the SAG enterprise.  The services might sell the products, and vice versa.  It worked and still works.  I started hiring help almost immediately as the demand for products and services was growing.

By the way, Sewickley is a small town near Pittsburgh where the oil, steel, coal barons of the 1800s and early 1900s had their summer homes up the hill from the river.  The 100-year-old horse-and-wagon trails and stone walls around the estates can still be seen.”

The Sewickley Bridge spans the Ohio River between Sewickley and Moon Township, PA.

Some interesting facts about Treehouse Software, Inc.

By the end of the 20^th century, Treehouse designed, developed, and released ten products, most notably:

• TRIM, ADABAS and Natural Performance Monitor
• Auditre, Auditing Tool for ADABAS
• N2O, Change Management System for Natural
• SECURITRE, Security System for ADABAS, Natural, and ADABAS Utilities
• tRelational/DPS, ADABAS-to-RDBMS Data Replication Package
• DPSync, Real-time ADABAS-to-RDBMS

Treehouse also marketed several additional products from companies around the world.  Most were related to ADABAS.

Other miscellaneous Treehouse happenings last century:

• Taught classes for over 4000 students, ADABAS and Natural related, and related to Treehouse products. 
• Partnered with 20 affiliates or marketing representatives in various countries, topping out at 14 around 1990.
• Produced 55 issues of the popular Treetips newsletter, with a hard-copy circulation of 13,000 per issue.
• Attended 11 Software AG conferences.
• Presented at 90 regional and local Software AG meetings in various countries.
• Visited affiliates, customers, and partners in at least 24 countries on all continents except Antarctica.
• Invited to Oracle Conference in 1995, in Philadelphia, where Emilie Szakach entertained the crowd with an on-stage dance with Chubby Checker.
• Bought four IBM mainframes for the company.
• Hired many top notch programmers and analysts, tech writers, support personnel.
• Along with most of the world, we awaited and prepared for the feared Y2K debacle, which didn’t happen. However, as the year 2000 approached, one of Treehouse’s most popular and most requested give-aways ever, was our Y2K desk clock…

Cha, Cha, Cha, Changes…

Along with many new products, innovations, and memorable, contributing staff moving through the doors of the Treehouse, we’ve also had several iterations of the company logo and colors, as seen in the following graphic. Do you have a favorite?

Coming soon… Part 3: Change is in the Air, and in the Clouds…    

---

About Treehouse Software

Since 1983, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.

Contact Treehouse Software

Treehouse Software – 40 Years and Still Moving Forward (Part 1)

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc. 

Introduction

Many readers know that Treehouse Software has been around since 1983, serving enterprises worldwide with industry-leading software products and outstanding technical support. However, this blog series will dig a little deeper into Treehouse Software’s origins and explore how founder and president, George Szakach blazed a trail from being a systems programmer in the early 60s, to creating and growing his own software company from the early 80s up to the present.

The beginnings… 1960’s.  Moon Landing, Flower Power, the Righteous Brothers, and Punched Cards

George is a Vietnam-era veteran and started working with mainframes in 1960 while in the Army. 

After programming school in Fort Monmouth, NJ, George was assigned to Fort Huachuca, Arizona where he wrote army related applications on the IBM 709.

Before leaving the army in 1963, George had many job offers.  Three years of programming experience was unheard-of back then, so his skillset was very valuable.  He was even offered a job by the president of Informatics, working in Houston at the NASA Johnson Space Center to “put a man on the moon.”  He declined.

Throughout the rest of the 60s, George worked at Burroughs, Univac, and Leasco. During the 70s through 1982, George worked for Ocean Data Systems, Data General, Optical Recognition Systems, Software AG (his longest stint – 7 years), and Superior Oil.

Punched card…

IBM 709 Computer System…

George’s archeological finds from his time at Univac…

With all of this foundational mainframe experience combined with his skills honed at Software AG, the seeds were planted for George’s future: take those roots, move to the trees, and build a house… 

Coming soon… Part 2: Treehouse Software’s first generations.    

---

About Treehouse Software

Since 1983, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.

Contact Treehouse Software

AWS Services Provide Advanced Monitoring and Analytics of tcVISION’s Mainframe CDC Processing

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Many Treehouse Software mainframe modernization customers have requirements for continuous near-real-time replication of mainframe data in order to keep a copy of the data synchronized on the Cloud. These customers are using tcVISION from Treehouse Software for changed data capture (CDC) for this synchronization, which allows changes occurring in any mainframe application data to be tracked and captured, and then published to a variety of AWS targets, including Amazon Simple Storage Service (S3). Some of these customers are also now asking us to recommend the best Cloud-based tools and methods to monitor and gain insights to these complex data processes. Coincidentally, while working with a current tcVISION customer, our technicians are testing out two particularly good, fully managed AWS services that can work hand-in-hand to address this need:

Amazon Athena

Since tcVISION supports Amazon S3 as a target, customers modernizing their mainframe systems on AWS can use Amazon Athena for monitoring and analysis of CDC processing from an S3 bucket.

Amazon Athena is a serverless, interactive analytics service built on open-source frameworks, supporting open-table and file formats. Athena provides a simplified, flexible way to analyze data from an S3 Bucket, as well as many other data sources, including on-premises data sources or other Cloud systems. Athena is built on open-source Trino and Presto engines and Apache Spark frameworks, with no provisioning or configuration effort required.

Figure 1: Example of an Athena query showing bulk-load statistics per table

Amazon QuickSight

Once Athena is setup for monitoring an S3 Bucket, users can easily view their CDC processing and analytics with Amazon QuickSight. QuickSight utilizes advanced machine learning-powered insights and intuitive dashboards, so end users can make the best and quickest data-driven business decisions.

Figure 2: Example of Amazon QuickSight monitoring the throughput of our data to Snowflake

Figure 3: Example of Amazon QuickSight pie chart showing the resulting rows loaded for each Snowflake table:

Figure 4: Example of Amazon QuickSight chart showing statistics for our data bulk-load into Snowflake:

Figure 5: Example of Amazon QuickSight chart showing our load time into Snowflake per table:

View the Amazon QuickSite video here…

---

Interested in seeing a live, online demo of tcVISION?

Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

---

Treehouse Software Salutes Franco Harris

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc. 

With the recent passing of Pittsburgh Steelers great running back and Pro Football Hall of Famer, Franco Harris, we would like to revisit April of 1993, when Treehouse Software held an international consultant’s symposium. The symposium brought together attendees and speakers from many consulting and technology companies, and schools from around the world. Since Treehouse Software is located in the greater Pittsburgh area, company president George Szakach was acquainted with Franco and invited him to deliver a fascinating and entertaining address, where he spoke about his career, several business ventures he was pursuing, as well as his budding interest in computer technology.

Franco Harris at Treehouse Software’s Consultant’s Symposium (April 1993)

A few years ago, George reminded Franco about his visit to Treehouse back in 1993. He remembered and they shared some laughs and memories. 

We would also like to mention Franco’s well-known sense of community and accessibility in Pittsburgh. Many staff members have met Franco over the years and have fond memories of his friendliness and willingness to spend time engaging in conversations. Those who come in to the Pittsburgh International airpot can see a sculpture depicting Franco’s famous “Immaculate Reception” from 1972.  Thousands of people, especially recently, have selfies taken with the sculpture. Franco will be missed by his many friends and the community.

Franco Harris sculpture at Pittsburgh International Airport.

---

About Treehouse Software

Since 1982, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.

Contact Treehouse Software

Providing a High Availability Framework for Mainframe-to-AWS Data Replication

by Dan Vimont, Cloud Solutions Architect at Treehouse Software, Inc.

Treehouse Software customers are using tcVISION to enable mission-critical mainframe-to-AWS data replication pipelines.  Some of these production pipelines are providing vital near-real-time synchronization between source and target, and thus can’t afford any significant downtime in the event of failure.  So it’s only natural that a number of our customers have been asking for advice in setting up a high availability configuration for their tcVISION components that run on AWS EC2 instances.  The High Availability Framework discussed here provides for a Failover EC2 instance to automatically pick up tcVISION processing should the Primary instance (running in another Availability Zone) go down.

The Core Components:  Primary Instance & Failover Instance

The core components of a tcVISION high availability framework consist of two EC2 instances running in different Availability Zones:  a Primary EC2 instance and a Failover EC2 instance.  Both identically-configured EC2 instances are attached to a shared working-storage file system (either an EFS or FSx volume), which allows the Failover instance to seamlessly and quickly pick up tcVISION processing should the Primary instance suddenly become unavailable.

Use a Step Function to Automate the Failover Process

In the event of failure of the Primary instance, the recommended framework calls for automatic triggering of a Step Function for reliable failover processing, with steps that include the following:

• verify that the Primary instance is unavailable (The tcVISION service cannot be active on both instances simultaneously, so this verification is vital.)
• redirect all network traffic from the Primary instance to the Failover instance (via Route 53)
• start tcVISION processing on the Failover instance

When Ready, Use a Step Function to Automate the Restoration Process

After operations personnel have completed recovery of the Primary EC2 instance, another Step Function may be manually triggered to reliably transfer tcVISION processing back to the Primary instance.

Many More Details are Available Upon Request to Treehouse Customers

Full details regarding our recommended High Availability Framework for tcVISION are available upon request to Treehouse customers.  AWS services utilized in the complete recommended framework include Step Functions, Lambda Functions, EventBridge rules, CloudWatch alarms, SNS topics, a Route 53 Private Hosted Zone, and more.  The following diagram is a partial visual inventory of the recommended framework components.

Interested in seeing a live, online demo of tcVISION?

Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

---

Some are calling mainframes “dinosaurs”, but many of us see that as a good comparison!

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Since the dinosaur analogy has been used so much to describe mainframe computer systems in recent years, I would like to use this blog to take a look at the parallels of dinosaurs and mainframes as it relates to the current buzz about modernization on the Cloud.

Of course, dinosaurs and mainframes have been around for a long time and are extremely resilient and successful. I especially say “are” in relation to dinosaurs, because many are not extinct at all, and the fossil record shows that several types have adapted to the changing world by evolving into birds. Additionally, during the age of dinosaurs, they branched off into countless varieties during a span of about 165 million years – hardly a failed species. Also, like the dinosaurs, the mainframe has thrived and survived for over six decades and is continuing to adapt – albeit not nearly as long as the reign of the dinosaurs, but an impressive run, nonetheless.

And the mainframe isn’t finished yet! Mainframe systems are still very much in use, running major banking processes, healthcare systems, government IT services, and critical business operations of many Global 2000 companies. As a matter of fact, IBM has been reporting growth year after year, as the IBM Z platform continues to see important innovations, such as with Cloud-native development capabilities, as well as impressive improvements in processing power.

Looking up and moving forward…

As with the dinosaurs who did not fear looking to the clouds and taking wing to ensure survival, the new breed of mainframers envision bold and exciting possibilities in Cloud computing. Many see remarkable opportunities for business advantage by modernizing their mainframe environments. This modernization includes replicating mainframe data on Cloud platforms in order to quickly capitalize on the latest Cloud services, such as analytics, auto scaling, machine learning and artificial intelligence (AI), high availability, advanced security, etc., or to move data to a variety of newer Cloud databases, streaming services, container services, and much more. With the proper data replication technology and planning, all of this modernization can occur while keeping the legacy mainframe environment active as long as it is needed!

The IBM Z mainframe isn’t going anywhere, and with visionary and daring leadership, it can continue to evolve and adapt to whatever develops in the Cloud… and beyond.

Ready to move forward, adapt, and evolve? Treehouse Software is here to help!

Treehouse Software is your partner on your journey into future mainframe modernization plans. With our “data first” approach, we can help accelerate digital transformation and successfully leverage Cloud and Hybrid Cloud initiatives on the IBM Z platform, storing sensitive data on a private Cloud or local data center, and simultaneously leveraging leading technologies on a managed public Cloud.

Through an innovative changed data capture (CDC) technology, our tcVISION product tracks and captures changes occurring in any mainframe application data, and then publishes them to a variety of Cloud targets. The customer moves only the right data to the right place at the right time – as much, or as little as they want.

The tcVISION data replication solution has a modular design, which enables it to support mass data load from one source to one or more targets, as well as continuous data exchange processes in real-time via CDC. This modular architecture and the provided APIs gives customers unlimited future potential for continued evolution, and use of new and emerging technologies.

---

Want to see tcVISION in action?

You can schedule a live, online demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database. Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION Mainframe-to-Cloud data replication demonstration.

How to Synchronize Data in Real Time Between the Mainframe and AWS with Treehouse Software’s Enterprise CDC Tool

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Many mainframe integration scenarios require continuous near-real-time replication of relational data to keep a copy of the data synched in the Cloud. Change Data Capture (CDC) is used for this near-real-time transactional replication by capturing change log activity to drive changes in the target dataset.

Just what is CDC anyway?

Simply put, and in relation to Mainframe-to-Cloud and open systems data replication, CDC is the use of processes to identify when data has been changed in a source system, so the replicated upstream or downstream (depending on how you look at it) target can be kept in sync with the changes.

In a recent AWS Architecture Blog, readers learn about integration using mainframe data to build Cloud native services with AWS, including transactional replication-based integration via CDC.

As mentioned in the blog, AWS Partner CDC Tools are available for connecting data center mainframes to the various data targets, and Treehouse Software’s tcVISION is one of those tools available in the AWS Marketplace.

tcVISION allows changes occurring in any mainframe application data to be tracked and captured, and then published to a variety of target AWS databases and applications. tcVISION provides an easy and fast approach for Hybrid Cloud projects, enabling real-time and bi-directional data replication between the hardware and AWS.

Example of Db2-to-AWS CDC using tcVISION Mainframe Manager:

tcVISION supports several CDC methods available, depending on each customer’s use case:

Bulk Transfer

• Efficient transfer of entire databases
• Analysis for data consistency (verification)
• Initial load (ETL) and periodic mass data transfer
• One-step data transfer

Log Processing

• Transfer of changed data near-realtime or scheduled time frame
• Reads both active logs and archived logs

Batch Compare

• Comparison of data snapshots using checksums
• Efficient transfer of changed data since last processing
• Flexible processing options (SORT etc.)
• Automatic creation of deltas by tcVISION

DBMS Extension

• Real-time capture of changed data directly from the DBMS
• Secure data storage even across DBMS restart
• Flexible propagation methods

Interested in seeing a live, online demo of tcVISION CDC?

Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

---

Should You Stay, or Should You Go? You Can Do Both by Incrementally Replicating Your Mainframe Data on the Cloud While Keeping Both Sides Synchronized

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Many of Treehouse Software’s enterprise customers are not close to considering the retirement of their mainframe systems, but instead have long-term data replication projects, or want to indefinitely have their legacy systems co-exist with a new Cloud platform. These organizations are looking for solutions that allow their legacy mainframe environment to continue while replicating data – in real time and bi-directionally – to take advantage of the latest Cloud services, such as analytics, auto scaling, machine learning and artificial intelligence (AI), high availability, advanced security, etc., or move data to a variety of newer Cloud databases, streaming services, container services, and more.

The Transition Doesn’t Have to be a Sudden Big Bang

Much of an enterprise’s mission critical mainframe data is stored in legacy mainframe databases, and the cost to maintain these databases is high.  An added complication is that the data is utilized by many interlinked and dependent programs that have been in place for many years, and sometimes decades. Unlocking the value of this legacy data is also difficult due to many very different types of mainframe databases (e.g., Db2, Adabas, CA Datacom, CA IDMS, etc.).

Immediate data replication on the Cloud is enabling government, healthcare, supply chain, financial, and a variety of public service organizations to meet spikes in demand for vital information, especially in times of crisis. The globalization of markets, increase of data volumes, 24×7 operations, changing business conditions, and high demand for up-to-date information also requires new data transfer and exchange solutions for heterogeneous IT architectures.

The Data-First Solution

Treehouse Software is here to help enterprise mainframe customers accelerate digital transformation and successfully leverage Hybrid Cloud initiatives on the IBM Z platform, storing sensitive data on a private Cloud or local data center and simultaneously leveraging leading technologies on a managed public Cloud. Our tcVISION replication solution focuses on changed data capture (CDC) when transferring information between mainframe data sources and modern databases and applications. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to a variety of targets. The customer moves only the right data to the right place at the right time – as much, or as little as they want.

The tcVISION replication solution has a modular design, which enables it to support mass data load from one source to one or more targets, as well as continuous data exchange processes in realtime via CDC. This modular architecture and the provided APIs gives customers unlimited potential for growth and use of new technologies.

tcVISION allows bi-directional, real-time data synchronization of changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected back on mainframe). The customer can then modernize their application on the cloud, open systems, etc. without disrupting the existing critical work on the legacy system.

In the following example high level architecture diagram, bi-directional data replication between Db2 z/OS and AWS using tcVISION is shown:

tcVISION utilizes a Windows-based GUI Control Board, which is ideal for non-mainframe programmers.  While mainframe experts are required in the design/architecture phase and occasionally during implementation, the requirement for their involvement is limited. The tcVISION Control Board acts as a single point of administration, data modeling and mapping, script generation, and monitoring. Comprehensive monitoring and logging of all data movements ensure transparency across all data exchange processes. In the following example, the mainframe can be seen communicating to an Amazon EC2-based tcVISION replication manager. The tcVISION Control Board shows the user a graphical representation of this replication:

Additionally, tcVISION supports complex data replication scenarios between multiple data sources and targets, as seen here:

With tcVISION, data replication projects can be implemented within a few of months, depending on the complexity of the project.  This includes the proof of concept and design/architecture stages.  After these stages are complete, the customer can start the first production implementation sprint, immediately providing business value.  We suggest successive agile sprints to allow for incremental deployment of additional file replication, sprint by sprint.

Supported Sources and Targets

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data replication for Mainframe-to-Cloud and Open Systems application modernization projects.

---

Contact Treehouse Software for a tcVISION Demo Today…

Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION demonstration. This will be a live, on-line demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database.

Treehouse Software Customer Case Study: A State Government Agency’s Real-time Data Synchronization Between IBM Mainframe Adabas and AWS

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Software AG’s Adabas is a mainframe database that is still heavily used by government sites throughout the U.S. and the world, and this blog focuses on a current Treehouse Software customer – a U.S. State Government Agency that uses Adabas on their mainframe system.

Business Issue

The Agency’s modernization team was looking for a Change Data Capture (CDC) technology solution that enables them to synchronize their mainframe Adabas data on AWS, particularly an Amazon RDS. As with most Treehouse customers, the State’s mainframe contains vital data that must always be highly available, so rather than attempting a complete migration from the mainframe, the modernization teams decided to implement a multi-year data replication plan. This allows the mainframe legacy teams to maintain existing critical applications, while the modernization team develops new applications on AWS.

After researching various technologies, the Agency discovered tcVISION on the AWS Parter Network Blog and contacted Treehouse Software to discuss their project and to see a demonstration of Mainframe-to-AWS data replication.

Addressing the Uniqueness of Adabas

Having specialized in tools and services complementary to Adabas/Natural applications since 1982, Treehouse Software has successfully encountered and addressed many unique scenarios within the Adabas environment. The Treehouse technical team documented three primary issues with Adabas/Natural that the Agency needed to consider when they began planning data replication on AWS:

1. Adabas has no concept of “transaction isolation”, in that a program may read a record that another program has updated, in its updated state, even though the update has not been committed.  This means that programmatically reading a live Adabas database—one that is available to update users—will almost inevitably lead to erroneous extraction of data.  Record modifications (updates, inserts and deletes) that are extracted, and subsequently backed out, will be represented incorrectly—or not at all—in the target. Because of this, at Treehouse we say “the only safe data source is a static data source”—not the live database.
2. Many legacy Adabas applications make use of “record typing”, i.e., multiple logical tables stored in a single Adabas file.  Often, each must be extracted to a separate table in the target RDBMS.  The classic example is that of the “code-lookup file”.  Most shops have a single file containing state codes, employee codes, product-type codes, etc.  Records belonging to a given “code table” may be distinguished by the presence of a value in a particular index (descriptor or superdescriptor in ADABAS parlance), or by a range of specific values.  Thus, the extraction process must be able to dynamically assign data content from a given record to different target tables depending on the data content itself.
3. Adabas is most often used in conjunction with Software AG’s Natural 4GL, and “conveniently” provides for unique datatypes (“D” and “T”) that appear to be merely packed-decimal integers on the surface, but that represent date or date-time values when interpreted using Software AG’s proprietary Natural-oriented algorithm. The most appropriate way to migrate such datatypes is to recognize them and map them to the corresponding native RDBMS datatype (e.g., Oracle DATE) in conjunction with a transformation that decodes the Natural value and formats it to match the target datatype.

The tcVISION Technology Solution...

After technical discussions and a successful proof of concept (POC) that proved out a set of use cases, all teams at the Agency determined that tcVISION real-time mainframe data replication capabilities were the perfect fit for meeting their goals.

tcVISION‘s modeling and mapping facilities are utilized to view and capture logical Adabas structures, as documented in Software AG’s PREDICT data dictionary, as well as physical structures as described in Adabas Field Definition Tables (FDTs).  Given that PREDICT is a “passive” data dictionary (there is no requirement that the logical and physical representations agree), it was necessary to scrutinize both to ensure that the source structures were accurately modeled.

Furthermore, tcVISION generates appropriate mappings and transformations for converting Adabas datatypes and structures to corresponding target datatypes and structures, including automatic handling of the proprietary “D” and “T” source datatypes.

The teams examined the three ways that tcVISION can access Adabas data:

1. ETL – read the active database nucleus
2. ETL – read datasets containing unloaded Adabas files created by the ADAULD utility
3. CDC – read the active and archived PLOGs datasets

It was decided to access the data by reading the active and archived PLOGs datasets. The schema, mappings, and transformations from the metadata import were tailored to the customer’s specific requirements.  It is also now possible to import an existing RDBMS schema and retrofit it, via drag-and-drop in tcVISION, to the source Adabas elements.

Additionally, the Agency’s teams are very pleased with tcVISION‘s minimal usage of mainframe resources. The product’s “staged processing” methodology accomplishes this, whereby the only processing occurring on the mainframe is the capture of changes from Adabas PLOGs. The bulk of the processing occurs on the AWS side, minimizing tcVISION’s footprint on the mainframe as seen in this diagram:

The user defines on which platform stage their processing should be done. Do as little as possible on the mainframe: Stage 0 – capture data and send data (internal format) to target, and process data in Stages 1 – 3 in AWS.

Customer Outcome

All requirements were met by tcVISION, which led to a successful project implementation.

---

Contact Treehouse Software for a tcVISION Demo Today…

No matter where you want your mainframe data to go – the Cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

---

Further reading:

Many more mainframe data migration and replication customer case studies can be read on the Treehouse Software Website.