Mainframe VSAM Change Data Capture (CDC) to Cloud and Open Systems with tcVISION from Treehouse Software

by Joseph Brady, Director of Business Development and AWS and Cloud Alliance Leader at Treehouse Software, Inc.

tcVISION_Mainframe_VSAM

Treehouse Software is the worldwide distributor of tcVISION, the innovative software product that allows immediate data replication between an impressive array of Mainframe sources and Cloud and Open Systems targets. This blog focuses on tcVISION‘s support of VSAM mainframe data sources (batch and CICS on z/OS, and CICS on z/OS and z/VSE).

tcVISION performs VSAM Change Data Capture (CDC) either via its own “DBMS-Extensions”, or via IBM’s CICS VR product. tcVISION has separate DBMS-Extensions to capture changes from CICS (using the CICS External Interface) and batch (via a JCL wrapper). All captured changes, regardless of whether they are performed by tcVISION or CICS VR are written to the z/OS Logstream on the mainframe. tcVISION then reads the Logstream and transfers the transactions to a tcVISION server running in the Cloud or on-prem, which is responsible for queueing, transforming, and applying the captured changes to the specified target.

Additionally, when planning VSAM CDC there are a number of operational items to consider, such as volume of batch transactions, data changes that occur during periods of time while the VSAM file is offline, etc.

In this instructional video, tcVISION is shown capturing changes from VSAM on z/OS and writing them to SQL Server on Windows:

 


__tsi_logo_400x200

Contact Treehouse Software Today for a tcVISION Demonstration…

No matter where you want your mainframe data to go – the Cloud, Open Systems, or any LUW target – tcVISION from Treehouse Software is your answer.

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

How to Replicate Mainframe Data to a Big Data Environment via Kafka with tcVISION

by Joseph Brady, Director of Business Development and AWS and Cloud Alliance Leader at Treehouse Software, Inc.

tcVISION from Treehouse Software allows enterprise customers to replicate data between mainframe, Cloud, or Hybrid Cloud while maintaining their legacy environments, and one of the more popular targets for mainframe modernization that we have been seeing is Apache Kafka®.

tcVISION_Mainframe_To_Kafka

What is Kafka? 

Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. A data pipeline processes and moves data from one system to another, and a streaming application is an application that consumes streams of data.

Kafka is reliable, stable, flexible, robust, and scales well with numerous consumers, working seamlessly with most popular data warehouses and data lakes like Hadoop, Redshift, S3, BigQuery, Azure, etc. Kafka can also be used for real-time analytics, as well as to process real-time streams to collect Big Data.

See how tcVISION easily connects mainframe systems to Kafka…

Kafka handles massive volumes of data and remains responsive, making Kafka a preferred platform when the volume of the data at the mainframe level –> BIG.

Kafka is a supported target in tcVISION, and in this instructional video, tcVISION is shown synchronizing data in real-time from Db2 on z/OS via Kafka to a Big Data environment:

Additional Reading: Treehouse Software is a Confluent technology partner and we recently co-authored a blog entitled, “Enterprise Mainframe Change Data Capture (CDC) to Apache Kafka with tcVISION and Confluent”.


__tsi_logo_400x200

Contact Treehouse Software Today…

No matter where you want your mainframe data to go – the Cloud, Open Systems, or any LUW target – tcVISION from Treehouse Software is your answer.

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

New Faces at Treehouse Software

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

__TSI_LOGO

Treehouse Software is growing and on the move! We are proud to have many staff members who have been here for 20+ years, and we have recently brought on several experienced business, mainframe, and Cloud experts. Meet our newest team members:

John Szakach, Chief Operating Officer

John joined Treehouse as a Business Strategy Consultant and in 2021 was promoted to Chief Operating Officer. While new to Treehouse, John brings over 40 years of relevant work experience to the organization and is an AWS Certified Cloud Practitioner as well as a Certified Project Management Professional. John has held a variety of management roles in different industries including VP of Organizational Effectiveness, VP of Quality Assurance, and VP of New Product Development. He has also held positions as Director of Flight Standards and Quality Control, and Director of Operations. In addition to over 51 years of total flight experience, including 20 years as a pilot for United Airlines, he has received numerous awards including the United Airlines Captain of the Year and the FAA Master Pilot Award, the FAA’s highest award for safety and compliance. John has a Bachelor’s Degree in aviation management.

Dan Miley, Product Support

Dan is a software engineer with deep experience and understanding of IBM Assembler, COBOL, JCL, IDMS, SAP ECC. He has worked with some of the world’s largest organizations, including president/consultant of his own company for over 10 years. Dan has already been instrumental in landing some major mainframe-to-Cloud data modernization customers for Treehouse Software.

Sasha Efron, Senior Technical Representative

Sasha is a mainframe technical specialist and DBA with over 25 years experience in in systems analysis, design, development, enhancement, testing, implementation and maintenance in insurance and banking systems with specialization in Software AG and IBM Mainframe technologies. He also has been involved in legacy modernization projects for several worldwide companies.

Joseph Rogan, Senior Technical Representative

Joseph is a Senior Technology Leader with 30+ years experience working in multiple industries, including transportation (specifically rail), logistics, education, financial services (banking, re-insurance, and trading systems), commercial insurance, and state government. His core competencies include database design and implementation, OLTP, OLAP, and data warehouse design, project planning, and project management. Joseph is also a highly trusted, conceptual, business partner and leader with excellent presentation, negotiating, management, mentoring, and strategic planning skills.

Daniel Vimont, Senior Technical Representative

Daniel brings 30+ years experience in multiple computer languages, databases, frameworks, and distributed processing for mainframe, Cloud, and open systems. He is very familiar with the principles of ETL and CDC in mainframe data transformation and migration. Dan is a Certified AWS Cloud Practitioner and has experience in designing and developing AWS/SDK (boto3) framework for on-premises invocation/monitoring of AWS services. Additionally, Dan’s versatile background as a data and software engineer, educator, and business advisor is a valuable asset to Treehouse’s vision of being a close partner in our customers’ planning and modernization efforts.

Treehouse Software Experts are Our Best Assets

management-team

When asked by prospective customers, “What are your primary differentiators?”, we immediately point to our people who have decades worth of experience in helping mainframe customers with innovative tools, services, and training. Our extensive experience, deep knowledge, and wide-ranging capabilities in mainframe technologies make the company a valued partner for third-party solution providers and a trusted advisor to customers.

We are fortunate to have a staff with a wealth of knowledge and skills that span not only Mainframe, but Cloud, LUW, and Open Systems technologies. Treehouse Software‘s technicians have installed products and trained end-users in some of the largest mainframe sites around the world, and our highly-rated 24X7 technical support is second to none.

The Treehouse Team Approach

Treehouse Software’s expert staff has proven its ability to work effectively as part of a larger team to meet clients’ complex business goals. AWS, Google, Microsoft, IBM, Oracle, Deloitte, Accenture, Confluent, and other large vendors have selected our expertise, technology, services, and training for their mainframe data modernization practices.


__tsi_logo_400x200

Contact Treehouse Software Today…

Treehouse Software has been helping enterprises mainframe customers since 1982, and in recent years, we have been developing a strong presence in the Cloud market space relating to mainframe data replication and modernization. As a result, Treehouse Software is currently working with technical and sales leaders from all popular Cloud platform companies and major systems integrators to take advantage of our deep mainframe skills and our tcVISION Mainframe-to-Cloud data replication solution.

No matter where you want your mainframe data to go – the Cloud, Open Systems, or any LUW target –Treehouse Software is here to help. Contact us to discuss your needs.

Replicating Mainframe Data on Cloud-based Relational Databases

by Joseph Brady, Director of Business Development / AWS and Cloud Alliance Leader at Treehouse Software, Inc.

Right_Data_Right_Place_Time02

Treehouse Software has been helping enterprises mainframe customers since 1982, and in recent years, we have been developing a strong presence in the emerging Cloud market space relating to mainframe data replication and modernization. As a result, Treehouse Software is currently working with technical and sales leaders from all popular Cloud platform companies and major systems integrators to take advantage of our deep mainframe skills and our tcVISION Mainframe-to-Cloud data replication solution.

The Choice is Yours…

tcVISION provides the means for customers to easily replicate relevant data between most mainframe data sources (IBM Db2, IBM VSAM, IBM IMS/DB, Software AG Adabas, CA IDMS, CA Datacom, or even sequential files) and the most popular Cloud platforms, including AWS, Google Cloud, Microsoft Azure, Confluent Cloud, and Oracle Cloud

Today, customers are finding it easier than ever to set up, operate, and scale relational databases in the Cloud. Here is a quick look at some Cloud relational database systems that tcVISION supports…

Amazon Aurora relational database, a MySQL and PostgreSQL-compatible relational database built for the Cloud that combines the performance and availability of traditional enterprise databases with the simplicity and cost-effectiveness of open source databases:

Google Cloud SQL, a fully managed relational database service for MySQL, PostgreSQL, and SQL server. You can connect with nearly any application, anywhere in the world. Cloud SQL automates backups, replication, and failover to ensure your database is reliable, highly available, and flexible to your performance needs:

Azure SQL Database is an intelligent, scalable, relational database service built for the Cloud. Optimize performance and durability with automated, AI-driven features that are always up to date. Focus on building new applications without worrying about storage size or resource management with serverless compute and Hyperscale storage options that automatically scale resources on demand:

How Does tcVISION Work?

tcVISION focuses on changed data capture (CDC) when transferring information between mainframe data sources and Cloud-based databases and applications. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to the targets.

tcVISION allows bi-directional, real-time data synchronization of changes on either platform to be reflected on the other platform (e.g., a change to a Cloud PostgreSQL table is reflected back on mainframe). The customer can then modernize their application on the cloud, open systems, etc. without disrupting the existing critical work on the legacy system.

Here is a high-level walkthrough of tcVISION mainframe data replication on Cloud and open systems…


__tsi_logo_400x200

Contact Treehouse Software Today…

No matter where you want your mainframe data to go – the Cloud, Open Systems, or any LUW target – tcVISION from Treehouse Software is your answer.

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

What are the Benefits of Replicating Mainframe Data on Cloud or Hybrid Cloud Systems?

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software

Enterprise customers with mainframe systems have begun their movement of data to the Cloud or hybrid Cloud (a mixed computing, storage, and services environment made up of on-premises infrastructure, private Cloud services, and public Cloud) to benefit from new and powerful technologies that deliver significant business benefits and competitive advantage. Compared to the number of mainframe shops that are in the planning stages of their Cloud projects, existing adopters’ numbers are still relatively small.

Today, it is easier than ever for customers to take advantage of cutting edge, Cloud-based technologies, changing the way they manage, deploy, and distribute mission-critical data currently residing on mainframe systems. During the planning phase of a Cloud or hybrid Cloud modernization strategy, some benefits that are quickly discovered include:

Trade Capital Expense for Variable Expense – Instead of having to invest heavily in data centers and servers before customers know how they are going to use them, they pay only when they consume computing resources, and pay only for how much they consume.

Global Deployments – Cloud platforms span many geographic regions globally. Enterprises can easily deploy applications in multiple regions around the world with just a few clicks. This means there can be lower latency and a better experience for customers at minimal cost.

Economies to Scale – By using Cloud computing, customers can achieve a lower variable cost than they can get on their own, because usage from hundreds of thousands of customers is aggregated in the Cloud. Providers such as AWS, Google Cloud, etc. can achieve higher economies of scale, which translates into lower pay as-you-go prices.

Scale of Services – Cloud-based products offer a broad set of global services including compute, storage, databases, analytics, machine learning and AI, networking, mobile, developer tools, management tools, IoT, security, and enterprise applications. These services help organizations move faster, lower IT costs, and scale.

World Class Security – All major Cloud platforms offer advanced and strict security that complies with the most stringent government and private sector requirements.

Extreme High Availability (HA) – Major Cloud platforms span many geographic regions around the world.  By designing services and applications to be redundant across regions, HA is enhanced far beyond a single on-premises data center.

Testing at Scale – Cloud servers and services can be created and charged on demand for a specific amount of time.  This allows customers to create temporary large-scale test environments prior to deployment that are not practical for on-premises environments.  Large scale testing reduces deployment risks and helps to provide a better customer experience.

Auto Scaling and Serverless Deployments – Major Cloud platforms have many serverless and autoscaling options available, allowing for scalable computing capacity as required.  Customers pay only for the compute time they consume – there is no charge when the code is not running. Another example is the ability for a Cloud database to automatically start up, shut down, and scale capacity up or down based on the application’s needs.

Customer Agility and Innovation – In a Cloud computing environment, new IT resources are only a click away, which means that customers reduce the time to make those resources available to developers from weeks to just minutes. This results in a dramatic increase in agility for the organization, since the cost and time it takes to experiment and develop is significantly lower.

Many companies who haven’t started their modernization journeys yet are looking for tools that allow their legacy mainframe environments to continue, while replicating data – in real time – on a variety of Cloud and open systems platforms. Treehouse Software is the worldwide distributor of tcVISION, a software tool that provides an easy and fast approach for Cloud and hybrid Cloud projects, enabling bi-directional data replication between the hardware source and many targets, including (mainframe): Db2 z/OS, Db2 z/VSE, Adabas, VSAM, IMS/DB, CA IDMS, CA DATACOM, etc. and (Cloud and open systems): AWS, Google Cloud, Microsoft Azure, Kafka, PostgreSQL, etc..

If your enterprise is planning on a Mainframe-to-Cloud data modernization project, we would welcome the opportunity to help get you moving immediately with an online demonstration of tcVISION. Contact Treehouse Software for a tcVISION demonstration today!

Enterprises with Vast Amounts of Data on Mainframe Systems are Perfectly Positioned to Take Advantage of Advanced Cloud-based Machine Learning and AI Technology

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

AI_02

Machine learning and artificial intelligence (AI) are not only revolutionizing the computing and business worlds, but the value proposition for Treehouse Software’s customer base who are replicating large amounts of mainframe data on the Cloud can be profound. Machine learning is proving to be a powerful force in improving customer experience, optimizing business operations, and accelerating innovation. This powerful, predictive technology helps improve customer engagement and conversion by creating personalized web experiences that are based on individual customer preferences and behaviors. Intelligent machine learning search services also boost business productivity and customer satisfaction by delivering accurate and useful information faster from various information sources across the organization. Additionally, business forecasting machine learning and AI tools can accurately predict demand and streamline supply-demand decisions by combining historical time series data with additional variables, such as product features, pricing, and holiday demand.

Treehouse Software is partnered with AWS, Google Cloud, and Microsoft, and has been in the mainframe market space since 1982. Our tcVISION product helps customers replicate their mainframe data, in real-time and bi-directionally, between a vast array of source databases and Cloud technologies. As soon as their mainframe data is made available within Cloud-based data stores, customers can use the wide variety of available machine learning and AI technologies.

Machine Learning and AI technologies are available at your fingertips… 

The dominant Cloud platforms are making incredible leaps in creating the most sophisticated AI tools available today — all literally at one’s fingertips. Never before has such powerful and useful technology been so easily available to so many, bringing the deepest set of machine learning services and supporting Cloud infrastructures instantly into the hands of developers, data scientists, etc. The following are a few examples of today’s most popular Cloud-based tools…

__AWS_On_White

Some examples of leading machine learning and AI product available on AWS:

  • Amazon SageMaker – A fully managed service that provides every developer and data scientist with the ability to build, train, and deploy ML models at scale.
  • AWS pre-trained AI Services – Provide ready-made intelligence for your applications and workflows.
  • More information on AWS AI and Machine Learning products

GoogleCloudPartner

Some examples of leading machine learning and AI product available on Google Cloud:

Microsoft

Some examples of leading machine learning and AI product available on Microsoft Azure:


The Journey Begins with Treehouse Software

__tsi_logo_400x200

Treehouse Software is here to help enterprise mainframe customers accelerate digital transformation and successfully leverage Hybrid Cloud initiatives on the IBM Z platform, storing sensitive data on a private cloud or local data center and simultaneously leveraging robust machine learning and AI resources on a managed public cloud.

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data replication for mainframe application modernization projects. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both Cloud and on-premises.

tcVISION_Overall_Diagram_Cloud_OS

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.


Further reading: tcVISION Mainframe-to-AWS data replication is featured on the AWS Partner Network Blog…

The AWS Partner blog talks about tcVISION’s Mainframe-to-AWS data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe Db2 z/OS data on Amazon Aurora. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.

Starting a Mainframe Data Replication Project? Consider Your Use Cases Carefully

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software

___Data_Center_To_Targets_Overview

Planning for Real-time and Bi-directional Mainframe Data Replication with tcVISION

A customer’s mainframe data may be utilized by many interlinked and dependent programs that have been in place for several years, and sometimes, decades, so unlocking the value of this legacy data can be difficult. Therefore, careful planning must occur for a mainframe data modernization project, beginning with identifying uses cases for the project and a Proof of Concept (POC) of data replication software, such as tcVISION, the Mainframe-to-Hybrid Cloud and Open Systems data replication product from Treehouse Software.

This blog serves as a general guide for organizations planning to replicate their mainframe data on Cloud and/or Open Systems platforms using tcVISION.

Questions and Considerations for Your Use Cases

A general principle should be to prove out the data replication technology (tcVISION) by using identified use cases. Listed below are some examples of questions and use cases to assist customers in planning and experiencing successful Mainframe-to-Cloud and/or Open Systems data replication projects:

  • What is/are the mainframe source database(s)? Obvious, yes, but the software solution vendor and outside consultants need this information.
  • What are the critical issues you need to test? Are there any areas that you believe that would be challenging for the vendor? Examples would be specific transformations, data types (e.g., BLOB), required CDC SLAs, field/column changes, specific security requirements, data volume requirements, etc. Document all of the critical test items.
  • Select the minimal set of files (generally 3-10) with representative conditions that will enable you to test all of your critical items. If there are multiple source databases, ensure specific test use cases are defined for each source.
  • What are the target databases? Will you be replicating to a Cloud database manager, such as AWS RDS? Are there additional requirements to replicate to S3, Azure BLOB, GCP Cloud Storage, Kinesis, and Kafka? What needs to be tested?
  • What are your bulk or initial load requirements? tcVISION can load data directly from mainframe databases, mainframe unloads, or image copies. What are the data volumes? Do you have sufficient bandwidth between your mainframe manager and on-premises or Cloud VM to handle your volume requirements?
  • What are your Change Data Capture (CDC) requirements?
  • Do you have plans for bi-directional replication in the future. What are your specific requirements? Since bi-directional replication can be complex and greatly lengthen a modernization project, the customer generally will perform one bi-directional use case for conceptual proof. Will this suffice for your organization?
  • What are your specific high availability requirements? Can they be handled by a technical discussion, or is a specific use case required?
  • What are your general security requirements for data at rest and data in transit? Do you have any specific security regulations to follow, such as HIPPA or FIPS? What are your PII / data masking requirements?
  • What are your schema requirements? For example, tcVISION creates a default schema based on your input mainframe data. Major changes to the default schema usually require a staging database.
  • Do you have staff available to perform the required tasks for the project? For example, for the length of a tcVISION POC you will need part-time staff, 2-4 hours per day. A part-time mainframe administrator will generally require 2-8 elapsed hours. Other staff will include Windows/Linux/Cloud administrators. 2-4 hours of project management may also be required.
  • Are business data transformations required? tcVISION handles minor transformation via point and click (e.g., date format transformations). Major transformations can require C++ or product scripting.
  • Are there any triggers or stored procedures? tcVISION performs CDC replication processing using a database that utilizes these database features.

Of course, each project will have unique environments, goals, and desired use cases. It is important that specific use cases are determined and documented prior to the start of a project and a tcVISION POC. This planning will allow the Treehouse Software team and the customer develop a more accurate project timeline, have the required resources available, and realize a successful project. 

More About tcVISION from Treehouse Software…

__Plans_To_Reality

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both Cloud and on-premises.

tcVISION acquires data in bulk or via CDC methods from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, and sequential files), and transform and deliver to a wide array of Cloud and Open Systems targets, including AWS, Google Cloud, Microsoft Azure, Confluent, Kafka, PostgreSQL, MongoDB, etc. In addition, tcVISION can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix, and PostgreSQL.


__TSI_LOGO

Contact Treehouse Software for a tcVISION Demo Today…

Simply fill out our Product Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.

What’s Your Mix?

Real-time and Bi-directional Data Replication Between Mainframes and Virtually Any Target

Cloud004_Swirl

tcVISION from Treehouse Software — Enterprise ETL and Real-Time Data Replication Through Change Data Capture (CDC)

Planning a data replication project between Mainframe, Cloud, and Open Systems platforms? tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both Cloud and on-premises.

tcVISION acquires data in bulk or via CDC methods from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, and sequential files), and transform and deliver to a wide array of Cloud and Open Systems targets, including AWS, Google Cloud, Microsoft Azure, Confluent, Kafka, PostgreSQL, MongoDB, etc. In addition, tcVISION can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix and PostgreSQL.

Whatever your mix, Treehouse Software has got it covered with tcVISION.


__TSI_LOGO

Moving the right data to the right place at the right time

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Quickly Begin Replicating Mainframe Data on Cloud and Open Systems During a tcVISION Proof of Concept.

by Joseph Brady, Director of Business Development / Cloud Alliance Leader at Treehouse Software, Inc.

___Start_Moving_Now_Graphic

Customers can start moving mainframe data within days during a tcVISION POC…

An online tcVISION Proof of Concept (POC) is approximately 10 business days, with the customer providing a representative subset of data, use cases, and goals for the POC. A Treehouse Software consultant will assist in downloading and installing tcVISION, and conduct a limited-scope implementation of a tcVISION application. This application uses customer data and executes on customer facilities, in a non-production environment. A document is provided beforehand that outlines the requirements and agenda for the POC.

By the end of the POC, customers can begin replicating mainframe data to their Cloud or Open Systems target database.  It can happen that fast!.

About tcVISION

More Cloud, Open Systems, and Systems Integration partners are recommending tcVISION, Treehouse Software’s Mainframe-to-Cloud data replication product for modernization projects. tcVISION focuses on changed data capture (CDC) when transferring information between mainframe data sources and Cloud and Open System databases and applications. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to a variety of RDBMS and other targets.

tcVISION_Overall_Diagram_Cloud_OS

Further reading…

Treehouse Software is an AWS, Google Cloud, and Microsoft Technology Partner, and the AWS Partner Network published a blog about tcVISION, which describes how tcVISION allows legacy mainframe environments to continue, while replicating data on highly available and secure Cloud platforms.


__TSI_LOGO

Contact Treehouse Software for a tcVISION Demo Today…

Simply fill out our Product Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.

Treehouse Software is Helping Government Agencies with Mainframe Adabas Data Take Advantage of Cutting Edge Cloud-based Technologies

by Joseph Brady, Director of Business Development / Cloud Alliance Lead at Treehouse Software, Inc.

___Gov_IT_Cloud

Many government agencies have large volumes of mission critical and historical data stored in a variety of legacy mainframe databases (for this blog, we are focusing on Adabas). These government agencies combine a broad range human service programs, including employment assistance and job training, child and adult protection, child support enforcement, cash assistance, and services for the developmentally disabled, all of which constantly require accurate, up-to-date, and secure data.

To support the needs of clients, government agencies are generally broken into multiple divisions, including the division that Treehouse Software works with the most — Technology Services. The Technology Services divisions provide technical and systems services for the development, maintenance, and enhancement of automated business systems. They also ensure that the production and sub-production databases are running smoothly and efficiently by performing necessary maintenance of the databases.

Rapidly changing national health and economic conditions are making fast access to the most current information more important than ever for government agencies and the public.  As a result, a top priority for Technology Services divisions is modernizing critical data residing on long-standing mainframe databases. Unlocking the value of this important data can be difficult, because the data can be utilized by numerous interlinked and dependent programs that have been in place for many years, and sometimes decades.

Many Treehouse government customers are now looking for modernization solutions that allow their legacy mainframe environments to continue, while replicating data in real time on highly available Cloud-based platforms, such as AWS, Google Cloud, and Microsoft Azure. tcVISION from Treehouse Software allows a “data-first” approach, whereby immediate data replication to the Cloud helps government agencies begin strategies to meet spikes in demand for vital information, especially in times of crisis.

Just what is it about Adabas?

Adabas is a mainframe database that is still heavily used by government sites throughout the U.S. and the world. Having specialized in tools and services complementary to Adabas/Natural applications since 1982, Treehouse has successfully encountered and addressed many unique issues within the Adabas environment. This excerpt from a Treehouse technical document outlines three primary issues with Adabas/Natural that must be considered:

  1. Adabas has no concept of “transaction isolation”, in that a program may read a record that another program has updated, in its updated state, even though the update has not been committed.  This means that programmatically reading a live Adabas database—one that is available to update users—will almost inevitably lead to erroneous extraction of data.  Record modifications (updates, inserts and deletes) that are extracted, and subsequently backed out, will be represented incorrectly—or not at all—in the target. Because of this, at Treehouse we say “the only safe data source is a static data source”—not the live database.
  2. Many legacy Adabas applications make use of “record typing”, i.e., multiple logical tables stored in a single Adabas file.  Often, each must be extracted to a separate table in the target RDBMS.  The classic example is that of the “code-lookup file”.  Most shops have a single file containing state codes, employee codes, product-type codes, etc.  Records belonging to a given “code table” may be distinguished by the presence of a value in a particular index (descriptor or superdescriptor in ADABAS parlance), or by a range of specific values.  Thus, the extraction process must be able to dynamically assign data content from a given record to different target tables depending on the data content itself.
  3. Adabas is most often used in conjunction with Software AG’s Natural 4GL, and “conveniently” provides for unique datatypes (“D” and “T”) that appear to be merely packed-decimal integers on the surface, but that represent date or date-time values when interpreted using Software AG’s proprietary Natural-oriented algorithm. The most appropriate way to migrate such datatypes is to recognize them and map them to the corresponding native RDBMS datatype (e.g., Oracle DATE) in conjunction with a transformation that decodes the Natural value and formats it to match the target datatype.

For discovery and analysis of legacy source data structures, tcVISION’s modeling and mapping facilities view and capture logical Adabas structures, as documented in Software AG’s PREDICT data dictionary, as well as physical structures as described in Adabas Field Definition Tables (FDTs).  Note that PREDICT is a “passive” data dictionary—there is no requirement that the logical and physical representations agree, so it is necessary to scrutinize both to ensure that the source structures are accurately modeled.

Furthermore, tcVISION generates specification and implementation of appropriate mappings and transformations for converting Adabas datatypes and structures to corresponding RDBMS datatypes and structures, including automatic handling of the proprietary “D” and “T” source datatypes.

There are three ways tcVISION can access Adabas data:

  1. ETL – read the active database nucleus
  2. ETL – read datasets containing unloaded Adabas files created by the ADAULD utility
  3. CDC – read the active and archived PLOGs datasets

It is important to note that the schema, mappings and transformations that result from metadata import can be tailored to any specific requirements after the fact.  It is even possible to import an existing RDBMS schema and retrofit it, via drag-and-drop, to the source Adabas elements.

Using the tcVISION Control Board, a Windows GUI interface, the most complex transformations can be specified. Source fields can be combined into a single column, decomposed into separate columns, and be subject to calculations, database lookups, string operations and even programmatic manipulation. Furthermore, mapping rules can be implemented to specify that data content from a source Adabas record be mapped to one or more target RDBMS tables—each with its own different structure, as desired—based on the data content itself. Target tables can even be populated from more than one source file.

tcVISION supports complex replication scenarios…

tcVISION_Complex_Replication_Scenarios

tcVISION GUI Control Board functions as a central point of administration…

tcVISION_Control_Board

Automatically apply target schema within the Control Board…

tcVISION_Target_Schema

It is impossible to discuss all the features and capabilities of tcVISION within a high-level overview.  Given the maturity, wealth of functionality and relative low cost of tcVISION, as compared to the effort, complexity and risk entailed in a “Do-It-Yourself”, solution there is no reason why a legacy renewal project should run aground on data migration.

tcVISION’s minimal footprint on the mainframe…

Customers are very happy with tcVISION‘s “staged processing” methodology, where the only processing occurring on the mainframe was the capture of changes from Adabas PLOGs. The bulk of the processing occurs on the target platform, minimizing tcVISION’s footprint on the mainframe as seen in this diagram…

tcVISION_Staged_Processing

The user defines on which platform stage their processing should be done. Do as little as possible on the mainframe: Stage 0 – capture data and send data (internal format) to target, and process data up to Stage 3 in receiving environment.

Moving forward with tcVISION…

Treehouse Software has been helping local, state, and federal government agencies with Adabas and Natural in the areas of data migration, security, control, auditing, performance enhancement, etc. for decades. Over the years, Treehouse has expanded its capabilities to address new requirements for modernizing legacy mainframe Adabas databases on various Cloud platforms. By using Treehouse Software’s tcVISION Mainframe-to-Cloud data replication product, our government customers are able to immediately utilize some of the most advanced Cloud tools and services in the world.

tcVISION enables government agencies to synchronize mainframe Adabas data with various highly available and secure Cloud databases, data warehouses. etc.. Additionally, bi-directional, real-time data synchronization will enable changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected back on the mainframe database). This allows governments to modernize  applications on Cloud platforms without disrupting the existing critical work on the legacy system, and modern tools can now be used in the new environment, greatly enhancing agility.

Replicating mainframe data on the Cloud can happen within days during a tcVISION Proof of Concept (POC)…

tcVISION_Overall_Diagram_General_Cloud01

An online tcVISION POC is approximately 10 business days, with the customer providing use case and goals for the POC. A Treehouse Software consultant will assist in downloading and installing tcVISION and conducting a limited-scope implementation of a tcVISION application. This application uses customer data and executes on customer facilities, usually in a non-production environment. A document is provided beforehand that outlines the requirements, use cases, and agenda for the POC.

By the end of the 10-day POC, customers can begin replicating mainframe data to their Cloud target database.  It can happen that fast!

Further Reading…

_0_AWS_Logo

Treehouse Software is an AWS Technology Partner, and the AWS Partner Network published a blog about tcVISION, our Mainframe-to-Cloud data replication product, which describes how tcVISION allows legacy mainframe environments to continue, while replicating data on highly available and secure Cloud platforms:

https://aws.amazon.com/blogs/apn/real-time-mainframe-data-replication-to-aws-with-tcvision-from-treehouse-software/


__TSI_LOGO

Contact Treehouse Software for a tcVISION Demo Today…

Simply fill out our Product Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.