Starting a Mainframe Data Migration Project? Consider Your Use Cases Carefully

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software

___Data_Center_To_Targets_Overview

Planning for Real-time and Bi-directional Mainframe Data Replication with tcVISION

A customer’s mainframe data may be utilized by many interlinked and dependent programs that have been in place for several years, and sometimes, decades, so unlocking the value of this legacy data can be difficult. Therefore, careful planning must occur for a mainframe data modernization project, beginning with identifying uses cases for the project and a Proof of Concept (POC) of data replication software, such as tcVISION, the Mainframe-to-Cloud and Open Systems data replication product from Treehouse Software.

This blog serves as a general guide for organizations planning to replicate their mainframe data on Cloud and/or Open Systems platforms using tcVISION.

Questions and Considerations for Your Use Cases

A general principle should be to prove out the data replication technology (tcVISION) by using identified use cases. Listed below are some examples of questions and use cases to assist customers in planning and experiencing successful Mainframe-to-Cloud and/or Open Systems data replication projects:

  • What is/are the mainframe source database(s)? Obvious, yes, but the software solution vendor and outside consultants need this information.
  • What are the critical issues you need to test? Are there any areas that you believe that would be challenging for the vendor? Examples would be specific transformations, data types (e.g., BLOB), required CDC SLAs, field/column changes, specific security requirements, data volume requirements, etc. Document all of the critical test items.
  • Select the minimal set of files (generally 3-10) with representative conditions that will enable you to test all of your critical items. If there are multiple source databases, ensure specific test use cases are defined for each source.
  • What are the target databases? Will you be replicating to a Cloud database manager, such as AWS RDS? Are there additional requirements to replicate to S3, Azure BLOB, GCP Cloud Storage, Kinesis, and Kafka? What needs to be tested?
  • What are your bulk or initial load requirements? tcVISION can load data directly from mainframe databases, mainframe unloads, or image copies. What are the data volumes? Do you have sufficient bandwidth between your mainframe manager and on-premises or Cloud VM to handle your volume requirements?
  • What are your Change Data Capture (CDC) requirements?
  • Do you have plans for bi-directional replication in the future. What are your specific requirements? Since bi-directional replication can be complex and greatly lengthen a modernization project, the customer generally will perform one bi-directional use case for conceptual proof. Will this suffice for your organization?
  • What are your specific high availability requirements? Can they be handled by a technical discussion, or is a specific use case required?
  • What are your general security requirements for data at rest and data in transit? Do you have any specific security regulations to follow, such as HIPPA or FIPS? What are your PII / data masking requirements?
  • What are your schema requirements? For example, tcVISION creates a default schema based on your input mainframe data. Major changes to the default schema usually require a staging database.
  • Do you have staff available to perform the required tasks for the project? For example, for the length of a tcVISION POC you will need part-time staff, 2-4 hours per day. A part-time mainframe administrator will generally require 2-8 elapsed hours. Other staff will include Windows/Linux/Cloud administrators. 2-4 hours of project management may also be required.
  • Are business data transformations required? tcVISION handles minor transformation via point and click (e.g., date format transformations). Major transformations can require C++ or product scripting.
  • Are there any triggers or stored procedures? tcVISION performs CDC replication processing using a database that utilizes these database features.

Of course, each project will have unique environments, goals, and desired use cases. It is important that specific use cases are determined and documented prior to the start of a project and a tcVISION POC. This planning will allow the Treehouse Software team and the customer develop a more accurate project timeline, have the required resources available, and realize a successful project. 

More About tcVISION from Treehouse Software…

__Plans_To_Reality

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both Cloud and on-premises.

tcVISION acquires data in bulk or via CDC methods from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, and sequential files), and transform and deliver to a wide array of Cloud and Open Systems targets, including AWS, Google Cloud, Microsoft Azure, Confluent, Kafka, PostgreSQL, MongoDB, etc. In addition, tcVISION can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix, and PostgreSQL.


__TSI_LOGO

Contact Treehouse Software for a tcVISION Demo Today…

Simply fill out our Product Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.

What’s Your Mix?

Real-time and Bi-directional Data Replication Between Mainframes and Virtually Any Target

Cloud004_Swirl

tcVISION from Treehouse Software — Enterprise ETL and Real-Time Data Replication Through Change Data Capture (CDC)

Planning a data replication project between Mainframe, Cloud, and Open Systems platforms? tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both Cloud and on-premises.

tcVISION acquires data in bulk or via CDC methods from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, and sequential files), and transform and deliver to a wide array of Cloud and Open Systems targets, including AWS, Google Cloud, Microsoft Azure, Confluent, Kafka, PostgreSQL, MongoDB, etc. In addition, tcVISION can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix and PostgreSQL.

Whatever your mix, Treehouse Software has got it covered with tcVISION.


__TSI_LOGO

Moving the right data to the right place at the right time

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Where Do You Want to Go?

Enterprise ETL and Real-Time, Bidirectional Data Replication Between Virtually Any Source and Target.

Mainframe_to_Cloud

Planning a data replication project between mainframe, Linux, Unix, and Windows platforms? We now have downloadable data sheets that individually focus on supported data replication sources and targets.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

___tcVISON_Big_Data_002

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

 

Treehouse Software Eases the Challenges for Mainframe-to-Modern Platform Transformation Programs for Systems Integrators

Unlock Your Customers’ Business Value Faster with Frequent Production Deliveries, Using Agile Methods

Treehouse Software’s product, tcVISION has proven to be vital for the ease and success of many mainframe modernization initiatives.

A big problem for a Systems Integrator (SI), or any mainframe modernization vendor, is the myriad of dependencies between mainframe and open systems data and components. As part of a mainframe modernization project, these vendors need to determine the ordering of phases or waves of applications to be migrated. There are often shared databases that can make it very difficult to move one set of applications without moving others. This can result in the conclusion that they must go with a “big bang” approach, which can lead to a bid for a program with one large delivery at the end with all of the integrated components in place. This is very unappealing to customers as they are paying out big dollars without the corresponding business value of a production deployment. This approach also significantly increases the project risk.

___Mainframe_To_LUW

With Treehouse Software’s bi-directional replication product, tcVISION, a subset of applications can be migrated, and tcVISION’s data synchronization capabilities can be used to bridge specific shared legacy mainframe databases to the modern database versions on open systems. This allows them to deliver in phases, thus getting modernized modules in production faster. In addition, it decreases risk as issues can be found much faster with the more frequent production deployments. When the last modules are deployed to production, they would no longer need the synchronization.

As an example, a large Treehouse customer is currently involved in gradually modernizing and replacing a set of applications, and is using a similar approach. The customer will be using their mainframe Adabas database synced with Oracle on Linux, while in the process of modernizing the applications. Treehouse Software’s experts and tcVISION product are making sure the customer’s transition goes smoothly and seamlessly.


Enterprise ETL and Real-Time Data Replication Between Virtually Any Source and Target

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

_0_tcvision_connection_overview

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Hadoop/HDFS, MongoDB, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix, PostgreSQL, etc.


__tsi_logo_400x200

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

TREETIP: Integrate Mainframe Data Sources In Your Big Data Initiatives

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

Analysts have observed that perhaps 80 percent of the world’s corporate data still resides on mainframes. So it’s no surprise that Bloor Research (http://www.bloorresearch.com/research/spotlight/big-data-and-the-mainframe/), notes that “it is necessary today to place the mainframe as a ‘first-class player’ in any enterprise Big Data strategy.”

In February 2017 we highlighted tcVISION’s support for replication to the leading NoSQL database MongoDB. MongoDB continues to increase in popularity as a back end for operational applications with real-time requirements.

tcVISION also supports analytics and “mainframe offload” Big Data use cases that generally leverage Hadoop HDFS and/or streaming data transport. With tcVISION, data from a wide variety of IBM mainframe data source can be quickly and easily replicated to Big Data targets, requiring minimal mainframe know-how and having minimal impact on the mainframe.

___tcVISON_Big_Data_001

Boost the return on investment for your Big Data initiatives using tcVISION!


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix, and PostgreSQL.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

TREETIP: tcVISION Allows for Surprisingly Innovative Uses

tcVISONv6

Treehouse Professional Services consultants help Adabas / Natural customers in a variety of ways, including DBA services, performance tuning, change management implementation and training, and data replication planning and training. Our experience and long history of service to the Adabas / Natural community helps us create innovative solutions for our customers’ challenges.

Recently Treehouse Senior Technical Representative Chris Rudolph assisted a customer with a tricky data replication problem. The customer uses tcVISION to perform bi-directional replication between Adabas and an RDBMS during the phase-in of a new application. Unfortunately, the new application incorrectly updated certain columns in the RDBMS, which were then replicated to Adabas. The customer attempted to address the issue by running a series of ADASEL reports against the Adabas PLOG and manually checking for “bad” transactions, which was a very time consuming process that pulled the Adabas DBA away from her normal duties.

Chris explained that tcVISION could expedite the process by replicating all transactions for the Adabas file to a journal table capturing the “before” and “after” values of the problematic columns. The developers working on the new application could then identify invalid values, correct the application, and patch the data themselves. This also allowed the Adabas DBA to return to their normal duties.

The journal table now includes columns to display the “before” and “after” values of the corrupted column, Adabas transaction time, end transaction time, operation and Adabas userid. The customer’s developers immediately recognized immense value from being able to query the journal table to find bad data, patch the data, prove that corruption is no longer taking place, and verify that all corrupted instances of the data have been patched. Journal tables have been added for all replicated Adabas files, and the developers now rely on the journal tables for all of their data patches.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.