High Availability Requirements for Mainframe Data Modernization — Running tcVISION in Global Availability Zones

by Joseph Brady, Director of Business Development / Cloud Alliance Lead at Treehouse Software, Inc.

Many customers embarking on Mainframe-to-Cloud data replication projects with Treehouse Software are looking at high availability (HA) as a key consideration in the planning process. The goal with HA is to ensure that systems are always functioning and accessible, with deployments located in various Availability Zones (AZs) worldwide. Having an HA architecture in place protects against data center, availability zone, server, network, and storage subsystem failures to keep businesses running without downtime or human intervention.

In this blog, we will give a high-level overview of tcVISION HA Architecture, using AWS as an example. However, HA basic principles are essentially the same across all Cloud platforms.

Example of the tcVISON HA Architecture on AWS

During tcVISION’s Change Data Capture (CDC) processing for Mainframe-to-Cloud data replication, HA must be maintained in the AWS environment. The Amazon Elastic Compute Cloud (Amazon EC2), which contains the tcVISION Manager, is part of an Auto Scaling Group that is spread across AZs with Amazon EC2 instance(s).

___tcVISION_AWS_HA_Architecture

Upon failure, the replacement Amazon EC2 instance tcVISION Manager is launched and communicates its IP address to the mainframe tcVISION Manager. The mainframe tcVISION Manager then starts communication with the replacement Amazon EC2 tcVISION Manager.

Once the Amazon EC2 tcVISION Manager is restarted, it continues processing at its next logical restart point, using a combination of the LUW and Restart files. LUW files contain committed data transactions not yet applied to the target database. Restart files contain a pointer to the last captured and committed transaction and queued uncommitted CDC data. Both file types are stored on a highly available data store, such as Amazon Elastic File System (EFS).

For production workloads, Treehouse Software recommends turning on Multi-AZ target and metadata databases.

To keep all the dynamic data in an HA architecture, tcVISION uses EFS, which provides a simple, scalable, fully managed elastic file system for use with AWS Cloud services and on-premises resources. It is built to scale on-demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth.


Treehouse Software can help organizations immediately start moving their mainframe data to the Cloud and take advantage of the most advanced, scalable, secure, and highly available technologies in the world with tcVISION

tcVISION_Overall_Diagram_General_Cloud

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Cloud, Open Systems, Linux, Unix, and Windows platforms.

View the Unequalled List of Environments Supported by tcVISION Here


__TSI_LOGO

___AWS_Select_Partner_Badge ___Google_Cloud_Partner_Badge

Contact Treehouse Software for a Demo Today…

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION demonstration. This will be a live, on-line demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database.

Now, more than ever, enterprises with mainframes are looking to modernize their legacy systems

by Joseph Brady, Director of Business Development / Cloud Alliance Lead at Treehouse Software, Inc.

Rapidly changing global health, economic, and political conditions are making fast access to the most current information more important than ever for official agencies and the public.  As a result, modernizing information systems is taking center stage and top priority, especially for organizations with critical mainframe data residing on a variety of long-standing databases, often still updated by COBOL applications! These databases include Db2, VSAM, IMS/DB, Adabas, IDMS, Datacom, or even sequential files. Unlocking the value of this important data can be difficult, because the data can be utilized by numerous interlinked and dependent programs that have been in place for many years, and sometimes decades.

Many organizations are now looking for modernization solutions that allow their legacy mainframe environments to continue, while replicating data in real time on highly available Cloud-based platforms (AWS, Google Cloud, Microsoft Azure, etc.). With a “data-first” approach, immediate data replication to the Cloud is enabling government, healthcare, supply chain, financial, and a variety of public service organizations to meet spikes in demand for vital information, especially in times of crisis.

Treehouse Software can help organizations immediately start moving their mainframe data to the Cloud and take advantage of the most advanced technologies in the world with tcVISION

tcVISION_Overall_Diagram_General_Cloud

Whether an enterprise needs to take advantage of the latest Cloud services, such as big data analytics, artificial intelligence (AI), rapid global database deployments, high-level security, etc., or move data to a variety of newer Cloud or Open Systems databases, the transition doesn’t have to be a sudden big bang.

A Phased Approach

Treehouse Software has extensive mainframe experience and subject matter experts to help organizations incrementally replicate their mainframe data to the Cloud and other modern systems, while keeping both sides synchronized.

Treehouse Software’s expert technical representatives help customers develop a phased plan that includes installation and implementation of the tcVISION mainframe data modernization product, script customization, data replication mapping, high availability, security, monitoring, training, etc..

After defining the architecture, the production deployment phase begins with incremental sprint-like deployments. Additional files are then deployed into production regularly.

This phased plan enables tcVISION to synchronize critical mainframe data to a Cloud / Open Systems database. Bi-directional, real-time data synchronization allows changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected back on mainframe). The customer can then modernize their application on the cloud, open systems, etc. without disrupting the existing critical work on the legacy system.

Additionally, tcVISION customers see drastically reduced mainframe MIPS costs, and increased ability to quickly respond to business environment changes.

Enterprise ETL and Real-time and Bi-directional Data Replication Through Change Data Capture with tcVISION

tcVISION uses an intuitive Windows GUI interface for administration, mapping and modeling, script generation, and monitoring. The product focuses on changed data capture (CDC) when transferring information between mainframe data sources and modern databases and applications. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to a variety of targets.

tcVISION – Supported Sources and Targets

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Cloud, Open Systems, Linux, Unix, and Windows platforms.

View the Unequalled List of Environments Supported by tcVISION Here


__TSI_LOGO

___AWS_Select_Partner_Badge ___Google_Cloud_Partner_Badge

Contact Treehouse Software for a Demo Today…

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION demonstration. This will be a live, on-line demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database.

STAY AND GO… AT THE SAME TIME! Incrementally Replicate Your Mainframe Data to the Cloud and Other Modern Systems While Keeping Both Sides Synchronized

Is your mission critical mainframe data residing on IBM Db2, IBM VSAM, IBM IMS/DB, Software AG Adabas, CA IDMS, CA Datacom, or even sequential files? Are you looking for a solution that allows your legacy mainframe environment to continue while replicating data – in real time and bi-directionally – on a variety of newer Cloud, LUW, or Open Systems databases? Whether your enterprise wants to take advantage of the latest cloud services, such as analytics, artificial intelligence (AI), etc., or move your data to a variety of newer Cloud, LUW, or Open Systems databases, the transition doesn’t have to be a sudden big bang.

tcVISION_Overall_Diagram

tcVISION: Enterprise ETL and Real-time and Bi-directional Data Replication Through Change Data Capture

The globalization of markets, the increase of data volumes, 24×7 operations, changing business conditions and high demand for up-to-date information require new data transfer and exchange solutions for heterogeneous IT architectures.

The tcVISION solution focuses on changed data capture (CDC) when transferring information between mainframe data sources and modern databases and applications. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to a variety of targets.

tcVISION allows bi-directional, real-time data synchronization of changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected back on mainframe). The customer can then modernize their application on the cloud, open systems, etc. without disrupting the existing critical work on the legacy system.

tcVISION – Supported Sources and Targets

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Cloud, Linux, Unix, and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

View the Unequalled List of Environments Supported by tcVISION Here


__TSI_LOGO

Contact Treehouse Software for a Demo Today…

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION demonstration. This will be a live, on-line demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database.

Treehouse Software’s tcVISION on the AWS Marketplace… AWS Architectural Considerations and Challenges

Andy Jones, Senior Technical Representative for Treehouse Software recently gave a presentation at the Pittsburgh AWS User Group Meetup that explores our experiences in bringing the tcVISION Mainframe-to-AWS Data Replication Solution to the AWS Marketplace.  Andy discussed issues that had to be addressed, such as High Availability and AWS Availability Zones, Autoscaling, Security, creating the tcVISON AMIs, etc. Here is his presentation:

Additionally, Treehouse Software’s tcVISION is Featured on the AWS Partner Network Blog. Read the Blog Here.


__tsi_logo_400x200

Treehouse Software is an AWS Technology Partner and a global leader in providing real-time and bi-directional data replication between a variety of mainframe and non-mainframe sources, including (Mainframe – z/OS and VSE): VSAM, IMS, Db2 z/OS, Adabas, CA Datacom, and CA IDMS; and (Non-mainframe): AWS, Microsoft Azure, PostgreSQL, Oracle, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix, Kafka, MongoDB, Hadoop, SAP Hana, and more.

Contact us today to discuss your project!

Treehouse Software Customer Success: Element Fleet Management Corp. Real-time, Bi-directional Synchronization Between IBM Mainframe Datacom and PostgreSQL

Element_Building

As a leading global fleet management company, Element provides a comprehensive range of fleet services that span the total fleet lifecycle, helping customers optimize their strategies at every stage – from acquisition and financing to program management and vehicle remarketing. These services, together with exceptional tools, technologies and consulting, empower customers to realize extraordinary results.

Business Background

Element Fleet Management has over one million vehicles under management, and serves more than 2,800 customers. The company employs over 2,700 staff members across 12 offices. Transaction sizes range from $500,000 to $40 million. 50 countries are served through the Element-Arval Global Alliance. Top industries served by Element Fleet Management are agriculture, business services, chemical, construction, consumer products, education and non-profit, energy, food and beverage, insurance, manufacturing, pharmaceutical and healthcare, professional services, telecommunications, transportation, and utilities.

Business Issue

Much of Element’s mission critical mainframe data is stored in a legacy IBM mainframe database. The cost to maintain these legacy databases is high and they lack the features required for modernizing the data architecture. The data is utilized by an extensive number of interlinked programs dependent on this legacy data. Element required a solution that would allow the legacy environment to continue while having real time replicated data on a modern database/platform. This enables them to utilize modern programing and data tools to be competitive in their market.

Element_Diagram

Technology Solution

Treehouse software and the customer developed a phased plan to prove out and implement tcVISION. This started with a proof-of-concept to prove out the technology’s effectiveness and technical suitability. This was proceeded by architectural phase to define the architectural solution including, configuration management, environments and promotion, high availability, security, disaster recovery, monitoring, operations procedures, script customization, data replication mapping and training plans. After defining the architecture, the production deployment phase started with incremental sprint-like deployments. Additional files were deployed into production every few weeks.

This project enabled the tcVISION product to synchronize mission critical financial IBM mainframe Datacom data to a modern PostgreSQL database. Bi-directional, real-time data synchronization enables changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected on mainframe Datacom). This allows their business to modernize the application on PostgreSQL without disrupting the existing critical work on the legacy system.

NOTE: Many modern tools are using the PostgreSQL database, which greatly enhances business agility (e.g., other divisions within Element are already using Amazon Web Services (AWS), so having tcVISION in place now makes mainframe data delivery to the AWS cloud an easy option). See a step-by-step video of Mainframe-to-PostgreSQL on AWS here. 

Successful, real-time mainframe data replication to an open systems application allows web and mobile access and updates. Element indicated their satisfaction with the results and the solution, especially reduced mainframe MIPS costs, and enhanced business ability to respond to business environment changes.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

tcVISION_Connection_Overview_Web01

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including PostgreSQL, Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, and IBM Informix.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

 

PRODUCT SPOTLIGHT: tcVISION v6 Overview and Updates

tcVISONv6

Several exciting new features and updates are now in tcVISION v6, including new output targets Adabas LUW, DB2 BLU, EXASOL, Hadoop,and MongoDB. Additionally, new input sources include z/OS VSAM Logstream (CICS and Coupling Facility / Shared VSAM),z/OS VSAM Batch Extension, z/OS DBMS to Logstream, CA IDMS v17, CA Datacom CDC, IMS Active Log and SMF data.

Another feature recently announced is the tcVISION “Direct Loader” for BULK_LOAD processing. The function does not require output to a sequential file, and the loader utility for the target DBMS is called via API with data passed directly. Direct loader supports PostGreSQL, Microsoft SQL Server and DB2 LUW / DB2 BLU. The advantage of using the Direct Loader is the elimination of disk access in writing and reading the sequential loader data file. File output is still supported (e.g., where loader data is to be distributed to other machines).

Finally, as mentioned in a previous Treehouse Blog, with tcVISION v6 comes the newly enhanced web statistics functionality and web server. Any standard web browser can access this server (Internet Explorer, Firefox, Opera, Chrome, Safari, etc.)

This valuable feature enables users to view data from the tcVISION Manager Monitor, and statistical and operational information from the tcVISION Manager network.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

tcVISION TREETIP: Navigating The Repository Editor

When modeling and mapping a new metadata structure, tcVISION’s powerful Repository Editor is used to define and modify Input and Output Objects. The rich features within the Editor allow users to:

  • import new objects
  • copy or create new objects
  • import metadata for input and/or output objects
  • define target DBMSs
  • specify input and/or output objects for import
  • define DMBS-specific options
  • define target object names
  • replace/supersede existing object metadata

To assist new and even experienced tcVISION users, we’ve put together the following screen shots to give a quick overview of the Repository Editor’s functionality buttons / icons used for setting up modeling and mapping of sources and targets.

Here is an example of what the tcVISION Repository Editor looks like:

_0_tcVISION_RepositoryEditor01

Repository Editor icon definitions: Input objects

_0_tcVISION_RepositoryEditor02Input

Repository Editor icon definitions: Output objects

_0_tcVISION_RepositoryEditor02output

Repository editor right click menu options

_0_tcVISION_RepositoryEditor03OutputOptions

Repository editor extended output target options

_0_tcVISION_RepositoryEditor03RightClick


Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION_Architecture

tcVISION provides real-time data replication through change data capture, and allows easy and fast data migration for mainframe application modernization projects. Enterprises looking for a product that enables bi-directional heterogeneous data replication between mainframe, Linux, Unix, and Windows platforms need look no further than to tcVISION from Treehouse Software.

To learn more about tcVISION, or to request a demonstration, contact Treehouse Software today.