A real-world example of how Treehouse Software is helping enterprises with their Mainframe-to-Cloud / LUW / Open Systems data migration and replication initiatives…

by Joseph Brady, Director of Business Development for Treehouse Software

Many medium to large size companies are still investing in new mainframe systems, because they have been relying on them for decades to handle mission critical data transaction processing. For these companies, their mainframes are deeply entrenched, and have been rock-solid reliable over the years, allowing many users and applications to simultaneously access the same data without interfering with each other. Mainframes also perform large-scale and extremely fast transaction processing (thousands of transactions per second). Chances are, most transactions for which you use your debit or credit cards have a mainframe processing the data in the background.

There have been countless predictions of the demise of the mainframe over the years, but they have been proven wrong every time, and there’s no sign of extinction any time soon. Mainframes remain a vital part of IT infrastructures used by companies spanning a variety of industries around the world, including banking, insurance, healthcare, government, aviation, and retail.

Today, most companies that have their enterprise data stored on a mainframe system are discovering the advantages of newer cloud and open systems databases and services, including instantaneous global database deployments (e.g., Amazon Aurora), economics to scale, high availability, elasticity, security, testing at scale, disaster recovery, etc. Consequently, these companies are looking for an automated solution that allows real-time data replication between the two environments, and many are turing to Treehouse Software.


_0_Airline_Manitenance

Customer Scenario: A Look at a Large Airline’s Mainframe Data Replication Project

Treehouse Software has a major airline customer that is a long-time user of Software AG’s Adabas database on their mainframe system. Their desire was to move their data to more modern applications off of the mainframe. However, their mainframe contains vital airline maintenance and parts data that must always be highly available, so rather than trying to move off of their mainframe, the customer decided on utilizing a data warehouse with a requirement of real-time data replication. This allows the mainframe legacy teams to maintain existing applications, while the modernization team develops applications on the newly created RDBMS. All teams determined that tcVISION real-time mainframe data replication was the perfect fit for meeting their goals.

How tcVISION Addressed the Customer Needs...

The high-level environment for the airline customer using tcVISION for data replication looks like this…

___Airline_tcV_Overview

  • Adabas: Mainframe data source containing business critical information replicated to the RDBMS.
  • Oracle: Open Platform RDBMS chosen by customer as replication target for both data warehouse and modernization project. The tcVISION Manager also uses Oracle as a repository for the metadata (field mappings).
  • tcVISION Manager: Software component deployed on both source and target systems. It is responsible for provisioning resources for:
    • Processing scripts
    • Metadata import
    • Scheduling
  • tcSCRIPT: Software component deployed on both source and target systems. It works in conjunction with the tcVISION Manager to:
    • Perform initial load of Adabas into Oracle
    • Ongoing near realtime CDC (Change Data Capture) and replication from Adabas to Oracle
    • tcSCRIPT processes data from:
      • Direct from the DBMS (initial load) or from DBMS extracts
      • Adabas Protection log (PLOG)
  • tcVISION Control Board: Software component deployed on a Windows machine which provides graphical user interface for a single point of control to administer the tcVISION environment:
    • Metadata import, creates metadata rules governing relationship between mainframe and open platform data structures
    • Replication rule maintenance
    • DDL creation
    • Creation of ETL and replication processes
    • Start, stop, and schedule replication processes

Customer’s Technical Requirement for tcVISION

The airline requires that the replication solution must support online and batch Adabas transactions and provide data replication between Adabas and Oracle. Additionally, their large databases contain millions of rows (e.g., > 50 million) that must be supported, as well as support for database change audit requirements (datetime and type of operation), transaction management, and notification of exception events.

There must also be support for configuration management between development, QA, and production.

Finally, the solution must address hardware and software failures on the mainframe, Linux, and database nodes.

Customer Outcomes

All requirements were met by tcVISION, which led to a successful project implementation.  Here is a look at the customer outcomes and benefits:

Business Outcome Customer Benefit
Data warehouse is always in sync with Adabas using tcVISION data replication Improved timeliness and reliability of reporting data
Reduced usage of mainframe MIPS Reduced cost
All data required for modernization project was successfully replicated to target environment Increased business efficiently and agility

Many more mainframe data migration and replication customer case studies can be read on the Treehouse Software Website.


__TSI_LOGO

Contact Treehouse Software for a Demo Today…

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer. 

_0_Treehouse_tcV_Cloud_OpenSystems

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.


Further reading: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software

Treehouse Software’s Mainframe-to-AWS Data Replication Solution is Featured on the AWS Partner Network Blog

“We are very excited about the APN blog showcasing tcVISION’s Mainframe-to-AWS capabilities to such a substantial and relevant audience. The AWS partner and customer base not only has access to our solution via the tcVISION AMI in the AWS Marketplace, they can now use the tcVISION blog as a valuable educational and marketing tool!”

– Joseph Brady, Director of Business Development and AWS Alliance Lead at Treehouse Software

READ THE BLOG HERE


__TSI_LOGO

Since 1982, Treehouse Software has been serving enterprises worldwide with industry-leading software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data migration, replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, business intelligence and analytics, and application modernization.

U.S. Government Customers Can Now Use Treehouse Software’s tcVISION Mainframe Data Replication on the AWS GovCloud

Treehouse Software’s tcVISION Mainframe-to-AWS data replication product is now available in the AWS GovCloud (US) Region of the AWS Marketplace.

The AWS GovCloud (US) Region is designed to host sensitive data, regulated workloads, and address the most stringent U.S. government security and compliance requirements.

The AWS GovCloud (US) is available to government customers, organizations in government-regulated industries, and other private entities that meet AWS GovCloud (US) requirements.

AWS GovCloud (US) gives government customers and their partners the flexibility to architect secure cloud solutions that comply with: the FedRAMP High baseline, the DOJ’s Criminal Justice Information Systems (CJIS) Security Policy, U.S. International Traffic in Arms Regulations (ITAR), Export Administration Regulations (EAR), Department of Defense (DoD) Cloud Computing Security Requirements Guide (SRG) for Impact Levels 2, 4 and 5, FIPS 140-2, IRS-1075, and other compliance regimes.

From Personally Identifiable Information (PII), sensitive patient medical records, and financial data to law enforcement data, export controlled data and other forms of CUI, AWS GovCloud (US) Regions can help customers address compliance at every stage of their cloud journey.


_0_AWS_Marketplace_Logo01

The tcVISION Amazon Machine Images (AMIs) are available on the AWS Marketplace. A tcVISION AMI automatically sets up the product on the AWS Cloud and allows customers to use it with either backup/recovery mainframe files or Windows files, or to set up connectivity for real-time, on-premises mainframe data replication to AWS. An enterprise with mission critical data in a mainframe environment can rapidly deploy databases globally within minutes, and with minimal administration requirements. U.S Government sites can simply choose the region “AWS GovCloud (US)” under “Pricing Information” when choosing and AMI.

_0_tcV_To_AWS_Cloud

There are three versions of the tcVISION AMI on the AWS Marketplace, and #1 and #3 require two-week trial license keys provide by Treehouse Software:

1. tcVISION Mainframe Batch Integration (MBI) for AWS – Bring Your Own License (Customers must contact Treehouse Software for license key to run the software): This AMI does not require an active connection to the mainframe, because the existing backup and recovery files from the mainframe, such as Db2 imagecopies, z/OS archive log files or IMS Unloads, Adabas unload, and PLOG files can be used.

2. tcVISION Distributed Database Integration (DDI) for AWS – Pay as You Go on AWS: This AMI license installation includes security keys for Amazon S3 access and a license to use one source (PostgreSQL, Oracle, MySQL, MariaDB; MSSQL on Windows OS only) and one target (AWS S3, AWS Aurora, RDS – PostgreSQL, Oracle, MySQL, MariaDB; MSSQL on Windows OS only).

3. tcVISION Enterprise Change Data Capture (CDC) Integration for AWS – Bring Your Own License (Customers must contact Treehouse Software for license key to run the software): This AMI allows transfer of mainframe data to AWS targets continuously and in real-time. Using this tcVISION AMI, a customer can deploy databases globally within minutes.


About the AWS Marketplace…

_0_AWS_Marketplace_Logo01

The AWS Marketplace is an online software store that helps customers find, buy, and immediately start using the software and services that run on AWS.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Try out Mainframe Data Replication on the AWS Cloud with tcVISION Amazon Machine Images (AMIs)

Treehouse Software is pleased  to announce that the tcVISION Amazon Machine Images (AMIs) are now officially released on the AWS Marketplace. The tcVISION AMI automatically sets up the product on the AWS Cloud and allows customers to use it with either backup/recovery mainframe files or Windows files, or to set up connectivity for real-time, on-premises mainframe data replication to AWS. An enterprise with mission critical data in a mainframe environment can rapidly deploy databases globally within minutes, and with minimal administration requirements.

_0_tcV_To_AWS_Cloud

There are three versions of the tcVISION AMI on the AWS Marketplace, and #1 and #3 require two-week trial license keys provide by Treehouse Software:

1. tcVISION Mainframe Batch Integration (MBI) for AWS – Bring Your Own License (Customers must contact Treehouse Software for license key to run the software): This AMI does not require an active connection to the mainframe, because the existing backup and recovery files from the mainframe, such as Db2 imagecopies, z/OS archive log files or IMS Unloads, Adabas unload, and PLOG files can be used.

2. tcVISION Distributed Database Integration (DDI) for AWS – Pay as You Go on AWS: This AMI license installation includes security keys for Amazon S3 access and a license to use one source (PostgreSQL, Oracle, MySQL, MariaDB; MSSQL on Windows OS only) and one target (AWS S3, AWS Aurora, RDS – PostgreSQL, Oracle, MySQL, MariaDB; MSSQL on Windows OS only).

3. tcVISION Enterprise Change Data Capture (CDC) Integration for AWS – Bring Your Own License (Customers must contact Treehouse Software for license key to run the software): This AMI allows transfer of mainframe data to AWS targets continuously and in real-time. Using this tcVISION AMI, a customer can deploy databases globally within minutes.

tcVISION Offers a Wide Variety of Connectivity

_0_tcVISION_Connection_Overview01

tcVISION supports AWS Relational Database Service (RDS) databases, such as Aurora, PostgreSQL, MySQL, SQL Server, and Maria DB as targets, in addition to traditional Linux/Unix/Windows (LUW) targets such as Oracle, IBM Db2 LUW, Software AG Adabas LUW, IBM Informix, Sybase, Microsoft SQL Server, PostgreSQL, and ODBC.

tcVISION on-premises acquires large amounts of data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transforms and delivers it to the AWS Cloud.

Users can easily model and map their data, and create links between data sources and target systems with tcVISION‘s GUI interface. tcVISION also has features to prevent looping and provide conflict detection for bidirectional data processing, allowing mainframe updates to be reflected on the AWS databases and changes on the AWS databases to be reflected back to the mainframe again.

About the AWS Marketplace…

_0_AWS_Marketplace_Logo01

The AWS Marketplace is an online software store that helps customers find, buy, and immediately start using the software and services that run on AWS.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Mainframe-to-Cloud and Open Systems Data Replication: tcVISION in 90 Seconds

Give us 90 seconds of your time, and we’ll quickly show you how tcVISION enables real-time and bi-directional Mainframe-to-Cloud and Open Systems data replication. Our video walk-through shows how tcVISION‘s GUI interface allows users to easily model and map data, and create links between data sources and target systems. tcVISION‘s intuitive interface supports simple to complex replication scenarios.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Cloud, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

tcVISION_Connection_Overview_Web01

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Amazon Aurora, PostgreSQL, Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, and IBM Informix.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

tcVISION and Amazon Web Services (AWS) – A Winning Combination

Why use tcVISION with AWS?

tcVISION_Mainframe_To_AWS_635

IBM mainframes are in use today at many of the world’s largest corporations.  They often contain key, mission critical data that is often difficult to exploit with the latest technologies, due to the proprietary nature and data formats of their databases.  tcVISION changes this and allows your corporation to exploit this data with the power of the world’s leading cloud platform – AWS.

tcVISION is a software tool that enables data replication between legacy mainframe, cloud, and open systems environments.  Its mainframe database support scope is broad and it supports the key legacy mainframe databases in use today. tcVISION has the intelligence to convert the proprietary data formats of Adabas, Datacom, IDMS, VSAM, IMS, and Db2 using data wizards specific to each database.  Its intelligence handles the required transformations, including EBCDIC to ASCII, endianness, copybooks to columns, embedded dependencies transforming these proprietary formats to databases such as Hadoop, Oracle, PostgreSQL, Aurora, and MySQL.

tcVISION_Connection_Overview_Web01

tcVISION has ETL bulk load functionality and uses native change data capture (CDC) mechanisms, keeping the data between mainframe databases and modern databases in constant real time synchronization.  Its support of bi-directional replication allows changes to be captured and updated in two different environments, including modern databases.  tcVISION is a powerful tool for data and application modernization, allowing legacy mainframe teams to continue their work while modern teams use leading edge technology on the same data.  It enables modern data warehouses and databases to have access to key corporate data residing on legacy systems.

While tcVISION can be utilized in your corporate datacenter, using tcVISION with AWS has several key advantages:

  • Rapid global deployments to 18 key geographic regions
  • Database as a Service– Relational Database Instances (for Oracle, PostgreSQL, MySQL, SQL Server, MariaDB, and Aurora) greatly simplifying administration
  • Powerful, low cost, highly available databases, such as Aurora
  • Extreme high availability using availability zones
  • Numerous infrastructure options to optimize the environment and enhance performance
  • Bursting capabilities to accommodate peak loads
  • Controlling costs through infrastructure optimization
  • Moving capital expense to operational expenses
  • Extreme security at every layer, maintained by world class security experts

“Security, scalability, resiliency, recoverability, and cost of applications in the cloud are better than what almost any private enterprise could achieve on its own”:

https://www.cloudcomputing-news.net/news/2017/may/17/cloud-computing-goes-beyond-tipping-point-financial-services-says-dtcc/

Rapid Global Connectivity

___Global_Connectivity

The AWS Cloud spans 55 Availability Zones within 18 key geographic Regions around the world.  When establishing an EC2 instance (virtual Windows or Linux instance) or a Relational Database instance, the geographic region and availability zone is a parameter.  In minutes, you can establish tcVISION instances with a fully functional database instance in key global business centers, including the Americas, Europe, and Asia.   Once the data is replicated, your global customers can benefit from the low latency of the data center proximity. Global data can be synchronized using the Amazon’s high speed global network using technologies such as Cloud Front and S3 Acceleration. Whatever your company size, leveraging these capabilities allows your customer to gain low latency global access to the key corporate data residing on your IBM mainframe.

Powerful and Cost Effective “Relational Database as a Service” Instances

Amazon Relational Database Service (Amazon RDS) makes it easy to set up, operate, and scale a relational database in the cloud.  Amazon RDS is available on several database instance types – optimized for memory, performance, or I/O – and provides you with six familiar database engines to choose from, including Amazon Aurora, Maria DB, PostgreSQL, Oracle, and Microsoft SQL Server.

tcVISION enables you to migrate or perform real-time replication between your on-premises mainframe and Amazon RDS databases.  Your business can rapidly deploy databases globally within minutes with minimal administration requirements.

There are several advantages of a Amazon RDS databases.  One especially powerful database is AWS Aurora which can be established as either mySQL or PostgreSQL API compatible.  Your business can leverage these key advantages:

  • Serverless Aurora: With serverless Aurora there is no database size pre-allocation required.  The database automatically scales up and down based on load and you pay only for the resources utilized.
  • High availability: HA is built into the architecture of Aurora, with six copies of data maintained across three Availability Zones. That is two copies in each Availability Zone (data center in a region), boosting availability with minor disruption even if an entire Availability Zone is down. In addition, the database is continuously backed up to Amazon S3, so you can take advantage of the high durability of S3 (99.999999999) for your backups.
  • Performance: Amazon Aurora provides 5X the throughput of MySQL and 3X the throughput of standard PostgreSQL running on the same hardware
  • Manageability: Amazon Aurora simplifies administration by handling routine database tasks such as provisioning, patching, backup, recovery, failure detection, and repair.
  • Storage Scaling: Aurora storage scales automatically up to 64TB per database instance, growing and rebalancing I/O across the fleet to provide consistent performance.

Scalable and high-performance

___Scalability

Using AWS tools, Auto Scaling, and Elastic Load Balancing, your application can scale up or down based on demand.

Unlike a traditional data center, with the AWS infrastructure environment, you pay only for what you use, and your company is charged only for resource utilization.  Users can bring up EC2 virtual Windows or Linux instances for any length of time. For example, if the mainframe database bulk loads only take an hour to complete, the tcVISION EC2 instances can be brought up for the two hours needed, and shut down when the processing is finished.  AWS EC2 instances can be dynamically stopped and restarted while retaining its data on AWS Elastic Block Storage. This way, your business pays only for what is needed and the instance sizes (CPU, memory, and I/O bandwidth) can be optimized for the required mainframe data migration. or data replication process.

AWS resources, such as EC2 instances, storage, and databases can be rapidly provisioned and de-provisioned.  The allows your corporation to approach the architecture with an agile mentality.  Once the initial design and architecture is complete, it can be rapidly tested and adjusted as required to optimize all aspects, including security, availability, performance, and operation readiness. The AWS environment allows users to experiment with various architectures and sizing.  For example, a customer can test various databases and EC2 instance sizes and types, and make changes based on test results to optimize sizing as required.

tcVISION CDC replication typically doesn’t need consistently high levels of CPU, but benefits significantly from having full access to very fast CPUs when needed. For example, there are significant spikes in mainframe database update activities, and tcVISION can benefit from very fast CPUs.  T2 instances are engineered specifically for these use cases.

Using AWS Auto Scaling, you maintain optimal application performance and scalability, even when workloads are periodic, unpredictable, or continuously changing.  AWS Auto Scaling continually monitors your applications to make sure they are operating at your desired performance levels.  When demand spikes, AWS Auto Scaling automatically increases the capacity of constrained resources, so you maintain a high quality of service.

Monitoring of environmental health

Support

In traditional data center environments, vendor supplied infrastructure monitoring tools are often deployed to monitor the infrastructure environmental health.  They measure and alert on environmental factors such as CPU utilization, memory usage, network bandwidth, and disk space.

In the AWS environment, the CloudWatch environment monitoring is integrated with AWS resources such as EC2 instances and Relational Database instances. You can use Amazon CloudWatch to collect and track metrics, collect and monitor log files, set alarms, and automatically react to changes in your AWS resources. This allows you to set alerts at user specified thresholds and take proactive measures such as notifying individuals and taking automated actions on the AWS environmental resources.  Given the dynamic nature of AWS resources, proactive measures including the dynamic re-sizing of infrastructure resources can be automatically initiated.

World Class Security

___Security_Eye

The AWS environment is enabled for security at every layer.

AWS resources are contained within your Virtual Private Cloud (VPC).  The Amazon VPC enables you to launch AWS resources into an isolated virtual private network that you define.  The Amazon VPC provides features that you can use to increase and monitor security, and security groups act as a firewall for associated EC2 instances, controlling the inbound and outbound traffic.  Network access control lists (ACLs) act as a firewall for associated subnets controlling inbound and outbound traffic at the subnet level.  Flow logs capture information about the IP traffic going to and from network interfaces in your VPC.

AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely.  You can create users in IAM, assign them individual security credentials (access, keys, passwords, and multi factor authentication devices).  You create roles in IAM and manage permissions to control which operations can be performed by the entity, or the AWS service that assumes the role.  If you already manage user identities outside of AWS, you can use an IAM identify provider instead of creating IAM users in your AWS account.  With an identity provider, you can manage your user identities outside of AWS and give these external user identities permission to use AWS resources in your account.

Data can be encrypted at rest or in transit.  Data encryption capabilities are available in AWS storage, database services such as EBS, S3 Glacier, Oracle RDS, SQL Server RDS, Redshift and many other services.  Flexible key management options, including AWS Key management Service, allows you to choose whether to have AWS manage the encryption keys or to enable you to keep complete control over the keys.


__TSI_LOGO

Contact Treehouse Software Today for an Online Demonstration of Mainframe-to-AWS Cloud Data Replication

Demo_Example

Request a tcVISION live demo, where we show tcVISION replicating data from a mainframe to AWS Aurora in the Cloud. Just fill out our demo request form, and a Treehouse representative will be in touch to schedule a convenient date and time.