Treehouse Software TREETIP: Data Replication for a Mainframe Database that has no Primary Key, using tcVISION

by Joseph Brady, Director of Business Development / AWS and Cloud Alliance Lead at Treehouse Software, Inc. and Chris Rudolph, Senior Technical Representative at Treehouse Software, Inc.

This blog takes a look at tcVISION’s support for a mainframe database that has no primary key at the source (a mainframe primary key is a column, or set of columns that uniquely identifies one row of a table). 

In situations where the source database does not contain any unique values, Treehouse technicians discuss with the customer how the application currently works and their expectations for how the data should be treated when planning replication/migration to a new target environment. Depending on the application, Journal Replication or Data Warehouse Replication may be a better fit than a “normal” RDBMS table definition.

tcVISION‘s Key ID management can be used to create a column to use as the key on the target table. Another option would be to use a SQL Lookup to query another table. Either way, target column(s) that contain unique values will need to be identified. Once these columns are identified, they can be marked with the key identifier, using the tcVISION repository editor, or Key ID Management can be used to create a new unique column, then those columns are used within the wizard.

For example, this source table does not contain a primary key:

___tcV_NoKey01

Analysis of the target data shows the columns FIRST_NAME, LAST_NAME and MIDDLE_NAME together will provide a unique value. These fields can be marked in the tcVISION repository as being members of a key:

___tcV_NoKey02

As mentioned earlier, the tcVISION Key ID Management wizard can also be used to create a new unique target column:

___tcV_NoKey03

Specify the key name:

___tcV_NoKey04

Specify the key value creation:

___tcV_NoKey05

Various options are available: 

___tcV_NoKey06


__tsi_logo_400x200

Contact Treehouse Software for a Demo Today…

tcVISION_Overall_Diagram

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.


Further reading: tcVISION Mainframe data replication is featured on the AWS Partner Network Blog…

___tcVISION_AWS_HA_Architecture

AWS recently published a blog about tcVISION’s Mainframe data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe Db2 z/OS data to Amazon Aurora. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.

Treehouse Software Customer Success: Data Replication from Mainframe Adabas to PostgreSQL using tcVISION

BMF_Building

The Bundesministerium der Finanzen (BMF) is Germany’s Ministry of Finance and establishes sustainable fiscal policy that ensures financial empowerment of the federal budget. From tax policy via development of federal budget, to regulation of national and international financial markets – for these and other fiscal and economic questions of principle, the BMF creates strategies and concepts, and implements them. The Federal Tax Administration is part of BMF, and controls not only the cross-border goods traffic, but acts against illegal employment and other crimes. The tax administration also imposes consumer taxes (e.g., energy and tobacco tax, car tax, etc.). Financial relations between federation, countries, and communities are also coordinated by BMF.

Department II (federal budget) is part of the German government in charge of establishing the budget and financial planning of the federation. Throughout the year, it monitors execution of the budget for eventual intervention (e.g., with a budget freeze, or supplementary budget). After closing the fiscal year, the budget and balance sheet will be presented. The budget is a supplement of the budget act, legally binding.

The central service organization of BMF is the Informationstechnikzentrum Bund – ITZBund (Information technic center).

BUSINESS BACKGROUND

Drawing up the budget is a yearly, highly time consuming, and formalized business process. All departments are involved in nearly every sub-process, and budgeting and financial planning is supported by the application, “Haushaltsaufstellung / Budgetgeneration”. Using the generated reports, various addressees/receivers are supported (e.g., German Federal Government, German Federal Parliament, Federal Council of Germany, finance department in BMF, the employees in the departments, and the public).

Technically, the budget plan of the federation is based on technologies, including the IBM Mainframe with z/OS running Adabas and Natural.

The challenge was to provide an environment for employees in all departments that enables them to do their work quickly, easily, and efficiently. In the BMF, users must have an editorless, end-user driven, and real-time creation of ready-to-print products.
An informative description of the workflow is shown on the website of the BMF.

The federal budget is available as download, or one can directly navigate through the data using the online application.

BUSINESS ISSUE

Some time ago, BMF decided to re-engineer the application for budget planning and port it to Open Source. To guarantee a seamless transition, the first step is propagation of data out of Adabas on z/OS to PostgreSQL, concluding with permanent synchronization.
The difficulties of this task are the complexities of setting up data definitions for the data structures in Natural and the propagation of data from Adabas on z/OS to PostgreSQL.

TECHNOLOGY SOLUTION: tcVISION

tcVISION_Overall_Diagram

After an analysis of the project, Treehouse Software proposed creating an extension to tcVISION’s change data capture (CDC) functionality for integration, so that tcVISION could enable BMF to continue using the implemented data definitions in a format suitable for the RDBMS.

The extension was developed within a few days, and a two-day on premise test demonstrated the solution fit the requirements of BMF.

BMF can now provide its data definitions from Natural LDA to the extension of tcVISION, and after the transformation, onto the PostgreSQL load process for processing. Another advantage of the tcVISION solution is that when needed, other targets can be integrated for propagation of data from the mainframe (e.g., Kafka, which BMF indicated is a future target environment).

Additionally, bi-directional propagation can be added in budget planning when BMF is ready.

Data structures are held in LDA, because this provides the advantages of higher flexibility in development and the adaption of new requirements to the data definitions. If definitions would have to be ported manually, in part, to PostGreSQL, it would have been a much bigger and error-prone effort.

Subsequent changes to Adabas structures can now use tcVISION’s newly developed extension to easily regenerate and load the correct definitions to the RDBMS, and tcVISION completely covers the customer’s requirements for special usage of *PEs and *MUs.

After thorough preparation and extensive testing, the solution was released to selected users first, then made available to all users.

* PEs and MUs are special Adabas formats for definition of tables. PE = Periodic Group, MU = Multiple Value Field.


__tsi_logo_400x200

Contact Treehouse Software for a Demo Today…

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.


Further reading: tcVISION Mainframe data replication is featured on the AWS Partner Network Blog…

___tcVISION_AWS_HA_Architecture

AWS recently published a blog about tcVISION’s Mainframe data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe Db2 z/OS data to Amazon Aurora. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.

Treehouse Software Helps Mainframe Customers Easily Access the Most Advanced AWS Machine Learning and AI Tools

by Joseph Brady, Director of Business Development / AWS and Cloud Alliance Lead at Treehouse Software, Inc.

___AI

Wow! Machine Learning and Artificial Intelligence (AI) are not only being “disruptive” to the computing and business worlds, but the value proposition for Treehouse Software’s customer base (those with mainframe systems) who are moving large amounts of data to the Cloud can be profound. The dominant Cloud platform today is AWS, and they are making incredible leaps in making the most sophisticated AI tools available, literally at one’s fingertips. Never before has such powerful and useful technology been so easily available to so many.

We have lots of mission-critical data on our mainframe system, and we want take advantage of the latest in Machine Learning and AI too!

___Mainframe_To_AWS

Treehouse Software is an AWS Technology Partner, and has been in the mainframe market space since 1982. We are here to help enterprise mainframe customers successfully move their data to the Cloud in order to utilize all of the most advanced Cloud services and tools available. Our tcVISION product can help customers replicate their mainframe data, in real-time and bi-directionally, between a vast array of source databases and many AWS Cloud technologies for analytics, global database deployment, forecasting, machine learning, security, storage, etc.

___tcVISION_AWS_Overall_Architecture

Take a look at some of the world-changing AWS Machine Learning and AI technologies that are available once your enterprise data is in the Cloud… 

AWS is focused on solving some of the toughest challenges that hold back machine learning from being in the hands of every developer. Amazon SageMaker provides every developer and data scientist with the ability to build, train, and deploy machine learning models quickly. Amazon SageMaker is a fully-managed service that covers the entire machine learning workflow to label and prepare your data, choose an algorithm, train the model, tune and optimize it for deployment, make predictions, and take action. Your models get to production faster with much less effort and lower cost.

Use your company’s data for time-series forecasting

Based on the same technology used at Amazon.com, Amazon Forecast uses machine learning to combine time series data with additional variables to build forecasts. Amazon Forecast requires no machine learning experience to get started. You only need to provide historical data, plus any additional data that you believe may impact your forecasts.

With Amazon Forecast, you can achieve forecasting accuracy levels that used to take months of engineering in as little as a few hours. You can import time series data and associated data into Amazon Forecast from your Amazon S3 database. From there, Amazon Forecast automatically loads your data, inspects it, and identifies the key attributes needed for forecasting.


Further reading: tcVISION Mainframe-to-AWS data replication is featured on the AWS Partner Network Blog…

___tcVISION_AWS_HA_Architecture

AWS recently published a blog about tcVISION’s Mainframe-to-AWS data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe Db2 z/OS data to Amazon Aurora. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.


__tsi_logo_400x200

Contact Treehouse Software for a Demo Today…

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

tcVISION_Overall_Diagram

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.

From Driving on the Test Track, to Driving Data to The Cloud – The Automotive Industry is Going Through Dramatic Changes

by Joseph Brady, Director of Business Development / AWS and Cloud Alliance Lead at Treehouse Software, Inc.

_0_AWS_Auto

As more automotive companies are also beginning to function as complex technology companies, many are coming to Treehouse Software to explore our comprehensive Mainframe-to-AWS data replication capabilities. A substantial number of these companies have large quantities of valuable data stored on-premises in their mainframe systems, as well as in other data sources (LUW, Open Systems, etc.). The goal is for a company’s various data types to be moved to – and integrated on – the AWS Cloud in order to take advantage of the myriad of advanced analytics, artificial intelligence (AI), and machine learning technologies. With this integration, it is now possible to build innovative connected products and services for every touchpoint of the customer journey. Providing a personalized, consistent, and engaging automotive experience across various channels has become key to building and maintaining brand loyalty.

A primary advantage of utilizing the AWS infrastructure, is that it can grow and shrink to meet a customer’s needs, and AWS only charges for the resources used. Additionally, AWS DevOps tools help manage complex environments at scale, and can support high velocity demanded by marketing teams. This allows a company to focus resources on building differentiated automotive brand experiences for customers, rather than managing a resource demanding IT infrastructure.

This video takes a quick look at AWS for the Automotive Industry…

Example: BMW Group’s Unified Configurator Platform on AWS

BMW Group turned to AWS to build its next-generation Unified Configurator Platform, moving from from an on-premises, monolithic application to a microservices-based platform in the AWS cloud:

Meanwhile, back on the mainframe…

Treehouse Software has been helping mainframe customers since 1982, and our extensive experience, deep knowledge, and wide-ranging capabilities in mainframe technologies makes us a valued partner and a trusted advisor to customers.

Treehouse Software is an AWS Technology Partner, and our focus today is software and services that enable mainframe customers to leverage their investment in legacy systems with data integration and replication, and take advantage of cutting edge Cloud services.

Our tcVISION product provides the means for many mainframe customers to easily replicate relevant data to the Cloud. The Automotive industry is quickly becoming aware of the need to tie together many desperate data sources within their IT environment, including their long-standing mainframe systems. Treehouse Software stands ready to show companies how they can tap into decades worth of valuable, mission critical mainframe data by using our mature and automated technology.

Further reading: tcVISION Mainframe-to-AWS data replication is featured on the AWS Partner Network Blog…

___tcVISION_AWS_HA_Architecture

AWS recently published a blog about tcVISION’s Mainframe-to-AWS data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe Db2 z/OS data to Amazon Aurora. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.


__tsi_logo_400x200

Contact Treehouse Software for a Demo Today…

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

tcVISION_Overall_Diagram

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.