Treehouse Software is Setting Sights on Many New Data Replication Projects

by Chris Rudolph, Senior Technical Representative for Treehouse Software and Joseph Brady, Marketing and Documentation Manager for Treehouse Software

Setting up a New Customer with tcVISION…

Treehouse Software representatives are busy these days, visiting customers and getting them started with our tcVISION product for complex data replication between mainframe, Linux, Unix, and Windows platforms.

As an example of a typical customer visit to implement data replication, we recently were on-site at a state government agency to configure tcVISION and set up bulk transfer and change data capture as well as train the State employees on using and managing tcVISION. Prior to our arrival, the customer had installed tcVISION and verified that its components were working correctly. After our three days on-site, the users were able to import their own ADABAS files, run scripts to perform bulk loads, and replicate transactions on the PLOG from ADABAS to SQL Server.

On the first day, we held a project kickoff meeting to discuss tcVISION’s architecture, the State’s ADABAS/NATURAL environment, and identified which ADABAS files should be replicated in the initial phase. We also presented a walkthrough of the product that covered metadata import, bulk transfer, and batch and real-time replication.

The group agreed that, for the purposes of the project at hand, real-time replication was not necessary immediately. It is possible that real-time replication and bi-directional replication may be needed in the future. Currently, monthly NATURAL extracts are used to transfer ADABAS data to SQL Server. The data is loaded into VARCHAR columns in the RDBMS, and then the data is copied to another set of tables. This process includes formatting dates, times, and numbers, but mostly ignores MUs and PEs (repeating fields in ADABAS). We showed how tcVISION can bulk transfer and replicate to a brand-new RDBMS schema created by tcVISION, as well as to an existing schema.

The setup process continued on the second day, and included creating two SQL Server databases for tcVISION, defining the connection to SQL Server, creating the tcVISION repository tables in one of the SQL Server databases, importing metadata from an ADABAS file into tcVISION, and creating the tables in SQL Server corresponding to that ADABAS file.

Next, we focused on populating these first tables. To do this, we ran the tcVISION bulk transfer script and then a control script to automatically execute the SQL Server bcp utility once the tcVISION Bulk Transfer completed.

We then imported another ADABAS file’s metadata into tcVISION. This second file contained several MUs, some of which we normalized, and others of which we denormalized on the parent table. After discussing the target structure, a bulk transfer on the ADABAS file was run. The ADABAS file contained over 100 fields and some 800,000 records. The bulk transfer ran in about 22 minutes, and the bcp into SQL Server ran for approximately two minutes. The customer’s existing NATURAL extract process takes a little under an hour for the same file, so tcVISION was able to perform the bulk transfer in about a quarter of the time.

As a last activity for the day, we demonstrated to the customer how to import an existing SQL Server table into tcVISION, and then map the table to an ADABAS file.

The tcVISION “Control Board” is the user’s MS Windows GUI Interface, providing a single and central point of administration, featuring the ability to view and manage transaction logs from other systems; built-in Wizards for mapping and modeling metadata; script generation for bulk transfer and data replication; and monitoring of replication status, script output, and DBMS extensions.

For our final activities on the third day, we ran a few more bulk transfers, and the users focused on importing ADABAS metadata and creating tables in SQL Server. During this time, we set up a new control script that allowed bulk transfer of multiple ADABAS files, instead of having to process each ADABAS file with a different script. The control script reads a list of ADABAS files, and then runs a bulk transfer script for each ADABAS file. The tcVISION manager uses a class setting to limit the number of bulk transfers running in parallel.

We then verified that the control script had submitted the processing script for each ADABAS file, and we demonstrated how to use tcVISION’s scheduler to automatically submit the control script. Once scheduling was working, we then turned our attention to processing the PLOG for change data capture.

A processing script was set up to read the PLOG on the mainframe, and then update the newly-populated SQL Server tables. We explained how some JCL could be added to the to the PLOG archive copy job to automatically process the PLOG whenever it flips.

The users were very pleased with the progress made during the three-day visit, and are well on their way to having tcVISION automatically handle and simplify their ADABAS replication efforts.

To learn more, or to request a tcVISION demonstration, contact Treehouse Software today.