Treehouse Proof of Concept: Bi-directional Replication Between Adabas and SQL Server

Chris Rudolph and Kevin Heimbaugh, Senior Technical Representatives for Treehouse Software, visited a customer site (a large retail and distribution company) to perform a five-day proof of concept (POC) of tcVISION with bi-directional replication between Software AG’s Adabas and Microsoft SQL Server.

Chris and Kevin initially met with the customer team, consisting of the DBA, Applications Manager, and a technical applications person. The agenda for the week was set to:

• Import metadata from several Adabas files
• Bulk load the Adabas data into SQL Server
• Set up replication from Adabas to SQL Server
• Add the bi-directional replication back to Adabas

Additionally, there were a few other items the customer wanted the Treehouse team to address, including support for date formats; timestamps for bi-directional replication to avoid update conflicts; using Predict views to define multiple SQL Server tables; and support for MUs and PEs. Chris noted that everything on the customer’s list is easily supported, and there are several options for the update scenarios that can be used.

After the tcVISION components were installed, the POC began by using tcVISION’s Control Board to define a metadata repository database in SQL Server. Once that was set, the teams moved on to import the first Adabas file’s metadata using tcVISON’s Metadata Import Wizard. As part of this process tcVISION generated Adabas to SQL Server schemas and field-to-column links as well as created target tables in SQL Server. Bulk Transfer scripts were created using a wizard to read the Adabas file on the mainframe, and load the data into SQL Server using the SQL Server bulk loader. Chris created a control script to show how tcVISION can concurrently bulk transfer multiple Adabas files into SQL Server This required increasing the tcVISION Manager’s VSE partition size to successfully test multiple load scripts executing in parallel.

The teams moved on to define the real-time change data capture (CDC) scripts necessary to process the Adabas PLOG. The tcVISION scripts use a two-phase approach to queue captured Adabas transaction on the open platform, then transform and apply the transactions to SQL Server. The scripts were set up to automatically generate detailed logs to track the PLOG transactions captured, SQL statements successfully applied to SQL Server, failed SQL statements, and informational items such as auto-corrected data and transactions rejected due to processing rules.

Now that several tables were defined and loaded, the bi-directional process was set up. SQL Server CDC was enabled for each table to be replicated. The team made a change within SQL Server and verified that the change show up in the SQL Server CDC tables. The SQL Server-to-Adabas mappings were defined in the tcVISION metadata repository, including the “back update check” to ensure only non-tcVISION transactions are captured, and the scripts on both Windows and mainframe were defined to create the LUWs from the SQL Server CDC and apply the changes to Adabas.

CDC from SQL Server to Adabas was successfully tested. Chris then showed the ability to create Journal replication where each change can be captured by replication type. The team spent time creating a few more mappings so multiple file / table updates could be tested, in addition to doing updates while the scripts were stopped to simulate a lost connection. This included setting up a new script to process copied PLOG datasets created by the ADARES utility.

The team defined the remainder of their Adabas files to the metadata repository. Some were set them up for bi-directional replication, and others were setup for unidirectional replication and Journal replication. Everything work as expected at the wrap-up meeting where the team provided a live demonstration to management of tcVISION and the items accomplished. The final tcVISION presentation and demo went very well, and everyone was pleased with the progress made during the week.

---

Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.

---

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

TREETIP: Integrate Mainframe Data Sources In Your Big Data Initiatives

tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects and enabling bi-directional data replication between mainframe, Linux, Unix and Windows platforms. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both on-premise and Cloud-based.

Analysts have observed that perhaps 80 percent of the world’s corporate data still resides on mainframes. So it’s no surprise that Bloor Research (http://www.bloorresearch.com/research/spotlight/big-data-and-the-mainframe/), notes that “it is necessary today to place the mainframe as a ‘first-class player’ in any enterprise Big Data strategy.”

In February 2017 we highlighted tcVISION’s support for replication to the leading NoSQL database MongoDB. MongoDB continues to increase in popularity as a back end for operational applications with real-time requirements.

tcVISION also supports analytics and “mainframe offload” Big Data use cases that generally leverage Hadoop HDFS and/or streaming data transport. With tcVISION, data from a wide variety of IBM mainframe data source can be quickly and easily replicated to Big Data targets, requiring minimal mainframe know-how and having minimal impact on the mainframe.

Boost the return on investment for your Big Data initiatives using tcVISION!

---

Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix, and PostgreSQL.

---

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

TREETIP: tcVISION Allows for Surprisingly Innovative Uses

Treehouse Professional Services consultants help Adabas / Natural customers in a variety of ways, including DBA services, performance tuning, change management implementation and training, and data replication planning and training. Our experience and long history of service to the Adabas / Natural community helps us create innovative solutions for our customers’ challenges.

Recently Treehouse Senior Technical Representative Chris Rudolph assisted a customer with a tricky data replication problem. The customer uses tcVISION to perform bi-directional replication between Adabas and an RDBMS during the phase-in of a new application. Unfortunately, the new application incorrectly updated certain columns in the RDBMS, which were then replicated to Adabas. The customer attempted to address the issue by running a series of ADASEL reports against the Adabas PLOG and manually checking for “bad” transactions, which was a very time consuming process that pulled the Adabas DBA away from her normal duties.

Chris explained that tcVISION could expedite the process by replicating all transactions for the Adabas file to a journal table capturing the “before” and “after” values of the problematic columns. The developers working on the new application could then identify invalid values, correct the application, and patch the data themselves. This also allowed the Adabas DBA to return to their normal duties.

The journal table now includes columns to display the “before” and “after” values of the corrupted column, Adabas transaction time, end transaction time, operation and Adabas userid. The customer’s developers immediately recognized immense value from being able to query the journal table to find bad data, patch the data, prove that corruption is no longer taking place, and verify that all corrupted instances of the data have been patched. Journal tables have been added for all replicated Adabas files, and the developers now rely on the journal tables for all of their data patches.

---

Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.

---

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

PRODUCT SPOTLIGHT: tcVISION v6 Overview and Updates

Several exciting new features and updates are now in tcVISION v6, including new output targets Adabas LUW, DB2 BLU, EXASOL, Hadoop,and MongoDB. Additionally, new input sources include z/OS VSAM Logstream (CICS and Coupling Facility / Shared VSAM),z/OS VSAM Batch Extension, z/OS DBMS to Logstream, CA IDMS v17, CA Datacom CDC, IMS Active Log and SMF data.

Another feature recently announced is the tcVISION “Direct Loader” for BULK_LOAD processing. The function does not require output to a sequential file, and the loader utility for the target DBMS is called via API with data passed directly. Direct loader supports PostGreSQL, Microsoft SQL Server and DB2 LUW / DB2 BLU. The advantage of using the Direct Loader is the elimination of disk access in writing and reading the sequential loader data file. File output is still supported (e.g., where loader data is to be distributed to other machines).

Finally, as mentioned in a previous Treehouse Blog, with tcVISION v6 comes the newly enhanced web statistics functionality and web server. Any standard web browser can access this server (Internet Explorer, Firefox, Opera, Chrome, Safari, etc.)

This valuable feature enables users to view data from the tcVISION Manager Monitor, and statistical and operational information from the tcVISION Manager network.

---

Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Join us at the Software AG TechEd User Conference in Austin, Texas

The third annual Software AG TechEd User Conference is planned for May 3-4 in Austin, Texas this year. In prior years, this event has developed a reputation for having the best source of technical information in our niche field, as we can learn from our peers and Software AG product experts about best practices, new features and use cases that you can apply on the job, and this year’s conference will be no exception. Of course, there is also the added benefit of being able to network with the largest collection of Adabas/Natural peers in North America.

Register here: http://www.softwareag.com/us/community/usergroup/events/default.asp

In addition to the presentations planned, there will be an Adabas Internals Class offered after the TechEd conference if there is enough interest. This class will be taught by Gene Miklovich with upgraded content for v8. Many people who attended last year’s TechEd conference expressed an interest in this topic, so if you were one of these people and/or are interested in this opportunity, please go to the Software AG Education portal to register.

---

 

 

Register Today for December’s Software AG TechEd 2015 User Conference!

By Wayne Lashley, Chief Business Development Officer for Treehouse Software

This year, TechEd returns to Philadelphia, PA on Monday, December 7, 2015. I, and others from Treehouse Software will be attending, and I will be presenting “Essential Capabilities for Adabas-to-Adabas Data Migration”.

Share ideas with, and gain insights from, your contemporaries. Hear how Software AG can help you rise to the challenge of driving your business forward on the shoulders of Technology and Innovation.

I strongly encourage all Adabas/Natural customers to sign up for the *FREE* TechEd 2015 HERE.

See you in Philadelphia!

 

TREETIP: TRIM’s Real-Time Monitor Provides a Window into Your Adabas/Natural World

by Joseph Brady, Manager of Marketing and Technical Documentation at Treehouse Software, Inc.

Adabas/Natural customers need a reliable, effective, and efficient way to analyze and optimize performance. TRIM is a robust performance monitor for Adabas and Natural. The TRIM Real-Time Monitor provides a wealth of key performance statistics, including:

• Adabas activity in various categories across time intervals
• Session statistics
• Adabas queues and tables
• And much more.

TRIM screen shot examples…

TRIM Real-time Monitor Commands Per Minute Screen:

TRIM Real-time Monitor Commands Within File Screen:

This information can help the site quickly find and eliminate performance problems, and online help, PF-Keys, Direct Commands, and other features simplify TRIM usage.

Reminder of the Special Offer For CA APAS/Insight and CA PLEU Customers…

If your company uses CA APAS/Insight and/or CA PLEU, you have been notified that support for the products is longer be provided by CA. As a result, we are pleased to extend a special offer to CA APAS/Insight and CA PLEU customers to adopt comparable Treehouse Software solutions, and enjoy the assurance of our commitment to supporting and updating TRIM, AUDITRE, NatCDCSP, NatQuery, and our other solutions that complement ADABAS and NATURAL.

Treehouse Software representatives are ready to help you implement our solutions to address your key requirements — all at specially-discounted pricing.

Visit the Treehouse Software website, or contact us to discuss your needs.

Register Now for Upcoming Software AG Eastern Region User Group Meeting in Reston, VA

You are invited to the upcoming Software AG Mid-Atlantic Region User Group Meeting. Share ideas with and gain insights from your colleagues in the Metro area. Hear how Software AG can help you rise to the challenge of driving your business forward or better reaching your agency’s mission on the shoulders of Technology and Innovation.

Wayne Lashley, Chief Business Development Officer, will present “Revitalize and Maximize Adabas/Natural” on behalf of Treehouse Software.

September 24, 2015 | 8:30 AM – 4:30 PM EST

AAFMAA Partner Location – MetroStar Systems, Reston, VA 20190

Click Here to Register>>

---

Register Now for Upcoming Software AG Webinar with ConnX and Treehouse Software

Wayne Lashley, Chief Business Development Officer, will present on behalf of Treehouse Software for this webinar. Wayne will show how our real-world customers are implementing strategies to integrate, modernize and extend mission-critical Adabas/Natural applications by leveraging Treehouse products and services.

Thursday, June 25, 2015 | 11:30 AM, EST
Click here to register for this webinar>>

Need Some Help? Look to Treehouse Professional Services for Adabas / Natural

Since 1982, Treehouse Software has been a leader in helping Adabas/Natural customers with our outstanding software and services, for both mainframe and LUW (Linux/Unix/Windows) platforms.

With Treehouse Professional Services, you can benefit from our decades of experience in Adabas/Natural optimization, systems security, control, auditing, performance enhancement and application development and support.

Give us a call today, and tell us about your current Adabas/Natural situation, and we can discuss how Treehouse Professional Services can ease your company’s financial undertow.

You can’t afford to have your mission-critical Adabas/Natural systems compromised, even if resources have been limited by budgetary constraints or expertise shortages.

Our expert consultants can provide Adabas/Natural performance analysis, environment tuning and optimization; development, training and implementation services involving any Adabas/Natural-related technology; product installation, upgrades and training; data replication and integration; application rehosting and modernization; and DBA services.

Instantly benefit from Treehouse Software’s 30+ years Software AG experience with our On-Site and Remote DBA Services

The Treehouse On-site and Remote DBA Services offer a proven, cost-effective alternative to full-time in-house systems programmers, system administrators and DBAs. Using our team of experts and monitoring techniques, Treehouse offers world-class, on-site and remote system support and DBA service at affordable prices. Treehouse Remote DBA Services offer various levels of system monitoring and administration. Our team will monitor critical factors within your Software AG environment, apply preventive maintenance for your systems and databases and keep your software up-to-date. We even provide z/OS systems programming services! We are strongly committed to our customers’ success and we deliver the highest quality DBA and support solutions.

Boost your Adabas/Natural team’s abilities today!

Visit the Treehouse Software website for more information on Treehouse Professional Services, or contact us today to discuss your needs.