tRelational / DPS Adabas-to-Oracle Success in South Africa

by Hans-Peter Will, Senior Technical Representative and Joseph Brady, Manager of Marketing and Technical Documentation at Treehouse Software, Inc.

Recently, Hans-Peter Will, Senior Technical Representative for Treehouse Software, traveled to South Africa to assist our partner Bateleur Software (pty) Ltd. with setting up a large public-sector customer’s data replication implementation using our Adabas-to-RDBMS tool set, tRelational / DPS (Data Propagation System).

Arriving in Johannesburg, Peter met with representatives from Bateleur and the IT Organization, where the key players discussed how tRelational / DPS was going to be used in the project. The customer initially wanted to populate sample data into Oracle, so Peter configured tRelational / DPS to process one of the smaller Adabas files to generate some data. He also recommended running an analysis with tRelational to determine whether the file contained a unique key. Peter took this opportunity to show the customer what other benefits they could realize out of the analysis information. Interestingly, they used a personnel file for analysis, and Peter was immediately able to show that 23 of the records had no gender entry and 180 of records had no surname. The customer was very pleased to see these revelations in the first analysis, and looked forward to identifying other data quality issues before commencing data replication.

Customer Replication Scenario with Treehouse Software Product Set…

_0_tReDPS_Replication_Scenario

The next step was to build the target structure in accordance with the Oracle DBA’s requirements. The DBA had specified that all columns were to be defined as VARCHAR2, except the date information. After the first model was completed, DDL and DPS parameters were generated and a quick materialization of data accomplished the desired result.

At the subsequent kickoff meeting, Peter provided a complete tRelational / DPS overview and discussed the target structure with the attendees and the Oracle DBA. The rest of the day was spent doing Adabas file implementations, analysis and modeling.

Setup was then completed for transferring extracted and transformed Adabas data into the customer’s Windows environment. Adabas Vista is used, so that one logical Adabas file was actually split into two files stored on different Adabas databases, and the customer wanted to combine them into the same target table in Oracle. While there was no unique descriptor, it was discovered that three fields in combination would make a unique key, enabling the model to be created to combine data from the separate physical files into a single Oracle table.

The team proceeded with file implementation, modeling, mapping, DPS executions, and resolving data issues. Various issues that were encountered, like invalid tab characters within the data, negative personnel numbers, duplicates in unique keys (maintained by the application) and the need to add an extra column to the output. These issues were resolved quickly by the customer’s staff.

Within a day, all the files were materialized and the PLOG copy process was modified so that from that point forward, every PLOG copy would automatically be processed through DPS Propagation to update the RDBMS on the target Windows machine.

The next day, Peter was asked by the customer, “How many of the files have been processed so far?”. Peter was pleased to report that every file was processed and was propagating successfully. The happy customer remarked that they never had a project that was completed this far ahead of the deadline.

Throughout the project, Peter never personally laid hands on a customer keyboard, but instead sat with staff, effectively training them and handing over comprehensive knowledge of tRelational / DPS. The customer was very excited to learn that their personnel can now easily use the product set to do any remaining work on their own.

A few days later, we received an e-mail from Bateleur:

“I had a very pleasant meeting with the customer today. They used tRelational to reject the non-unique keys, reran the Materialization, and reran DPS plus update into Oracle. The month-end update of Oracle that was taking nearly three days to complete, now takes five Minutes! Everyone delighted!”

Bjørn (Sam) Selmer-Olsen, Managing Director, Bateleur Software (pty) Ltd


About Treehouse Software’s tRelational / DPS Product Set

tReDPS_DIAGRAM

tRelational / DPS is a robust product set that provides modeling and data transfer of legacy Adabas data into modern RDBMS-based platforms for Internet/Intranet/Business Intelligence applications. Treehouse Software designed these products to meet the demands of large, complex environments requiring product maturity, productivity, feature-richness, efficiency and high performance.

The tRelational component provides complete analysis, modeling and mapping of Adabas files and data elements to the target RDBMS tables and columns. DPS (Data Propagation System) performs Extract, Transformation, and Load (ETL) functions for the initial bulk RDBMS load and incremental Change Data Capture (CDC) batch processing to synchronize Adabas updates with the target RDBMS.

Visit the Treehouse Software website for more information on tRelational / DPS, or contact us to discuss your needs.

TREETIP: tcVISION Supports Data Replication to MongoDB

tcVISONv6

The tcVISION cross-system integration platform is a robust, proven, and mature solution that is constantly under development to meet the requirements of new technologies, including support for MongoDB.

mongodb

MongoDB is among the leading NoSQL databases in the market and has been developed for the needs of today’s information technology. MongoDB supports a data model with dynamic schemata and is especially suitable to store large amounts of data using GridFS. It contains automatic failure protection using an integrated replication. MongoDB also offers native, idiomatic drivers for nearly all programming languages and frameworks.

Find out more about MongoDB here: https://www.mongodb.com

In addition to support for MongoDB, tcVISION features connectivity to other output targets, such as Hadoop (see previous blog about Hadoop support), Adabas LUW, DB2 BLU, and EXASOL. Additionally, new input sources include z/OS VSAM Logstream (CICS and Coupling Facility / Shared VSAM), z/OS VSAM Batch Extension, z/OS DBMS to Logstream, CA IDMS v17, CA Datacom CDC, IMS Active Log, and SMF data.

Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Treehouse Software Partners with Stone Bond Technologies to offer comprehensive enterprise Data Virtualization

Press Contacts: Joe Brady (+1.724.759.7070 x110; jbrady@treehouse.com); Irma Barrientos (+1 713.300.8882; ibarrientos@stonebond.com)

Pittsburgh, PA; Houston, TX, November 17, 2016. ­ Treehouse Software, Inc., of Sewickley, PA and Stone Bond Technologies, L.P. of Houston, TX today announced a partnership to bring agile integration and data virtualization capabilities to every enterprise. Now mainframe data sources can be combined into virtual schemata with relational databases, ERP data, Cloud sources and social media by combining Stone Bond’s Enterprise Enabler® with Treehouse’s tcACCESS.

Since the mid-1990s, Treehouse has been a global leader in mainframe data migration, replication and integration, offering robust and flexible solutions for ETL, CDC and real-time, multidirectional replication between databases on various platforms. With Enterprise Enabler, Treehouse customers benefit from a single platform for enterprise and Cloud integration to deliver integration projects with up to 90% time savings over other solutions.

Treehouse will provide worldwide product sales, marketing and support for Enterprise Enabler, bringing leading-edge integration capabilities to its hundreds of existing customers and to new customers pursuing initiatives such as Agile BI, Master Data Management, Logical Data Warehouses and Web Services.

One of the fastest-growing data virtualization platforms on the market, Enterprise Enabler has been leading the innovation of data management, data federation and data virtualization for nearly 15 years. To serve its expanding base of customers, Stone Bond has grown its set of AppComm™ connectors to accommodate the widest range of enterprise and line-of-business data sources. Treehouse’s tcACCESS effectively becomes the “Mainframe AppComm”, providing full read/write access to a wide range of mainframe data sources on both z/OS and z/VSE platforms.

Wayne Lashley, Treehouse Chief Business Development Officer, said, “Our reputation for capably solving mainframe data integration challenges now extends to other modern enterprise data sources including SAP, Salesforce.com and Microsoft SharePoint. With the push of a button, our customers can securely expose these disparate virtualized sources to popular BI tools like Tableau and to other applications in standard formats like REST and JSON. Current, past and new customers can look to Treehouse to address any imaginable integration scenario.”

Commented Tom Sieger, Stone Bond SVP, Corporate Strategy and Business Development, “Embracing mainframe data sources bolsters the “enterprise” in Enterprise Enabler. We are excited to work with Treehouse to grow Enterprise Enabler sales into new markets and geographies.”


__tsi_logo_400x200

About Treehouse Software, Inc.
Privately-held Treehouse Software was founded in 1982, and has hundreds of customers worldwide that benefit from the company’s industry-leading products and outstanding technical support. The traditional strengths of the company in performance management, security and software configuration management have been complemented by visionary leadership in mainframe data replication and integration and application modernization.

stonebond_logo

About Stone Bond Technologies, L.P.
Headquartered in the heart of Houston, Texas, Stone Bond Technologies was founded in 2002 with a commitment to innovation that continues today. More than just adding features to existing offerings, the company’s approach to information business systems is to make an enterprise’s data agile: to move data efficiently and easily, without barriers. Stone Bond’s client base continues to expand from its roots in the Energy industry to Financial Services, Life Sciences, Information Technology, Hospitality, Manufacturing and more.

TREETIP: tcVISION Allows for Surprisingly Innovative Uses

tcVISONv6

Treehouse Professional Services consultants help Adabas / Natural customers in a variety of ways, including DBA services, performance tuning, change management implementation and training, and data replication planning and training. Our experience and long history of service to the Adabas / Natural community helps us create innovative solutions for our customers’ challenges.

Recently Treehouse Senior Technical Representative Chris Rudolph assisted a customer with a tricky data replication problem. The customer uses tcVISION to perform bi-directional replication between Adabas and an RDBMS during the phase-in of a new application. Unfortunately, the new application incorrectly updated certain columns in the RDBMS, which were then replicated to Adabas. The customer attempted to address the issue by running a series of ADASEL reports against the Adabas PLOG and manually checking for “bad” transactions, which was a very time consuming process that pulled the Adabas DBA away from her normal duties.

Chris explained that tcVISION could expedite the process by replicating all transactions for the Adabas file to a journal table capturing the “before” and “after” values of the problematic columns. The developers working on the new application could then identify invalid values, correct the application, and patch the data themselves. This also allowed the Adabas DBA to return to their normal duties.

The journal table now includes columns to display the “before” and “after” values of the corrupted column, Adabas transaction time, end transaction time, operation and Adabas userid. The customer’s developers immediately recognized immense value from being able to query the journal table to find bad data, patch the data, prove that corruption is no longer taking place, and verify that all corrupted instances of the data have been patched. Journal tables have been added for all replicated Adabas files, and the developers now rely on the journal tables for all of their data patches.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

tcVISION acquires data in bulk or via change data capture methods, including in real time, from virtually any IBM mainframe data source (Software AG Adabas, IBM DB2, IBM VSAM, IBM IMS/DB, CA IDMS, CA Datacom, even sequential files), and transform and deliver to virtually any target. In addition, the same product can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM DB2 LUW and DB2 BLU, IBM Informix and PostgreSQL.


__TSI_LOGO

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Two Local Technology Companies Partner to Advance Cognitive Computing; Complementary Areas of Expertise Mean Better Data Integration and Individualization

Treehouse Software, Inc., of Sewickley, PA and Cognistx of Pittsburgh, PA announced a partnership to help customers with improved data integration and individualization to fully leverage the power of cognitive computing.

Technology industry leaders from Accenture to Gartner to McKinsey recognize the future of computing will be cognitive, calling it a disruptive force and estimating the industry to reach $200 billion by 2020. Cognitive computing is based on leading edge technology including artificial intelligence, natural language processing, Big Data, advanced analytics and machine learning algorithms.

cog_push_med2

The Treehouse – Cognistx partnership will allow customers to ingest massive amounts of data, whether that data is numbers, images, or audio files, and mine it to find insights that lead to action, and ultimately to increased revenue from improved customer engagement.

Since the mid-1990s, Treehouse Software has been a global leader in mainframe data migration, replication and integration, offering robust and flexible solutions for ETL, CDC and real-time, multidirectional replication between databases on various platforms.

Cognistx is an applied technology company harnessing state-of-the-art cognitive computing tools to help retailers reach individuals with intuitive, intelligent and individualized offers based on their past transactions, preferences, context and profile.

cog_mobile_tech_image

Cognistx complements Treehouse’s capability to deliver data with its machine learning algorithms that become more accurate with every transaction, delivering customized, personalized, prescriptive actions in the right context. Together, the two companies will co-market their capabilities, bringing new competitive advantages to customers who want to expand the use of their most valuable asset — data.

“We’re excited to partner with Cognistx to bring our world-class enterprise data acquisition capabilities to companies that recognize the massive opportunity cognitive computing represents,” said Wayne Lashley, Treehouse Chief Business Development Officer. “We provide the data foundation and Cognistx translates that data into insights, those insights into customer actions, and those actions into incremental revenue.”

“Few retailers do a good job of marrying technology with a customized customer experience that is tailored to their behaviors and timed according to how they might use a retailer’s offer,” said Sanjay Chopra, CEO of Cognistx. “With our proprietary algorithms and Treehouse’s enterprise data solutions, both our customers win. Only with large amounts of data can our system learn about the consumer and their preferences and how those change in order to deliver only the smartest, most individualized offers.”

About Treehouse Software, Inc.

Privately-held Treehouse Software was founded in 1982, and is a global leader in providing data migration, replication, and integration solutions for the most complex and demanding heterogeneous environments. Treehouse offers a comprehensive and flexible portfolio of software and tools for mainframe platforms, and also includes feature-rich, accelerated-ROI offerings for information delivery, and application modernization. http://www.treehouse.com

About Cognistx

Privately-held Cognistx was founded in 2015 and has a technology hub in Pittsburgh and operations offices in the Innovation Quarter in Winston-Salem, NC, and Raleigh. The company’s co-founders include Sanjay Chopra, a serial technology entrepreneur; Eric Nyberg, professor at Carnegie Mellon University’s School of Computer Science, who consulted with IBM on the Watson project and Jeffrey Battin, former owner of Communefx, a successful data analytics company. Other partners include Florian Metze, professor at Carnegie Mellon University’s School of Computer Science; Jill Zoria, SVP Enterprise Development; Pete Minnelli, SVP Creative; and Karen Barnes, SVP Operations. http://www.cognistx.com

PRODUCT SPOTLIGHT: tcVISION v6 Overview and Updates

tcVISONv6

Several exciting new features and updates are now in tcVISION v6, including new output targets Adabas LUW, DB2 BLU, EXASOL, Hadoop,and MongoDB. Additionally, new input sources include z/OS VSAM Logstream (CICS and Coupling Facility / Shared VSAM),z/OS VSAM Batch Extension, z/OS DBMS to Logstream, CA IDMS v17, CA Datacom CDC, IMS Active Log and SMF data.

Another feature recently announced is the tcVISION “Direct Loader” for BULK_LOAD processing. The function does not require output to a sequential file, and the loader utility for the target DBMS is called via API with data passed directly. Direct loader supports PostGreSQL, Microsoft SQL Server and DB2 LUW / DB2 BLU. The advantage of using the Direct Loader is the elimination of disk access in writing and reading the sequential loader data file. File output is still supported (e.g., where loader data is to be distributed to other machines).

Finally, as mentioned in a previous Treehouse Blog, with tcVISION v6 comes the newly enhanced web statistics functionality and web server. Any standard web browser can access this server (Internet Explorer, Firefox, Opera, Chrome, Safari, etc.)

This valuable feature enables users to view data from the tcVISION Manager Monitor, and statistical and operational information from the tcVISION Manager network.


Find out more about tcVISION — Enterprise ETL and Real-Time Data Replication Through Change Data Capture

tcVISION provides easy and fast data migration for mainframe application modernization projects and enables bi-directional data replication between mainframe, Linux, Unix and Windows platforms.

_0_tcVISION_Simple_Diagram

Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.

Mainframe CDC from Treehouse Software

The globalization of markets, increase of data volumes, and high demand for up-to-date information require new data transfer and exchange solutions for heterogeneous IT architectures, and as many customers have discovered, Treehouse Software has the right product (or combination of products) to meet any conceivable mainframe data migration, replication, or integration requirement. To meet many of these needs, Treehouse Software’s proven and mature tcVISION product moves data – as little as possible – as much as necessary. tcVISION is an innovative software solution that processes changed data in real time, in intervals, or event based.

ChangeDataCapture

The tcVISION solution focuses on changed data capture (CDC) when transferring information between mainframe data sources and LUW databases and applications. Changes occurring in any mainframe application data are tracked and captured, and then published to a variety of RDBMS and other targets.

_0_tcVISION_Simple_Diagram

tcVISION enables bidirectional replication for DB2, Oracle, and SQL Server running on Linux/Unix/Windows, and synchronizes each data source, first by doing a bulk load from source(s) to target and then by replicating only changes— only committed changes—from source(s) to target. So there can never be ambiguity as to whether a query against the target database involves uncommitted data.

Read some tcVISON customer success stories here.


Visit the Treehouse Software website for more information on tcVISION, or contact us to discuss your needs.