by Joseph Brady, Director of Business Development / Cloud Alliance Leader at Treehouse Software, Inc.
Many Treehouse Software customers have discovered that they can save weeks, or months in their mainframe modernization initiatives by doing a tcVISION Proof of Concept (POC) for Mainframe-to-Cloud data replication. Depending on the complexity of the customer’s project, a tcVISION POC generally lasts as little as 10 business days after the product is installed and all connectivity is set up between the mainframe and Cloud environments. Treehouse Software provides documentation beforehand that outlines all of the requirements and agenda for the POC, and Treehouse technicians assist in downloading and installing tcVISION.
The customer provides a representative subset of z/OS or z/VSE mainframe data (e.g., Db2, Adabas, VSAM, IMS/DB, CA IDMS, CA DATACOM, etc.), use case, and goals for the POC, and the Treehouse team mentors the customer’s technical team via remote screen sharing sessions. The application is executed on customer facilities, in a non-production environment, and a limited-scope implementation of a tcVISION application is conducted to prove that the product meets the customer’s desired use case.
By the end of the POC, customers will have replicated mainframe data on their Cloud target, tested out product capabilities, and demonstrated a successful, repeatable data replication process, with documented results. After the tcVISION POC, the customer has all the connectivity and processes in place to begin setting up the production phase of their mainframe data modernization project. The minimal cost, in terms of human resources and time makes a tcVISION POC a valuable ROI in the customer’s mainframe modernization journey.
A key advantage for customers is once tcVISION is up and running, their legacy mainframe environment can continue as long as needed, while they replicate data – in real time and bi-directionally – on the new Cloud platform. Now the enterprise can quickly take advantage of the latest Cloud services, such as analytics, machine learning and artificial intelligence (AI), etc., as well as move data to a variety of highly available and secure databases and data stores.
About tcVISION…
Many Cloud and Systems Integration partners are recommending tcVISION from Treehouse Software for Mainframe-to-Cloud modernization projects. tcVISION focuses on changed data capture (CDC) when transferring information between mainframe data sources and Cloud targets. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to a variety of RDBMS and other targets.
Additionally, tcVISION utilizes a Windows-based GUI Control Board, which is ideal for non-mainframe programmers. While mainframe experts are required in the design/architecture phase during the POC and occasionally during implementation, the requirement for their involvement is limited. The tcVISION Control Board acts as a single point of administration, data modeling and mapping, script generation, and monitoring. Comprehensive monitoring and logging of all data movements ensure transparency across all data exchange processes.
Further reading…
Treehouse Software is an AWS Technology Partner and tcVISION is a Validated AWS Qualified Software. The AWS Partner Network published a blog about tcVISION, which describes how tcVISION allows legacy mainframe environments to continue, while replicating data on highly available and secure AWS targets.
Contact Treehouse Software for a tcVISION Demo Today…
Simply fill out our tcVISION Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.
by Joseph Brady, Director of Business Development / Cloud Alliance Leader at Treehouse Software, Inc.
Treehouse Software, Inc. is pleased to announce that we were chosen by US Foods for their mainframe data modernization initiatives using the tcVISION Mainframe-to-Cloud and Open Systems data replication product.
Treehouse Software is the worldwide distributor of tcVISION, the leading tool for using change data capture (CDC) for synchronizing mainframe data with real-time and bi-directional data replication. tcVISION’s intuitive data modeling and mapping, and ease of migrating data, made it the ideal choice for helping to modernize the large mainframe environment at US Foods.
“The entire Treehouse Software team is excited about working with US Foods to make their modernization initiatives a success!” – George Szakach, CEO and President at Treehouse Software
About US Foods
With a promise to help its customers Make It, US Foods is one of America’s great food companies and a leading foodservice distributor, partnering with approximately 250,000 restaurants and foodservice operators to help their businesses succeed. With 70 broadline locations and more than 80 cash and carry stores, US Foods and its 28,000 associates provides its customers with a broad and innovative food offering and a comprehensive suite of e-commerce, technology and business solutions. US Foods is headquartered in Rosemont, Ill. Visit https://www.usfoods.com/ to learn more.
Interested in seeing a live, online demo of tcVISION?
Simply fill out our tcVISION Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
Introduction
Many readers know that Treehouse Software has been around since 1983, serving enterprises worldwide with industry-leading software products and outstanding technical support. This blog series has discussed Treehouse Software’s origins and the growth of the software company from the early 1980s up to the present.
Change is in the Air, and in the Clouds…
Parts 1 and 2 of this series illustrated the solid beginnings of Treehouse Software in the 80’s and 90’s. Several products were developed and introduced. Marketing representatives were acquired in several countries around the globe. Also, other companies with valuable products sought out Treehouse Software to sell and support their offerings.
In the late 90’s and the early 2000s, we began to experience certain customers’ needs to have Adabas data moved (migrated/copied/converted/distributed) to other database systems. We developed the tRelational/DPS product set to analyze their Adabas data and structure, and move this data to other relational database systems (RDBMS) such as Oracle, Db2, etc. This complicated product was our foray into the mainstream of our customers’ mainframe IT processing. Millions of records of initial data needed to be materialized (ETL) efficiently, and a Change Data Capture (CDC) capability was imperative. Mission accomplished.
In the early 2000s, some customers required real-time CDC. This led to the development of DPSync, so named because it kept Adabas data in Sync with the target RDBMS; but this was limited to uni-directional replication.
Requests began coming in for bi-directional (e.g., moving data back to Adabas from Oracle). Then it was Oracle-to-Db2. And vice versa – and more variations. We investigated companies purporting to do data migration from/to various database systems. In 2007, we found a company with a product already developed and proven, that could in fact move data from/to practically all known database systems at that time. We partnered with B.O.S. of Germany for Treehouse to do worldwide sales, marketing, support, demos, POCs, training, etc., for that impressive, growing product set, tcVISION.
tcVISION caught on quickly with some of our existing customers of Adabas, but the significant interest commenced when the Cloud took hold. The Cloud was not just a remote data center, or a place to archive large amounts of data, but has capabilities and features that would attract our types of customers with mainframes and terabytes of data. Enterprise cutomers needed tools that allowed them the connectivity to take advantage of Cloud-based technologies, such as highly available and scalable databases; advanced analytics and security; machine learning and artificalial intelligence; data warehouses and stores; and the list goes on. This shaped the future direction of Treehouse Software. More on this in the next blog.
About Treehouse Software
Since 1983, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
Introduction
Many readers know that Treehouse Software has been around since 1983, serving enterprises worldwide with industry-leading software products and outstanding technical support. This blog series will dig a little deeper into Treehouse Software’s origins and explore how founder and president, George Szakach blazed a trail from being a programmer and manager in the 1960s and 1970s, to creating and growing his own software company from the early 1980s up to the present.
Treehouse First Generations… 1980s – 90s
As mentioned in Part 1 of this series, George’s foundational mainframe experience from 1960 to 1975, combined with skills honed from 1975 to 1982 at Software AG, planted the seeds for the next step, building Treehouse Software. George explains:
“Treehouse Software was started during Christmas of 1982, with a consulting assignment I decided to take in the Kingdom of Saudi Arabia doing ADABAS performance analysis and tuning. In addition, I was asked to teach the customer how to use ADAMINT (which I get credit for developing from July 1975 through May 1979 – or blame, take your pick). I thoroughly enjoyed my 10 weeks in the Kingdom, where I became a big fan of pistachio nuts.
Treehouse Software was incorporated in mid-1984 in Sewickley, PA. The plan then was to provide consulting and educational services to SAG sites, while developing software products related to the SAG enterprise. The services might sell the products, and vice versa. It worked and still works. I started hiring help almost immediately as the demand for products and services was growing.
By the way, Sewickley is a small town near Pittsburgh where the oil, steel, coal barons of the 1800s and early 1900s had their summer homes up the hill from the river. The 100-year-old horse-and-wagon trails and stone walls around the estates can still be seen.”
The Sewickley Bridge spans the Ohio River between Sewickley and Moon Township, PA.
Some interesting facts about Treehouse Software, Inc.
By the end of the 20th century, Treehouse designed, developed, and released ten products, most notably:
Treehouse also marketed several additional products from companies around the world. Most were related to ADABAS.
Other miscellaneous Treehouse happenings last century:
Taught classes for over 4000 students, ADABAS and Natural related, and related to Treehouse products.
Partnered with 20 affiliates or marketing representatives in various countries, topping out at 14 around 1990.
Produced 55 issues of the popular Treetips newsletter, with a hard-copy circulation of 13,000 per issue.
Attended 11 Software AG conferences.
Presented at 90 regional and local Software AG meetings in various countries.
Visited affiliates, customers, and partners in at least 24 countries on all continents except Antarctica.
Invited to Oracle Conference in 1995, in Philadelphia, where Emilie Szakach entertained the crowd with an on-stage dance with Chubby Checker.
Bought four IBM mainframes for the company.
Hired many top notch programmers and analysts, tech writers, support personnel.
Along with most of the world, we awaited and prepared for the feared Y2K debacle, which didn’t happen. However, as the year 2000 approached, one of Treehouse’s most popular and most requested give-aways ever, was our Y2K desk clock…
Cha, Cha, Cha, Changes…
Along with many new products, innovations, and memorable, contributing staff moving through the doors of the Treehouse, we’ve also had several iterations of the company logo and colors, as seen in the following graphic. Do you have a favorite?
Coming soon… Part 3: Change is in the Air, and in the Clouds…
About Treehouse Software
Since 1983, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.
by Joseph Brady, Director of Business Development / Cloud Alliance Leader at Treehouse Software, Inc.
Everyone has seen the recent headlines about how aging and outdated technology nearly crippled the airline industry. As a result, modernizing and securing information systems has taken center stage and top priority again. Even before the airline IT disaster, the COVID-19 crisis was a critical event that forced modernization to become a strategic imperative for government, supply chain, healthcare, utilities/energy, financial, and defense industries. All of these sectors have critical data residing on a variety of long-standing mainframe databases (often still updated by COBOL applications), including Db2, VSAM, IMS/DB, Adabas, IDMS, Datacom, and sequential files. Unlocking the value of this important data can be difficult, because the data is often utilized by numerous interlinked and dependent programs that have been in place for decades.
“The Federal Aviation Administration’s 30-year-old hazard-notification system recently had its first crash ever to cause a nationwide grounding of flights. The incident is focusing a bright light on the outdated federal computer systems that, IT experts say, are increasingly vulnerable to failure and cyberattack.” – Source: Christian Science Monitor Daily
As a result of this renewed push to modernize IT systems, Treehouse Software has been seeing a significant increase in requests from Cloud platform partners, government agencies, and other infrastructure customers to evaluate modernization solutions that replicate data, in real time, on highly available and secure Cloud-based databases, data warehouses, etc.. Fortunately, Treehouse has the deep mainframe expertise and software tools to help.
Since 1983, Treehouse Software has been working with many of these enterprises with mainframes in the areas of data migration, security, control, auditing, performance enhancement, etc.. Treehouse has also expanded its capabilities to focus on new requirements for modernizing legacy mainframe databases on various Cloud and open systems platforms with the tcVISION mainframe data replication product. tcVISION is the primary tool in Treehouse Software’s “data-first” approach, whereby immediate data replication to the Cloud helps customers get on the fast track to meeting spikes in demand for vital information, especially in times of crisis.
Replicating mainframe data on the Cloud can happen within days during a tcVISION Proof of Concept (POC)…
After setup and installation, a tcVISION POC is approximately 10 business days, with the customer providing a small subset of data and use case for the POC. A Treehouse Software technician will assist in downloading and installing tcVISION and conducting a limited-scope implementation of a tcVISION application. This application uses a small subset of customer data and executes on customer facilities, usually in a non-production environment. A document is provided beforehand for the customer to fill out their requirements, use cases, and agenda for the POC.
By the end of the 10-day POC, customers can replicate and test mainframe data on their Cloud target database. It can happen that fast!
Further Reading…
Treehouse Software and AWS published a blog about tcVISION’s Mainframe-to-AWS data replication capabilities:
Contact Treehouse Software for a tcVISION Demo Today…
Simply fill out our tcVISION Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
Introduction
Many readers know that Treehouse Software has been around since 1983, serving enterprises worldwide with industry-leading software products and outstanding technical support. However, this blog series will dig a little deeper into Treehouse Software’s origins and explore how founder and president, George Szakach blazed a trail from being a systems programmer in the early 60s, to creating and growing his own software company from the early 80s up to the present.
The beginnings… 1960’s. Moon Landing, Flower Power, the Righteous Brothers, and Punched Cards
George is a Vietnam-era veteran and started working with mainframes in 1960 while in the Army.
After programming school in Fort Monmouth, NJ, George was assigned to Fort Huachuca, Arizona where he wrote army related applications on the IBM 709.
Before leaving the army in 1963, George had many job offers. Three years of programming experience was unheard-of back then, so his skillset was very valuable. He was even offered a job by the president of Informatics, working in Houston at the NASA Johnson Space Center to “put a man on the moon.” He declined.
Throughout the rest of the 60s, George worked at Burroughs, Univac, and Leasco. During the 70s through 1982, George worked for Ocean Data Systems, Data General, Optical Recognition Systems, Software AG (his longest stint – 7 years), and Superior Oil.
Punched card…
IBM 709 Computer System…
George’s archeological finds from his time at Univac…
With all of this foundational mainframe experience combined with his skills honed at Software AG, the seeds were planted for George’s future: take those roots, move to the trees, and build a house…
Coming soon… Part 2: Treehouse Software’s first generations.
About Treehouse Software
Since 1983, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
With the recent passing of Pittsburgh Steelers great running back and Pro Football Hall of Famer, Franco Harris, we would like to revisit April of 1993, when Treehouse Software held an international consultant’s symposium. The symposium brought together attendees and speakers from many consulting and technology companies, and schools from around the world. Since Treehouse Software is located in the greater Pittsburgh area, company president George Szakach was acquainted with Franco and invited him to deliver a fascinating and entertaining address, where he spoke about his career, several business ventures he was pursuing, as well as his budding interest in computer technology.
Franco Harris at Treehouse Software’s Consultant’s Symposium (April 1993)
A few years ago, George reminded Franco about his visit to Treehouse back in 1993. He remembered and they shared some laughs and memories.
We would also like to mention Franco’s well-known sense of community and accessibility in Pittsburgh. Many staff members have met Franco over the years and have fond memories of his friendliness and willingness to spend time engaging in conversations. Those who come in to the Pittsburgh International airpot can see a sculpture depicting Franco’s famous “Immaculate Reception” from 1972. Thousands of people, especially recently, have selfies taken with the sculpture. Franco will be missed by his many friends and the community.
Franco Harris sculpture at Pittsburgh International Airport.
About Treehouse Software
Since 1982, Treehouse Software has been serving enterprises worldwide with industry-leading mainframe software products and outstanding technical support. Today, Treehouse Software is a global leader in providing data replication, and integration solutions for the most complex and demanding heterogeneous environments, as well as feature-rich, accelerated-ROI offerings for information delivery, and application modernization.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
Since the dinosaur analogy has been used so much to describe mainframe computer systems in recent years, I would like to use this blog to take a look at the parallels of dinosaurs and mainframes as it relates to the current buzz about modernization on the Cloud.
Of course, dinosaurs and mainframes have been around for a long time and are extremely resilient and successful. I especially say “are” in relation to dinosaurs, because many are not extinct at all, and the fossil record shows that several types have adapted to the changing world by evolving into birds. Additionally, during the age of dinosaurs, they branched off into countless varieties during a span of about 165 million years – hardly a failed species. Also, like the dinosaurs, the mainframe has thrived and survived for over six decades and is continuing to adapt – albeit not nearly as long as the reign of the dinosaurs, but an impressive run, nonetheless.
And the mainframe isn’t finished yet! Mainframe systems are still very much in use, running major banking processes, healthcare systems, government IT services, and critical business operations of many Global 2000 companies. As a matter of fact, IBM has been reporting growth year after year, as the IBM Z platform continues to see important innovations, such as with Cloud-native development capabilities, as well as impressive improvements in processing power.
Looking up and moving forward…
As with the dinosaurs who did not fear looking to the clouds and taking wing to ensure survival, the new breed of mainframers envision bold and exciting possibilities in Cloud computing. Many see remarkable opportunities for business advantage by modernizing their mainframe environments. This modernization includes replicating mainframe data on Cloud platforms in order to quickly capitalize on the latest Cloud services, such as analytics, auto scaling, machine learning and artificial intelligence (AI), high availability, advanced security, etc., or to move data to a variety of newer Cloud databases, streaming services, container services, and much more. With the proper data replication technology and planning, all of this modernization can occur while keeping the legacy mainframe environment active as long as it is needed!
The IBM Z mainframe isn’t going anywhere, and with visionary and daring leadership, it can continue to evolve and adapt to whatever develops in the Cloud… and beyond.
Ready to move forward, adapt, and evolve? Treehouse Software is here to help!
Treehouse Software is your partner on your journey into future mainframe modernization plans. With our “data first” approach, we can help accelerate digital transformation and successfully leverage Cloud and Hybrid Cloud initiatives on the IBM Z platform, storing sensitive data on a private Cloud or local data center, and simultaneously leveraging leading technologies on a managed public Cloud.
Through an innovative changed data capture (CDC) technology, our tcVISION product tracks and captures changes occurring in any mainframe application data, and then publishes them to a variety of Cloud targets. The customer moves only the right data to the right place at the right time – as much, or as little as they want.
The tcVISION data replication solution has a modular design, which enables it to support mass data load from one source to one or more targets, as well as continuous data exchange processes in real-time via CDC. This modular architecture and the provided APIs gives customers unlimited future potential for continued evolution, and use of new and emerging technologies.
Want to see tcVISION in action?
You can schedule a live, online demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database. Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION Mainframe-to-Cloud data replication demonstration.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.
Many of Treehouse Software’s enterprise customers are not close to considering the retirement of their mainframe systems, but instead have long-term data replication projects, or want to indefinitely have their legacy systems co-exist with a new Cloud platform. These organizations are looking for solutions that allow their legacy mainframe environment to continue while replicating data – in real time and bi-directionally – to take advantage of the latest Cloud services, such as analytics, auto scaling, machine learning and artificial intelligence (AI), high availability, advanced security, etc., or move data to a variety of newer Cloud databases, streaming services, container services, and more.
The Transition Doesn’t Have to be a Sudden Big Bang
Much of an enterprise’s mission critical mainframe data is stored in legacy mainframe databases, and the cost to maintain these databases is high. An added complication is that the data is utilized by many interlinked and dependent programs that have been in place for many years, and sometimes decades. Unlocking the value of this legacy data is also difficult due to many very different types of mainframe databases (e.g., Db2, Adabas, CA Datacom, CA IDMS, etc.).
Immediate data replication on the Cloud is enabling government, healthcare, supply chain, financial, and a variety of public service organizations to meet spikes in demand for vital information, especially in times of crisis. The globalization of markets, increase of data volumes, 24×7 operations, changing business conditions, and high demand for up-to-date information also requires new data transfer and exchange solutions for heterogeneous IT architectures.
The Data-First Solution
Treehouse Software is here to help enterprise mainframe customers accelerate digital transformation and successfully leverage Hybrid Cloud initiatives on the IBM Z platform, storing sensitive data on a private Cloud or local data center and simultaneously leveraging leading technologies on a managed public Cloud. Our tcVISION replication solution focuses on changed data capture (CDC) when transferring information between mainframe data sources and modern databases and applications. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to a variety of targets. The customer moves only the right data to the right place at the right time – as much, or as little as they want.
The tcVISION replication solution has a modular design, which enables it to support mass data load from one source to one or more targets, as well as continuous data exchange processes in realtime via CDC. This modular architecture and the provided APIs gives customers unlimited potential for growth and use of new technologies.
tcVISION allows bi-directional, real-time data synchronization of changes on either platform to be reflected on the other platform (e.g., a change to a PostgreSQL table is reflected back on mainframe). The customer can then modernize their application on the cloud, open systems, etc. without disrupting the existing critical work on the legacy system.
In the following example high level architecture diagram, bi-directional data replication between Db2 z/OS and AWS using tcVISION is shown:
tcVISION utilizes a Windows-based GUI Control Board, which is ideal for non-mainframe programmers. While mainframe experts are required in the design/architecture phase and occasionally during implementation, the requirement for their involvement is limited. The tcVISION Control Board acts as a single point of administration, data modeling and mapping, script generation, and monitoring. Comprehensive monitoring and logging of all data movements ensure transparency across all data exchange processes. In the following example, the mainframe can be seen communicating to an Amazon EC2-based tcVISION replication manager. The tcVISION Control Board shows the user a graphical representation of this replication:
Additionally, tcVISION supports complex data replication scenarios between multiple data sources and targets, as seen here:
With tcVISION, data replication projects can be implemented within a few of months, depending on the complexity of the project. This includes the proof of concept and design/architecture stages. After these stages are complete, the customer can start the first production implementation sprint, immediately providing business value. We suggest successive agile sprints to allow for incremental deployment of additional file replication, sprint by sprint.
Supported Sources and Targets
tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data replication for Mainframe-to-Cloud and Open Systems application modernization projects.
Contact Treehouse Software for a tcVISION Demo Today…
Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your tcVISION demonstration. This will be a live, on-line demonstration that shows tcVISION replicating data from the mainframe to a Cloud target database.
by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software
Careful planning must occur for a Mainframe-to-Cloud data modernization project, including how a customer’s desired Cloud environment will look. This blog serves as a general guide for organizations planning to replicate their mainframe data on Cloud platforms using Treehouse Software‘s tcVISION.
A successful move to the Cloud requires a number of post-migration considerations and solutions in order to modernize an application on the Cloud. Some examples of these considerations and solutions include:
Personnel Resource Considerations
Staffing for Mainframe-to-Cloud data replication projects depends on the scale and requirements of your replication project (e.g., bi-directional data replication projects will require more staffing).
Most customers deploy a data replication product with Windows and Linux knowledgeable staff at varying levels of seniority. For the architecture and setup tasks, we recommend senior technical staff to deal with complex requirements around the mainframe, Cloud architecture, networking, security, complex data requirements, and high availability. Less senior staff are effective for the more repeatable deployment tasks such as mapping new database/file deployments. Business staff and system staff are rarely required but can be necessary for more complex deployment tasks. For example, bi-directional replication requires matching keys on both platforms and their input might be required. Other activities would be PII consideration, specifics of data transformation and data verification requirements.
An example of staffing for a very large deployment might be one very part-time project manager, a part-time mainframe DBA/systems programmer, 1-2 staff to setup and deployment the environment and an additional 1-2 staff to manage the existing replication processes.
Environment Considerations
As part of the architecture planning, your team needs to decide how many tiers of deployment are needed for your replication project. Much like with applications, you may want a Dev, QA, and Prod tier. For each of these tiers, you will need to decide the level of separation. For example, you might combine Dev and QA, but not Prod. Many customers will keep production as a distinct environment. Each environment will have its own set of resources, including mainframe managers (possibly on separate LPARs), Could VMs (e.g., EC2) for replication processing, and for managed Cloud RDBMSs (such as AWS RDS).
After the required QA testing, changes are deployed to the production environment. Object promotion test procedures should be detailed and documented, allowing for less experience personnel to work in some testing tasks. Adherence to details, processes, and extended testing is most import when deploying bi-directional replication, due to the high impact of errors and difficult remediation.
Rollout Planning
A data replication product is typically deployed using Agile methods with sprints. This allows for incrementally realized business value. The first phase is typically a planning/architecture phase during which the technical architecture and deployment process are defined. Files for replication are deployed in groups during sprint planning. Initial sprint deployments might be low value file replications to shield the business from any interruptions due to process issues. Once the team is satisfied that the process is effective, replication is working correctly, and data is verified on the source and targets, wide scale deployments can start. The number of files to deploy in a sprint will depend on the customer’s requirements. An example would be to deploy 20 mainframe files per 2–3-week sprint. Technical personnel and business users need to work together to determine which files and deployment order will have the greatest business benefit.
Security
For security, both on-premises and to the major Cloud environments, there are several considerations:
Data will be replicated between a source and target. The data security for PII data must be considered. In addition, rules such as HIPPA, FIPS, etc. will govern specific security requirements.
The path of the data must be considered, whether it is a private path, or if the data transverses the internet. For example, when going from on-premises to the Cloud the major Cloud providers have a VPN option which encrypts data going over the internet. More secure options are also available, such as AWS Direct Connect and Azure ExpressRoute. With these options, the on-premises network is connected directly to the Cloud provider edge location via a telecom provider, and the data goes over a private route rather than the internet.
Additionally, Cloud services such as S3, Azure Blob Storage, and GCP buckets default to route service connections over the internet. Creating a private end point (e.g., AWS PrivateLink) allows for a private network connection within the Cloud provider’s network. Private connections that do not traverse the Internet provide better security and privacy.
Protecting data at rest is important for both the source and target environments. The modern Z/OS mainframe has advanced pervasive and encryption capabilities: https://www.redbooks.ibm.com/redbooks/pdfs/sg248410.pdf. The major Cloud providers all provide extensive at-rest encryption capabilities. Turning on encryption for Cloud Storage and databases is often just a parameter setting and the Cloud provider takes care of the encryption, keys, and certificates automatically.
Protecting data in transit is equally important. There are often multiple transit points to encrypt and protect. First, is the transit from the mainframe to on-premises to the Cloud VM instance. A mainframe data replication product should provide protection employing TLS 1.2 to utilize keys and certificates on both the mainframe and Cloud. Second is from the Cloud VM to the Cloud target database or service. Encryption may be less important since often these services are in a private environment. However, encryption can be achieved as required.
High Availability
During CDC processing, high availability must be maintained in the Cloud environment. The data replication product should keep track of processing position. The first can be a Restart file, which keeps track of mainframe log position, target processing position, and uncommitted transactions. The second can be a container stored on Linux or Windows to store committed unprocessed transactions. Both need to be on highly available storage with a preference for storage across Availability Zones (AZs), such as Elastic File System (Amazon EFS) or Windows File Server (FSx).
The Amazon EC2 instance (or other Cloud instance) can be part of an Auto Scaling Group spread across AZs with minimum and maximum of one Amazon EC2 instance.
Upon failure, the replacement Amazon EC2 instance of the replication product’s administrator function is launched and communicates its IP address to the product’s mainframe administrator function. The mainframe then starts communication with the replacement Amazon EC2 instance.
Once the Amazon EC2 instance is restarted, it continues processing at the next logical restart point, using a combination of the LUW and Restart files.
For production workloads, Treehouse Software recommends turning on Multi-AZ target and metadata databases.
Scalable Storage
With scalable storage provided on most Cloud platforms, the customer pays only for what is used. The data replication product should require file-based storage for its files that can grow in size if target processing stops for an unexpected reason. For example, Amazon EFS, and Amazon FSx provide a serverless elastic file system that lets the customer share file data without provisioning or managing storage.
Analytics
All top Cloud platform providers give customers the broadest and deepest portfolio of purpose-built analytics services optimized for all unique analytics use cases. Cloud analytics services allow customers to analyze data on demand, and helps streamline the business intelligence process of gathering, integrating, analyzing, and presenting insights to enhance business decision making.
A data replication product should replicate data to several data sources that can easily be captured by various Cloud based analytics services. For example, mainframe database data can be replicated to the various Cloud ‘buckets’ in JSON, CSV, or AVRO format, which allows for consumption by the various Cloud analytic services. Bucket types include AWS S3, Azure BLOB Data, Azure Data Lake Storage, and GCP Cloud storage. Several other Cloud analytics type services also support targets including Kafka, Elasticsearch, HADOOP, and AWS Kinesis.
Kafka has become a common target and can serve as a central data repository. Most customers target Kafka using JSON formatted replicated mainframe data. Kafka can be installed on-premises, or using a managed Kafka service, such as the Confluent Cloud, AWS Managed Kafka, or the Azure Event Hub.
Monitoring
Monitoring is a critical part of any data replication process. There are several levels of monitoring at various points in a data replication project. For example, each node of the replication including the mainframe, network communication, Cloud VM instances (such as EC2) and the target Cloud database service all can require a level of monitoring. The monitoring process will also be different in development or QA vs. a full production deployment.
A data replication product should also have its own monitoring features. One important area to measure is performance and it is important to determine where any performance bottleneck is located. Sometimes it could be the mainframe process, the network, the transformation computation process, or the target database. A performance monitor helps to detect where the bottleneck is occurring and then the customer can drill down into specifics. For example, if the bottleneck is the input data, areas to examine are the mainframe replication product component performance, or the network connection. The next step is to monitor the area where the bottleneck is occurring using the data replication product’s statistics, mainframe monitoring tools, or Cloud monitoring such as AWS CloudWatch.
A data replication product should also allow the customer to monitor processing functions during the replication process. The data replication product should also have extensive logs and traces that allow for detailed monitoring of the data replication process and produce detailed replication statistics that include a numeric breakdown of processing statistics by table, type of operation (insert, update delete), and where these operations occurred (mainframe, or target database).
CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, providing customers with a unified view of AWS resources, applications, and services that run on AWS, and on-premises servers. You can use CloudWatch to set high resolution alarms, visualize logs and metrics side by side, take automated actions, troubleshoot issues, discover insights to optimize your applications, and ensure they are running smoothly.
Some customers are satisfied with a basic monitoring that polls every five minutes, while others need more detailed monitoring and can choose polls that occur every minute.
CloudWatch allows customers to record metrics for EC2 and other Amazon Cloud Services and display them in a graph on a monitoring dashboard. This provides visual notifications of what is going on, such as CPU per server, query time, number of transactions, and network usage.
Given the dynamic nature of AWS resources, proactive measures including the dynamic re-sizing of infrastructure resources can be automatically initiated. Amazon CloudWatch alarms can be sent to the customer, such as a warning that CPU usage is too high, and as a result, an auto scale trigger can be set up to launch another EC2 instance to address the load. Additionally, customers can set alarms to recover, reboot, or shut down EC2 instances if something out of the ordinary happens.
Disaster Recovery
IT disasters such as data center failures, or cyber attacks can not only disrupt business, but also cause data loss, and impact revenue. Most Cloud platforms offer disaster recovery solutions that minimize downtime and data loss by providing extremely fast recovery of physical, virtual, and Cloud-based servers.
A disaster recovery solution must continuously replicate machines (including operating system, system state configuration, databases, applications, and files) into a low-cost staging area in a target Cloud account and preferred region.
Unlike snapshot-based solutions that update target locations at distinct, infrequent intervals, a Cloud based disaster recovery solution should provide continuous and asynchronous replication.
Consult with your Cloud platform provider to make sure you are adhering to their respective best practices.
Many organizations lack the internal resources to support AI and machine learning initiatives, but fortunately the leading Cloud platforms offer broad sets of machine learning services that put machine learning in the hands of every developer and data scientist. For example, AWS offers SageMaker, GCP has AI Platform, and Microsoft Azure provides Azure AI.
Applications that are good candidates for AI or ML are those that need to determine and assign meaning to patterns (e.g., systems used in factories that govern product quality using image recognition and automation, or fraud detection programs in financial organizations that examine transaction data and patterns).
The list goes on…
Treehouse Software and our Cloud platform and migration partners can advise and assist customers in designing their roadmaps into the future, taking advantage of the most advanced technologies in the world.
Successful customer goals are top priority for all of us, and we can continue to work with our customers on a consulting basis even after they are in production.
Of course, each project will have unique environments, goals, and desired use cases. It is important that specific use cases are determined and documented prior to the start of a project and a tcVISION POC. This planning will allow the Treehouse Software team and the customer develop a more accurate project timeline, have the required resources available, and realize a successful project.
Your Mainframe-to-Cloud Data Migration Partner…
Treehouse Software is a global technology company and Technology Partner with AWS, Google Cloud, and Microsoft. The company assists organizations with migrating critical workloads of mainframe data to the Cloud.
Further reading on tcVISION from AWS, Google Cloud, and Confluent:
tcVISION supports a vast array of integration scenarios throughout the enterprise, providing easy and fast data migration for mainframe application modernization projects. This innovative technology offers comprehensive abilities to identify and capture changes occurring in mainframe and relational databases, then publish the required information to an impressive variety of targets, both Cloud and on-premises.
tcVISION acquires data in bulk or via CDC methods from virtually any IBM mainframe data source (Software AG Adabas, IBM Db2, IBM VSAM, CA IDMS, CA Datacom, and sequential files), and transform and deliver to a wide array of Cloud and Open Systems targets, including AWS, Google Cloud, Microsoft Azure, Confluent, Kafka, PostgreSQL, MongoDB, etc. In addition, tcVISION can extract and replicate data from a variety of non-mainframe sources, including Adabas LUW, Oracle Database, Microsoft SQL Server, IBM Db2 LUW and Db2 BLU, IBM Informix, and PostgreSQL.
Contact Treehouse Software for a tcVISION Demo Today…
Simply fill out our tcVISION Demonstration Request Form and a Treehouse representative will be contacting you to set up a time for your requested demonstration.
Treehouse Software's Blog 