Co-authored by OZSOFT and IT-Conductor Team, first published on https://www.itconductor.com/blog/automating-your-database-migration-strategy-to-the-cloud

The number of organizations embarking on their digital transformation journey has continued to increase. Part of the efforts being carried out includes migrating applications, databases, and other digital assets to the cloud.

While cloud migration is no longer a new topic among organizations, migrating databases to the cloud is still seen as a challenge especially for large enterprises housing several databases on-premises.

Why Migrate On-Prem Database to the Cloud?

Moving on-prem databases to the cloud allows an organization to leverage consumption-based pricing, less infrastructure complexity, and access to a broad selection of database management systems. Should an organization decide to switch to a different environment, it would be easier to move between cloud providers through cloud-to-cloud migration.

Another obvious reason that pushes organizations to migrate databases to the cloud is the fact that legacy systems will soon come to an end as we face the era of cloud computing.

Overcoming the Challenges in Database Migration to the Cloud

Database migration is a critical operation that requires technical expertise to implement. In this regard, there is an imposing challenge in having technical experts perform the migration or training people to do the job. This, in turn, accumulates additional operational costs to the business.

While investing in people is essential to the success of any organization, investing in technology and tools is just as important. The rise of automation is making human labor more valuable by eliminating the repetitive manual jobs that are being performed in the workplace. Automating database migration is not an exception especially for organizations with several databases on-premises.

OZSOFT for Database Migration to the Cloud

To help solve this hurdle for organizations, we started developing process definitions that can be used as standard automation workflows for migrating databases to the cloud. With our predefined workflows, the need for technical expertise or training people to manually perform the migration activities step-by-step will be eliminated. Our technical experts have already designed and developed workflows that can be reused repeatedly. In partnership with our sister company IT-Conductor, we enable standard approaches to cloud migration:

1) Discovery of source systems,

2) Distilling the source environment to baseline what exists on-prem

3) Designing the target environment and migration approach

4) Developing the customization of best-practices automation templates for provisioning

5) Deploying the automation to create target systems, and migrating data from on-prem to cloud.

Figure 1: SAP to Cloud Migration Methodology

In the following use case scenario, let’s dive into how we transpired this automated approach to migrate the Oracle database to the Azure cloud using the IT-Conductor platform.

Preparing the Infrastructure for the Automated Migration

The initial setup entails the provisioning of Azure VM along with the disk attachment and OS preparations. This is just a sample of what you would do manually if you are not leveraging automation. In our case, the provisioning of the virtual machine in Azure is to be executed using Terraform.

Figure 2: Sample Azure VM Creation Wizard

As we have discussed in this article, we have incorporated Terraform with the IT-Conductor platform to easily provision infrastructure. Setting up a virtual machine, pulling scripts from repositories, and executing terraform jobs are all possible in just one execution of the ITC activity we have programmed in this particular automation feature for Oracle DB migration to Azure.

Once all the required resources for the VM, including the disk and the OS image, have been installed, the execution of ITC activities related to the actual database migration will immediately run upon the trigger. Most of these ITC activities are programmed using Ansible scripts, woven together through the ITC concept called process definition, making it easier to automate complex processes.

Defining the Process Definition in the IT-Conductor Platform

Process Definition is a template for processes that can be scheduled to run at the desired time. It is developed for the purpose of eliminating the complexities in task executions when performing different activities in SAP systems.

The integration of Terraform and Ansible with the IT-Conductor platform makes up most of the process definition developed for the Oracle database migration to Azure cloud. But ITC activities, in general, can be anything like SQL and shell script as well.

Figure 3: Process Definition for Oracle DB Migration to Azure Cloud

We have identified around 18 steps (excluding the Start/End) for this particular activity. As you can see, the flow of the activities is very straightforward. The “Primary Server Setup” activity in the above diagram further contains several on-prem activities as shown below.

Figure 4: Process Definition for the Primary Server Setup (On-Prem)

From a high-level perspective, here are the significant steps to be executed in the process definition developed for Oracle DB migration to Azure:

1. Provisioning of Azure VM

2. Installation of the required software to Azure VM and mounting of the storage account

3. OS Configuration changes

4. Installation of Oracle

5. Patching of Oracle

6. Primary Server Setup (On-Prem)

7. Standby Server Setup (Azure VM) and issue duplicate command

8. Enabling brokers

9. Installation of APP Server on Azure VM

The types of ITC activities used in the process definition defined above are:

1. Terraform Activity – used for executing Terraform scripts

2. Ansible Activity – used for executing Ansible scripts

3. SQLProcedure Activity – used for executing PL/SQL scripts

4. SQLScript Activity – used for executing SQL queries

5. SQLGetter Activity – used for capturing the output of SQL queries

Automated Oracle DB Migration to Azure Cloud Implementation

Execution of ITC activities can be triggered automatically by scheduling it on the backend or manually triggering it from the Process Definitions window in the IT-Conductor platform. ITC allows for variables to be specified for each activity, such as SID (system ID), server name, software repository location, etc. Once a process definition is running, you can immediately see the progress as each of the activities change its color to green upon completion.

Figure 5: Process Instantiation for Oracle DB Migration to Azure Cloud (Implementation Completed)

ITC activities that are in blue signifies that the activity was not executed because of the condition was not met during runtime evaluation. Those activities that are in red either signify that the activity failed to execute or was not executed at all because the preceding activity resulted in an output that meets the other condition(s).

Figure 6: Process Instance for the Primary Server Setup (On-Prem) (Implementation Completed)

In the image above, you can see the “If NoArchiveLog” activity is in red color because the output of the “Get Database Log Mode” activity met the condition set for the “If ArchiveLog” activity.

Sample Logs & Scripts

As each activity executes, and when they complete, regardless if they completed successfully or failed to execute, an option to view the logs is available for viewing. While the activity is still running, the log display popup can auto-refresh so you can see the progress. This allows you to see more details that are helpful when troubleshooting failed attempts, resolving the issue manually, and trigger automation for the remaining activities once the issue has been solved.

Figure 7: Provisioning Azure VM Terraform Job log

Figure 8: Installing Oracle Ansible Tasks

Figure 9: PL/SQL Script in ITC Activity

Figure 10: SQL Script in ITC Activity

Figure 11: Standby Server Setup Ansible Job Log

Why Automate SAP OS/DB Migration to the Cloud?

To further understand the need to automate database migration to the cloud, let’s take a look at some of the benefits that organizations can reap right off the bat:

1.  Improved Process Efficiency

Imagine manually implementing a series of activities when migrating databases. Doing the same set of steps over and over again will eventually make you inattentive which can slow down the execution, or worst, result in errors and misconfigurations.

Implementing an automated approach to database migration makes the process more streamlined. With automation, external factors, such as human error, will be eliminated completely. It also improves the running time of the activities and thus, improving the overall process efficiency.

2.  Lower Risks

The reason why database migration is often done manually is because of the high risks that come with executing the activities. When automating a process, you are defining a step-by-step procedure for a particular system or tool to follow. If done correctly, it will considerably lower the risks because the human intervention is being eliminated and the steps are being standardized.

3.  Eliminates Manual and Repetitive Work

Automating database migration incorporates developing predefined process workflows that will be executed automatically. Unless there are errors encountered by the system, human intervention is not needed. Also, as we have mentioned above, automation eliminates the need to perform repetitive work. This should put an end to the hurdles of having to train people over and over again.

4.  Reduced Operational Costs

The decrease in manual labor allows organizations to get more done with fewer resources. Depending on the capacity of a particular system, two or more migration activities can be executed simultaneously. This allows organizations to reduce operational costs attributed to the energy required when running automated activities.

In addition, there is also a reduction in operational costs when it comes to the time saved when executing the migration activity manually versus the automated approach.

5.  Faster ROI

The ability of automated database migration to run by itself without human intervention allows organizations to schedule migration activities unattended, 24/7. This, in turn, results in an increase in the number of databases that will be migrated to the cloud. The lesser time it takes for the automated migration activity to run, the more migration activities will be successfully completed overtime. All of these factors combined drive faster return on investment (ROI) for the business.

Get Your Automation Started with OZSOFT Now

Regardless of the environment and infrastructure setup in your organization, the IT-Conductor platform can easily integrate with your database to perform automated database migration to the cloud. Furthermore, the application installation can be automated as well such as SAP Netweaver ABAP or JAVA. Even clustering can be added for High-availability. We have several GIT repository templates to get you started. We also customize process definitions for each customer to ensure seamless integration with the platform and coherent execution of database migration activities. All this can be done remotely and agentlessly with IT-Conductor cloud platform.