Data Migration from Oracle on premises Database to Big Query

Data Migration from Oracle on premises Database to Big Query

There can be numerous ways to migrate data from a database to Big Query like exporting to CSV, Parquet files, direct DB connection, etc. So in this article, we will focus on two ways- 

  • Using Fivetran connector 
  • Using a python script  

Fivetran connector 

Data Migration from Oracle on premises Database to Big Query

GCP provides numerous connectors for database connection/migration, one of which is Fivetran. 

 Using Fivetran we can establish a direct connection between Big Query and Oracle database for this we need to just configure it by providing the live hosting address of our database and its credentials,  

Once verified then it’s good to go for data migration. 

But using Fivetran connector is not free, it’s chargeable. 

Using Python script 

Data Migration from Oracle on premises Database to Big Query

This method is the best if the Oracle database is at the local system and does not have a hosting address.  

In this using python script, we will establish a type of connection between the Oracle database and Big Query, for that, we need to connect the python script with the oracle database using proper connection string and same way using the credential key provided by GCP we can establish a connection with Big Query. 

 After connection establishment, we need to get the data from the Oracle database for which we perform a SQL query inside python script using Sqlalchemy and Pandas and the result will be the desired data stored in Dataframe

Using Panda’s “to_gbq” method we can easily send our data from the data frame to the Big Query table.  

We can make separate text files for the connection string, SQL query, GCP Big Query credential and read them in python script for easy use. This makes it easier to transfer data; change the SQL script and run the python script, and your data will be transferred.  

Data Migration from Oracle on premises Database to Big Query
Data Migration from Oracle on premises Database to Big Query
Register a Free Cloud ROI Assessment Workshop

Register a Free Cloud ROI Assessment Workshop

Get a Detailed assessment report with recommendations with an assessment report

Schedule free Workshop
Register a Free Cloud ROI Assessment Workshop
Register a Free Cloud ROI Assessment Workshop

Liked what you read !

Please leave a Feedback

0 0 votes
Article Rating
guest
0 Comments
Inline Feedbacks
View all comments

Related articles you may would like to read

How can Docker Containerization Help in Reducing CICD Deployment Costs
0
Would love your thoughts, please comment.x
()
x
Proposals

Know the specific resource requirement for completing a specific project with us.

Blog

Keep yourself updated with the latest updates about Cloud technology, our latest offerings, security trends and much more.

Webinar

Gain insights into latest aspects of cloud productivity, security, advanced technologies and more via our Virtual events.

ISmile Technologies delivers business-specific Cloud Solutions and Managed IT Services across all major platforms maximizing your competitive advantage at an unparalleled value.

Request a Consultation