No, you can't generate compressed trail files now. You can only configure Oracle GoldenGate to compress trail files in its extract pump process where you can specify COMPRESS in the RMTHOST option. For example,
This feature reduces the bandwidth requirements. COMPRESS typically results in compression ratios of at least 4:1 and sometimes better. However, compressing data can consume CPU resources. Oracle GoldenGate (collector) decompresses the data before writing it to the trail. Therefore, you can't see the compressed trail files. (OG-51230)
Oracle GoldenGate 12.3 introduces a new Microservices Architecture (MA) to Oracle GoldenGate . The new design includes the following components:
The MA architecture is now supported by GoldenGate for Oracle Database Only.
All the services provide the REST APIs via HTTPs and JSON. The RESTful interfaces enables the remote configuration, monitoring and administration of Oracle GoldenGate deployments.
Oracle GoldenGate releases the 12.3 release on 8/18/2017. In this release, Oracle GoldenGate delivers:
For the Oracle Databass
For SQL Server
For DB2 z/OS
For DB2 i
Please refer to Oracle Data Integration Blog for up to date news and articles.
Oracle GoldenGate extract throughput can be represented in either the number of SQL operations or the number of data bytes processed by Oracle GoldenGate.
Oracle GoldenGate can replicate data across time zones. This blog explains the timezone related features and discusses the best practices.
You have to synchronize replication hub's operating system clock with the source/target database clock.
Time Zone Related Features
There are three places where time zone matters: the time-based data, the lag time report, the trail file I/O time.
Best Practices for Handling Time Zones in Remote Capture and Delivery
When performing remote capture and delivery across time zones, such as the replication hub architecture shown as follows, you need to configure Oracle GoldenGate to be aware of the time zone differences between the source database and the OS running Oracle GoldenGate process (including extract and replicat).
You can choose any time zone for a replication hub (e.g., an Oracle GoldenGate Cloud Service (GGCS) instance) because it doesn’t affect the replication.
Which time zone should I choose for my GoldenGate replication hub?
You can choose any time zone for your replication hub. No matter where you run the GoldenGate extract and replicat processes, the timezone in data records stay intact. They are replicated in source database timezone along with the source database time zone information in the trail file. If you have a choice, then choosing the same time zone as the source database.
If the source database timezone and the Oracle GoldenGate timezone have to be different, i.e. when you use downstream capture or remote capture, you then need to use SETENV to set to source DB OS time zone and use send <extract/replicat_group> status to retrieve the lag report. It is recommended to:
Lag time is an important metric to monitor the quality of a real-time data replication. This blog explains what a lag time is, how to track lag times in Oracle GoldenGate and the typical methods that you can use to reduce to the lag time.
What is Lag Time?
Lag time or sometimes just call the "lag" refers to the period between the database operation (DML or DDL) is committed on the source and the current point in time. In Oracle GoldenGate, there are lag times for the extract and replicat:
How to resolve a lag unknown issue?
If GoldenGate instance runs on an OS in a timezone different from the source database, you can get the lag unknown issue. Please refer to How to Resolve the Lag Time Unknown Issue for the details.
How to Monitor the Lag Time
To monitor lag time, you can query the extract and replicat processes or create a heartbeat table to check the end-to-end lag.
How to Reduce the Lag Time
To reduce lag time, you need to know what processes contribute to the lag time :
Created 7/2/2015, Last Updated 11/27/2017
Data replication tools like Oracle GoldenGate allows you capture, transform and deliver the real-time data across the heterogeneous platforms. The replication technology is used in a wide range of IT initiatives. But if this is the first you look at the technology, you may ask "Why do I need to replicat data in the first place?" Let's talk about this in this blog.
In summary, there are four main reasons:
1. Simplifying the IT Management
IT management is a complicated and stressful process. Replication tools like Oracle GoldenGate can streamline the process and reduce the stress in the following ways:
2. Reducing the Total Cost of Ownership (TCO)
Everyone wants to get more things done with less spending. Reducing the cost is a forever hot topic in the IT planning. Replication tool can save your cost in the following ways:
3. Setting up for Maximum Availability
You can't afford downtime for mission critical databases. Oracle GoldenGate is part of the database maximum availability strategy so that the system to continue working even if some of its parts have failed. . These includes:These are compliment to the native Database HA solutions.
4. Enabling the Creative Use of Data
The creative use of data can give you the advantages in business such as keeping the data geographically close to users for low latency and scaling out the number of machines that can provide data services. You can use Oracle GoldenGate to create the following services:
These are just some examples. You can use the technology in your own creative ways. Enjoy using the tool.
Created 8/15/2017, Last Updated 11/17/2017
Performance is essential for real-time data replications. This blog discusses Oracle GoldenGate replication performance by examining the following topics: clarifying the performance requirements, designing the system for high performance, and tuning the performance.
Configuring Oracle GoldenGate for the Best Performance with
1. Clarify the Performance Requirements
The performance requirements of real-time data replications typically refer to how much data can be delivered and how quickly the data can reach the target. How much data can be delivered is measured by the throughput, which is the amount of replicated data over time. How quickly the data can reach the target is represented by the lag time, which is the amount of time taken for a data change to be propagated to the targets. Real-Time data replication frequently asks for low latency and high throughput. You might hear your initial requirement like this:
"Can you synchronize my databases within 1 second latency?"
This is a very typical unclear requirement. It doesn't tell you how much data to replicate, nor if the latency is the maximum latency or an average latency. You need to clarify the requirements first. You can clarify the requirement with the following questions:
I need to replicate 200GB hr redo log with 100GB/hr trail files in 1 second average latency.
With this, you can start designing the replication system.
2. Designing Oracle GoldenGate Replications for High Performance
When designing a replication solution, You would take the following steps:
More to Read
For Oracle Database replications, please refer to the following Oracle MAA Technical White paper for the details :
2.1 Prepare the Server to Run Oracle GoldenGate
The server running the Oracle GoldenGate instance should have enough CPU and storage resources. The disk I/O speed is also crucial because Oracle GoldenGate replication needs a lot of for disk I/Os such as read/write trail files.
After setting up a server, you can perform a baseline test with the typical workload of your applications:
The result indicates the maximum replication throughput in the given server configuration. If the result can't meet the perforamnce requirement, you then need a better server.
2.2 Prepare the Network
For the network, you need to delivery data for Oracle GoldenGate from pre-generated trail files and find out the throughput and latency. When performing network estimation, you need to know answers to the following questions:
2.3 Configure the Databases
You also need to avoid unnecessary database index/triggers on the target.
2.4 Configure the Oracle GoldenGate processes
For data replication processes, you need to consider the following configurations:
3. Tuning the Performance
Tuning the performance should start from to find out the performance bottlenecks. Then, you can modify the configuration of your extracts, pumps and replicats. The following are the common performance problems and the tuning tips:
Created: 7/19/2017, Last Updated: 10/19/2017
This blog discusses when you would use the COMPRESSUPDATES or COMPRESSDELETES configurations in Oracle GoldenGate.
You would be cautious when using NOCOMPRESSUPDATES in Oracle GoldenGate because if you have tables with a lot of columns (e.g., 200 columns), this configuration will significantly increase the trail file size and thus impact the replication performance. If you don't have the proper key or unique indexes on the source database, you can consider using the KEYCOLS.
The similar applies to the COMPRESSDELETES. The supported databases are DB2 LUW, DB2 z/OS, Teradata version 12 or later, SQL Server, and Sybase. Note that this doesn't support for MySQL database.
(Created: 11/25/2013, Last Updated: 10/16/2017)
This blog answers the questions about Oracle GoldenGate including where it came from, what is the current status, and what is coming in the future.
Oracle GoldenGate - A Brief History
GoldenGate was a product of a small San Francisco-based company called GoldenGate Software Inc. The company was founded by Eric Fish and Todd Davidson in 1995. Back in the 90's, the GoldenGate product was designed for the fault tolerant Tandem computers(later known as HP NonStop), it then became popular within the financial industry as the resilient and fast data replication solution. For example, banks used GoldenGate software to send transaction data from their ATM networks to central servers.
By the time of 2009, GoldenGate is well known as a real-time data replication solution with more than 400 customers and 4000 solutions. In 2009, GoldenGate Software was acquired by Oracle . Now (by 2017), Oracle GoldenGate is part of the Oracle Data Integration solution with more than 5000 customers.
Oracle GoldenGate - The Product Now
Oracle GoldenGate is built on the change data capture (CDC) technology providing real-time data replication. Oracle GoldenGate now supports replications across heterogeneous systems, includes tools to monitor, manage and design/deploy Oracle GoldenGate replications, and enables flexible deployment with modular-based architecture.
The product portfolio now includes the following :
Oracle GoldenGate - The Future
Oracle GoldenGate would continue its journey to the cloud, enable rapid deployments, simplify its distributed management capability, and extend its integration with the data integration technology such as Oracle Data Integrator and Oracle Data Governance to provide comprehensive solutions.
Created 8/15/2017, Last Updated 10/12/2017