Oracle Data Integration - Upstate New York Oracle Users Group

Oracle Data Integration
Real Time Access to Real Time
Information
Tony Velardo
DIS Sales Manager
Valarie Bedard
Principle Sales Consultant
1
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle Data Integration
Moving Data
to Transform Business
IT Obstacles to Unifying Information
Avoiding the Accidental Architecture
Analytics
Packaged
Applications
Custom Reporting
Business
Intelligence
SOA Applications
Data
Replication
Data
Migration
Data Silos
Data
Warehousing
Data Marts
Data
Federation
Data Hubs
Batch Scripts
Data Access
SQL
Custom
OLTP & ODS
Systems
Fragmented
Data Silos
2
Data
Warehouse, Data Mart
Oracle
PeopleSoft, Siebel, SAP
Custom Apps
Slow
Performance
Out of sync
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Files
Excel
XML
Lack of
Trusted Data
Java
OLAP
What’s the
cost?
Choosing the Right Data Integration Platform
Key Criteria for Maximum ROI from Cloud
Enterprise-Class
High
Performance
Fast Time to Value
3
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
• Comprehensive set of capabilities
• Flexible to address multiple use cases
• Support for heterogeneous platforms
• High performance to handle spikes
• Easy and cost-efficient to scale
• Easy to deploy and manage
• Pre-integrated, certified solutions
• Low cost of ownership
Oracle Data Integration
Complete Offering for Enterprise Data Integration
Legacy
Oracle Data Integrator
Applications
Oracle GoldenGate
OLTP
Oracle Enterprise Data Quality
Unstructured
Modernization
4
Oracle Customers Report:
Active Data Guard
Custom
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
MDM
BI
 Complete and best-of-breed
approach for enterprise data
integration
 Maximum performance with
lower TCO, ease of use and
reliability
 Certified for leading
technologies to deliver fast
time to value
 80% lower TCO
 Five times higher performance
 70% reduction in development costs
Big Data
Synchronization
SOA
Data Integration Solutions
High Availability
Do you need 24/7 system availability?
Is your standby database sitting idle?
Is planned downtime at least as costly as unplanned?
Need to synchronize data across geographies?
Consolidation / Migration
Want to get onto lower cost standard infrastructure?
Can’t afford downtime?
Does your data cost $$ or make $$?
Why not fix data assets as well as HW/SW assets during an
infrastructure move?
Business Intelligence
Need fresh data 24X7 to optimize operations?
Mission-critical OLTP?
Minimize impact on source systems?
Need operational visibility in real time?
Growing data volumes and shrinking batch windows?
5
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
• Oracle Active Data Guard ensures
business continuity and resource
utilization
• Oracle GoldenGate reduces planned
downtime and enables Active-Active
• Oracle GoldenGate enables zero
downtime migrations and upgrades
• Oracle Data Quality for ODI moves
and improves data so it can be
trusted
• Oracle GoldenGate provides realtime data for analysis
• Oracle Data Integrator (ODI)
populates DW with high performance
and low TCO
Oracle GoldenGate
Real-time Data Integration
Legacy
Applications
Oracle GoldenGate
Log-based Change Data
Capture and Delivery
Bidirectional Replication
OLTP
Unstructured
Heterogeneous Sources
and Targets
Reliability and
Transaction Integrity
CEP Integration,
Real-time Events
6
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
 High-performance, lowimpact real-time data
integration
 Timely data for improved
business insight
 Continuous availability for
24/7 operations
Who’s Using GoldenGate?
• Leader in real-time data integration solutions
• Over 3,000 customers including Fortune 500
companies, representing Financial Services,
Communications, Healthcare, Public Sector,
Retail, Utilities industries
• 4 of top 5 largest commercial banks
• 3 of top 3 busiest ATM networks
• 5 of top 5 financial data services
companies
• 5 of top 5 telecommunications providers
• 3 of top 5 largest food & drug stores
7
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle GoldenGate Under the Covers
That’s the Magic of Macy’s
Shopping at the worlds largest retails and your credit card transactions are moved in real-time to data
centers around the world, before being approved, using Oracle GoldenGate (Retail Decision)
Good TV. Better TV. DirecTV.
If you call DirecTV customer service, your call data will move to their data warehouse via Oracle
GoldenGate, so the rep can provide you with personalized recommendations and promotions.
Say it with Flowers
Ordering from 1-800-Flowers, Oracle GoldenGate will move your order to their reporting system
to help process and track your order seamlessly
No Charge ATM Fees
3 of the 3 busiest ATM Networks use Oracle GoldenGate for Continuous Availability.
Get Sick in Pittsburgh
If you get a health check at UPMC, the clinical applications that doctors use are supported by
Oracle GoldenGate to enable high-availability
8
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
GoldenGate for Banking / Financial Services
Bank of America
High availability for ATM network
Eliminate downtime during database,
hardware & Siebel migrations
High availability for app. that assesses
online access risk
State Street Bank
Offload reporting from hosted
applications
High availability for private Exa cloud
Integrate relational data with
enterprise service bus
JP Morgan Chase
Eliminate planned downtime for
retail banking app.
Eliminate planned downtime for
treasury app.
Transform production data and
synchronize with reporting database
UBS
Standardize to reduce complexity &
cost
Distribute data among internal
applications
High availability for planned &
unplanned downtime
Fidelity
Eliminate planned downtime for
Siebel app.
High availability for tax accounting
app.
High availability for trade confirmation
system
SunTrust
Fraud detection for online banking
app.
Integrate relational data with
enterprise service bus
Reduce or eliminate batch extracts
Capital One
High availability for enterprise
imaging app.
Eliminate downtime during database
upgrades
Off-load reporting for car loan
origination app.
Chicago Mercantile Exchange, Deutsche Bank, US Bank, Swedebank, Visa, Prosa, FNBO, First Data, Retail Decisions, Zagreb
Bank, ADP, American Express, VISA, Desjardins Bank, Hyundai Securities, Bombay Stock Exchange, Nomura, RBS, Bankart
9
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle GoldenGate Across the Enterprise
Databases
O/S and Platforms
Oracle GoldenGate Capture:
Oracle
DB2 (all platforms)
 Microsoft SQL Server
 Sybase ASE
 MySQL
Teradata
 Enscribe
 SQL/MP
 SQL/MX
JMS message queues

Linux

Sun Solaris
Oracle GoldenGate Delivery:
All listed above, plus:
 PostgreSQL
TimesTen, Coherence
 Flat Files for Netezza, Greenplum, ETL products

10
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Windows 2000, 2003, XP
HP NonStop
HP-UX
HP OpenVMS
IBM AIX
IBM z Series
IBM i Series
zLinux
Oracle GoldenGate Use Cases
Low-Impact Real-Time Data Integration & Transactional Replication
Zero Downtime
Upgrade & Migration
New DB/HW/OS/APP
Highly Available /
Disaster Recovery
Mainframe
Fully Active Distributed DB
Query Offloading,
Disaster Recovery
Log-based,
changed data
Reporting Database
Real-time BI,
Operational
Reporting, MDM
Database
ODS
Data Integrator
Data Warehouse
Data Synchronization
across the Enterprise
Global Data Centers
Message Bus
Message Bus
11
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Event Driven
Architecture,
SOA
Sample GoldenGate Mainframe Customers
 Chicago Mercantile Exchange (DB2/zOS to/from
Oracle)
–
Migration off mainframe
 AutoZone (DB2/zOS to DB2/LUW DPF &
GreenPlum)
–
 AARP (DB2/zOS to Oracle and JMS)
–
Integrate mainframe data with downstream
applications and SOA/EDA
 Liberty Mutual (DB2/zOS & DB2/LUW to Teradata)
–
Real-time feed to data warehouse for “single view
of the business”
 Macy’s (DB2/zOS to Exadata)
–
Real-time integration of inventory for faster
shipping & replenishment
 Discover (DB2/zOS to Oracle)
–
Mainframe data integration with downstream
applications
 ACS-Xerox (DB2/zOS to Oracle)
–
12
Report off-loading to reduce cost of reports
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Report off-loading to save mainframe cycles and
speed reporting
 Citigroup (DB2/zOS to Oracle)
–
PeopleSoft application migration off mainframe
 Zagreb Bank (DB2/zOS to Exadata via ODI &
DB2/zOS to DB2/zOS)
–
Data center consolidation, Data warehouse feed
 Sears (DB2/zOS to GreenPlum)
–
Report off-loading
 Intercontinental Hotels Group (DB2/zOS to Oracle)
–
Mainframe data migration and sharing
 CIBC (DB2/zOS to HP NSK)
–
Data integration with mainframe and HP Non-stop
Across the Enterprise
Market Transparency with Real-Time Data Access
Business Challenges:

High rate of analysis required to ensure the market is functioning
properly

The ‘freshest’ data is required to track pricing, bidding and
purchasing so the correct ‘buy’ and ‘sell’ price is displayed

Trades must be monitored in ‘real-time’ to ensure market
visibility and integrity

Performing analysis against the production system negatively
impacts system performance

Alternatively, relying on nightly ‘batch’ loads caused delays in
uncovering potential issues
GoldenGate Solutions:

Live Reporting from Comprehensive Bid Management System to
an ODS

Real-time Data Integration to monitor market performance

Zero-downtime database upgrade from Oracle 9i to Oracle 11g
(not depicted)
Results:

Real-time transaction management, issue resolution and
automated analysis

Better marketplace performance and lower cost of ownership
13
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Live Reporting & Real-Time Data Integration
Real Time Pricing,
Load, Demand (OISR)
ODS for Reporting &
Analytics
Market Data (MIS)
Historical Data
Market Monitoring
Performance
Database DW
“GoldenGate has provided the data integration solution
we were looking for. We look forward to rolling this
product out for other projects across the organization.”
-John Bubb, Product Manager, Business Intelligence
Products The New York Independent System Operator
Across the Enterprise
Site License – Multiple Implementations of GoldenGate
Pervasive Use Today
Initially had handful of uses in 2008. Has
grown to over 50 implementations since
then. Combination of reducing barriers to use
technologies and spread of GoldenGate
knowledge.
CHALLENGES / OPPORTUNITIES
• Globally distributed bank with needs to
integrate data and ensure business
resiliency
• Multiple types of databases requiring
common data sharing in all facets of bank
• Experienced online verification outage that
increased risk to bank and impacted
customer satisfaction
14
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Some of the GoldenGate Implementations
• Supports sign-on authentication optimizing
performance, risk avoidance, and supporting
“always there” policy
• Multi-master global wire transfer application
• Zero downtime DB migration to new version
using phased approach (e.g. Exadata
consolidation project)
• Consolidation of wholesale credit from 50+
disparate data sources
• Always available, zero data loss reference
architecture
• ATM application fault tolerance
• Real-time feed to data warehouse and
business intelligence systems
• Payments application integration with
messaging technology for mass broadcast
Oracle GoldenGate Runtime Architecture
Capture: committed transactions are captured (and can be filtered) as they occur
by reading the transaction logs.
Trail: stages and queues data for routing.
Pump: distributes data for routing to targets.
Route: data is compressed, encrypted for routing to targets.
Delivery: applies data with transaction
integrity, transforming the data as required.
Manager
Source
Oracle & Non-Oracle
Database(s)
15
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Manager
Bi-directional
Target
Oracle & Non-Oracle
Database(s)
Oracle GoldenGate Adapters
ANY OGG CAPTURE: captures data from any source and creates
an OGG trail for normal routing and delivery
OGG for FLAT FILE DELIVERY: generates data and control
files from trails for batch consumption by third party systems
Oracle GoldenGate
for Flat File
CAPTURE
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
FILE
-FILE
FILE
FILE -UDF
UDF
FILE
UDF
FILE
UDF
FILE
-FILE
FILE
FILE -UDF
UDF
FILE
UDF
FILE
UDF
CONTROL
FILE
FILE DELIVERY
DATA
FILES
LAN / WAN / INTERNET
OVER TCP/IP
Source System
Target System
OGG for JAVA JMS CAPTURE: captures data from JMS text
messages and creates an OGG trail for normal routing and delivery
OGG for JAVA DELIVERY: delivers change data from any OGG
Capture to Java APIs or directly to JMS as formatted messages
Oracle GoldenGate
for Java
JMS DELIVERY
JMS CAPTURE
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
TRAIL
FILE
FILE
FILE
FILE--UDF
UDF
FILE
UDF
FILE
UDF
FILE
FILE
FILE
FILE--UDF
UDF
FILE
UDF
FILE
UDF
LAN / WAN / INTERNET
OVER TCP/IP
Source System
16
16
Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
JAVA DELIVERY
Target System
Oracle GoldenGate Flexible Deployment Models
Unidirectional
Bi-Directional
Peer-to-Peer
Query Offloading
Zero-Downtime Migration
Hot Standby or
Active-Active for HA
Load Balancing
Multi-Master
Broadcast
Integration/Consolidation
Data Distribution
Data Distribution
Data Warehouse
via Messaging
BPM
BAM
CEP
17
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle GoldenGate – Deliver Many
Integration Example
 GoldenGate’s flexibility allows for multiple integration schemes
 The Capture process is always the same
 Delivery can be through
ETL
Tool
– Staging tables
Integration via Staging Tables
– Intermediate flat files
– Messaging
Delivery
Integration via Files
Trail
Capture
Data
Pump
+ UE
data
data
files
data
files
files
ETL
Tool
Integration via Messaging
Data
Pump
+ UE
ETL
Tool
JMS / MoM
Queue or
Topic
18
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Management Pack for Oracle GoldenGate
Administration & Monitoring
Oracle GoldenGate Monitor & OEM
19

Centralized management of
GoldenGate modules

Advanced monitoring, alerting
(SNMP support), lag graphs,
and historical repository.

Browser based access to error
logs and process reports

Enterprise Manager Plug-in
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle GoldenGate Veridata
Data Comparison Without Downtime
Oracle GoldenGate Veridata

identifies and reports data discrepancies
between heterogeneous databases
Reduce financial/legal risk exposure
Speed and simplify IT work in comparing
data sources
No disruption to business systems
Improved failover/migration
Confident decision-making and reporting





20
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Streams Statement of Direction
– Oracle Database Upgrade Guide, Version 12c
http://docs.oracle.com/cd/E16655_01/server.121/e17642/deprecated.htm#BABEAJJE
Oracle Streams is deprecated in Oracle Database 12c and may be
desupported and unavailable in a later Oracle Database release.
– Oracle’s stated direction is to use Oracle GoldenGate for replication
implementations in place of Oracle Streams.
– Oracle Database Advanced Queuing is independent of Oracle Streams
and continues to be enhanced.
– Oracle Database Advanced Replication is deprecated in Oracle
Database 12c. Use Oracle GoldenGate to replace all features of
Advanced Replication, including multi-master replication, updatable
materialized views, hierarchical materialized views, and deployment
templates.
21
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Join the Data Integration Community
LinkedIn
“Oracle Data Integration”
Twitter or Facebook
twitter.com/ORCLGoldenGate
facebook.com/OracleDataIntegration
Oracle’s Data Integration blog
blogs.oracle.com/dataintegration
Oracle GoldenGate YouTube channel
youtube.com/oraclegoldengate
MOS Knowledgebase
http://support.oracle.com/
Oracle Learning Library
http://apex.oracle.com/pls/apex/f?p
=44785:1:0::NO
Oracle
Technology
Network
Oracle
Technology
Network
OTN: GoldenGate
http://www.oracle.com/technetwork/middle
ware/goldengate/overview/index.html
Oracle’s Public Help Forum
http://forums.oracle.com/forums/forum.j
spa?forumID=860&start=0
Oracle.com/goto/dataintegration
22
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
23
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Download Report