BDA pitch - AFCEA Portugal

<Insert Picture Here>
Cyber & Big Data Intelligence
Sharon Uziel
Cyber & Big Data Business Development Leader | Oracle Israel
Agenda
1
2
3
2
Introduction
Solution Architecture
Mapping Oracle Engineered Systems & Products
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Agenda
1
2
3
3
Introduction
Solution Architecture
Mapping Oracle Engineered Systems & Products
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
The Problem
• Insider threats!
• Assumption: Even heavily secured networks have
vulnerabilities and probably are already infected (HUMINT ,
lack of procedures, targeted attack)
• Networks become huge and complex
• New threats are not detected by conventional security
technologies ( FW, DLP…)
• How to identify new & unknown threats (APT’s) ?
4
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Solution - Network Monitoring System
• Building infrastructure for large scale cyber defense system
• Full cycle threat detection
• Collection, detection, analysis and reporting systems
• Unknown threats detection based on prediction algorithms
• Behavioral investigation tools to understand what’s happening in the
network
•
•
•
•
5
Recording of all network sessions
Graph database construction for network analysis
Indexing
Reporting and discovery
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Solution Building Blocks
• Collection
• Supports most of the protocol types
• Scalable deployment on any network level
• Transparent updates without downtime
• Aggregation
• distributed file system
• Recording & indexing
• Real-time and batch processing – filtering, aggregation & correlation
• Detection
• Prediction engine generates variants of existing threats
• Detectors searching threats on collected data
• Investigation
• Event management and discovery
• Network situational awareness
6
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Solution - Key Features
• Open architecture and scalable platform
• Low cost Switch/probes enable flexible deployment over the
network
• Central big data distributed system
• Innovative analytical tools
• Open API for external systems and 3rd algorithms
7
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Agenda
1
2
3
8
Introduction
Solution Architecture
Mapping Oracle Engineered Systems & Big Data Appliance
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Cyber Solution Architecture
Internal Network
Net Collector
Analysis Desktop
Port Mirror
Security Officer
All incoming
and outgoing
data packets
Net Aggregator
Log Files
Oracle Big Data
Platform
9
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Functional Architecture
Net Collector
Deep Packet Inspection (DPI)
Packet Multi-Core Processing
Transformation, Aggregation
Net Aggregator
Collection
Graph Repository
Metadata Mgmt.
Rules Repository
Navigated Search
Content Repository
Map & Network Search
Metadata Repository
Reporting
Dashboards & KPI’s
Entity Model
Rules/Policies/Filters
PCAP Transformation
External Repository
Ad hoc Reports
Analytics
Metadata Extraction
Anomaly Detection
Pattern Matching
BI Analytics
Statistics Extraction
Threats Prediction
NBAD Algorithms
Query Tool
Content Extraction
Link Analysis
3rd Algorithms
Unified Query Language
Data Store
Analytical Data Store
Information Security (Data Security & Access Management)
Holistic System Management
System API
10
Information Discovery
Semantic Search
Collector Mgmt.
Transformation
Virtual Probe
Analysis Desktop
Data Store
Staging
Network Probe
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Collaboration
Open UI\SOC\API
Conceptual Architecture
Big Data
Acquisition
Endeca Information
Discovery
Application
Oracle Business
Intelligence EE
NoSQL DB Driver
Real Time Access
Big Data
Connectors
Oracle Exalytics
ORE, OEP,
Endeca
Oracle Advanced Analytics
OLH, ODC, ODI,
External Tables
Batch Processing
HDFS, Hadoop, CDH
Map Reduce
Map Reduce
Map Reduce
ORCH - Stats
Pig - Sessionize
Hive - Activities
Oracle Big Data Appliance
11
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Oracle Exadata
Net Collector
Filtering, Aggregating, Reconstructing Net. Packets
•
Integrated low-cost switch/probes
–
Supports most of the protocol types
–
Scalable deployment on any network level
–
Transparent updates without downtime
•
Supports 5Mbps to 40Gbps of traffic
•
“Virtual Probe” based on standard Linux OS
•
–
Supports various of CPU’s types
–
Single to multi-core processing
–
Enabling robust packets transformation and aggregation on probe level
Embedded real-time data store based on Oracle NoSQL
–
Packets are indexed and organized as part of the DPI process
–
Durability can be configured from memory to disk based
–
Support transactions
–
Data stored and transport as standard Json\Avro
•
Extraction of metadata, content and statistics
•
Smart data transportation
12
–
Sense network workload
–
Transport data based on type, size, time and other configurable policies
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Switch\Probe
•
24GE + 2x10GE L2-4 Switch
•
Integrated 10 Gbps DPI Network Probe
•
Onboard storage
•
Onboard 20Gbps encryption engine
•
Capture & record all network data
–
Full session reconstruction
–
CDR’s, Metadata & statistics
–
Inline & Tap modes of operation
•
DPI based application based firewall
•
Low cost as regular Enterprise Network switch
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Stack Down
Monitoring port
Fan Tray
PS #1
HDD/SSD
Content
storage
PS #2
MULTICORE
CPU
(DPI Probe +
Management)
Packet Processor
Content and payload
•
13
Stack UP
PHY
PHY
12x 10/100/1000
12x 10/100/1000
FLASH
2 x 10GE
PHY
1x 10GE
DRAM
PHY
1x10GE
Net Aggregator
Managing and Analyzing the Collected Data
•
•
•
•
•
•
Multiple data models for different types of analysis
–
Graph, Key-Value store, relational, HDFS all synch between each other
–
Unique network model with inference capabilities
Unified system metadata
–
System meta-model: Support an abstract ontology
–
Defined across all system layers
Query language
–
Unified search engine that serves different applications
–
Support query federation on all system data stores
–
360 query and text search capabilities
Real-time and batch processing
–
Real-time event processing based on dynamic rule-based engine
–
Batch processing for complex network analytics
–
Each sub system feeds each other e.g. complex processing will produce new rules
Unified
Metadata
Low value density
data processing
User Defined
Algorithms
Insights, valuable data and threats will be managed in a separate data store
–
Sharing threats over the cloud
External repositories and API
–
14
Public repositories like known vulnerability can be imported to the system
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Query
Language
API
HDFS
Semantic Store
Unstructured & Semistructured Data
RDF Store, Inference,
Reasoners
Files
Filter
Index
Classify
Correlate
NoSQL
Unqualified Data Store
Unprocessed & Processed
Value –Pair Data
Standard FS, NAS, SAN, SSD,
etc.
Statistical
Analysis
High value data
processing
Relational &
NoSQL ETL
Analytical repository
–
Unified
Services
Enterprise
Applications
Relational
Other
Data Warehouses &
Transactional Systems
In memory, Graph DB,
Metadata, Public Ontology
Store
Analysis Desktop
Rich visualization and Information discovery
•
•
•
Interception view
–
Full session\s to session\s reconstruction
–
Including content e.g. VOIP, emails, audio, video, web sites and so on
Network & map view
–
Graph layout - Situational awareness status
–
Search engine and algorithms: SNA, NBAD, path finding, etc.
Guided navigation
–
•
Classification, filtering, tagging, sorting and so on
Data & text analytics
–
Sentiment analysis
•
Interactive dashboards
•
Reporting and publishing
15
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Threat Detection
• The Problem: Cat & Mouse Security
• Cyber defenders spend billions, hackers easily bypass their
defenses.
• 98% of malware are variants of known malware, modified to
evade security measures. Remaining 2% reuse known exploits
and techniques
• Building a complete attack chain from scratch is practically
impossible
• Un-patched environments are specially vulnerable to exploits’
reuse
16
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Attackers Reuse Techniques
Code ReUse
Incorporating code from one malware into another
Semantic Equability
Changing code, preserve functions
Different path, same target
Changing/Adding functions to evade heuristics
17
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Threat Detection – The Solution
The Problem:
The Solution:
Reactive, Cat-and-Mouse
security can’t fight Malware’s
Proliferation Rates
Predictive Security:
forecasting and preventing
tomorrow’s malwares
18
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Threat Detection – The Method
“Evasive, yet Malicious”
Understanding malware writers’ considerations
Simulation
Exploring the space of potential malware: predicts hundreds of thousands
of potential variants
Behavior Extraction
Building effective detectors using machine learning
Leveraging detection
Moving along the cyber kill chain
19
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
20
|
© 2011 Oracle Corporation
|
Insert Information Protection Policy Classification from Slide 18
Agenda
1
2
3
21
Introduction
Solution Architecture
Mapping Oracle Engineered Systems & Products
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Oracle Big Data Platform
Oracle’s Big Data Strategy
Big Data Appliance
Exadata
InfiniBand
InfiniBand
Stream
22
Acquire
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Organize
Analyze & Visualize
© 2011 Oracle Corporation
Mapping Engineered Systems
Unified
Metadata
Low value density
data processing
User Defined
Algorithms
Unified
Services
Query
Language
API
HDFS
Semantic Store
Unstructured & Semistructured Data
RDF Store, Inference,
Reasoners
Files
Filter
Index
Classify
Correlate
NoSQL
Unqualified Data Store
Unprocessed & Processed
Value –Pair Data
Standard FS, NAS, SAN, SSD,
etc.
Statistical
Analysis
High value data
processing
Relational &
NoSQL ETL
Enterprise
Applications
23
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Relational
Other
Data Warehouses &
Transactional Systems
In memory, Graph DB,
Metadata, Public Ontology
Store
Engineered Systems for Cyber Intelligence
Complete Infrastructure for Structured & Unstructured analysis
Big Data Appliance
HDFS
Stream
24
Exadata
Oracle NoSQL
Database
Oracle Loader
for Hadoop
Enterprise
Applications
Oracle Data
Integrator
Acquire
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Exalytics
Hadoop
(MapReduce)
Organize
DW
In-Database
Analytics
Data
Source
Analyze
Analytic
Applications
Decide
Oracle Big Data Connectors
ORACLE LOADER FOR HADOOP
• High Performance Data Access and Load
• 15TB per hour
•
•
•
•
Map-reduce and parallel query processing
Access to Hive tables and HDFS files
Convert data into Oracle ready data types
Load pre-processed data online or offline
• Online: Pre-process and load in the same job
• Offline: Write out data files on HDFS for load later
• Automated generation of external table
• Manage all your data in a single schema
• Use rich Oracle SQL instead of limited Hive queries
SQL Query
MAP
REDUCE
MAP
MAP
REDUCE
SHUFFLE
/SORT
MAP
MAP
MAP
External
Table
REDUCE
SHUFFLE
/SORT
HDFS Client
REDUCE
Oracle Database
Oracle Big Data Appliance
ODI FOR HADOOP
Transforms via MapReduce
Oracle Data
Integrator
Activates
Oracle Loader
for Hadoop
Oracle Big Data Appliance
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
OSCH
ODCH
ODCH
REDUCE
• Enhanced security
25
ORACLE SQL CONNECTOR FOR HDFS
Loads
Oracle Exadata
Unified Analysis with R
•
Oracle R Advantages
•
•
•
•
•
Keep the R tools
Keep the data where it sits (Relational or HDFS)
Keep the SQL Based BI Tools
Scale to LARGE data sets
Client Host
R Engine
Oracle Big Data
Appliance
Oracle Exadata
R Engine
ORE*
ORE*
ORHC
ORHC
Hadoop
Software
MapReduce
HDFS Nodes
Oracle R Enterprise (ORE)
•
•
•
•
ORACLE R FOR HADOOP
Database-centric environment for R analysis
In-database advanced analytics algorithms exposed through R
Eliminate R client memory limits
Oracle R Connector for Hadoop (ORCH)
•
•
R packages enabling Big Data analytics from R
to leverage a Hadoop Cluster with HDFS and MapReduce
from R
Prepackaged advanced analytics algorithms
Transparent manipulation of HIVE data
•
•
26
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
R Engine ORE*
ORHC
Oracle XQuery for Hadoop
Input / Output Data Formats
Input
HDFS
Oracle
NoSQL DB
Output
Text
Text
HDFS
CSV
Oracle
NoSQL DB
JSON
Oracle
NoSQL DB
XML
JSON
Avro
CSV
Avro
Oracle Database
XML
27
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Map/Reduce Job Counters
Oracle NoSQL
Scalable, Highly Available, Key-Value Database
28
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Application
Application
NoSQL DB Driver
NoSQL DB Driver
Storage Nodes
Storage Nodes
Data Center A
Data Center B
Endeca Information Discovery
Diverse and changing
information
Structured
Automatically unified and
enriched in Endeca Server –
no predefined model
required
Drag-and-drop application
composition
Interactive search,
navigation and analytics for
exploration and analysis
Semi-Structured
Unstructured
30
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Endeca
Information
Discovery
[email protected]
Dynamic Rule Engine
Real-time Process and Respond
31
Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Oracle Event Processing
Real-time Data
Streams
Immediate Automatic
Responses
Console Alerts
Aggregate, Correlate, Filter
Pattern Detection
SMS Message
Event capture
Workflow Initiation
Enrichment
Virtual Data
Repository
Real-time Dashboards
Securing Big Data
• Big Data must be protected
and audited
• This is no different than
critical data stored in an
RDBMS
The 3 A’s of Security
Securing Big Data
Authenticate Users
Authorize access to data and services
Audit activity and users
Kerberos Provides Strong Authentication
• Kerberos automatically
configured
• Strong authentication for
• All Hadoop services
• Oracle Big Data Connectors
Key Distribution
Center
Authenticate / Get
Ticket Granting Ticket
Kerberos Service
Registration
• Ensure users and services
are who they claim to be
Access Service Using Ticket
Client
Key Distribution
Center (Optional)
Big Data Appliance
Oracle Audit Vault and Database Firewall
Who? did What? When?
One
Consolidated, secure
repository for all audit data
Hadoop
Non-Relational Data
Audit Vault
Databases
Relational Data
Operating
Systems
Centralized platform for
audit reporting, alerting
and policy management
Data Encryption
•
•
•
•
Oracle Advanced Security Option
HDFS Network Encryption
HDFS On-Disk Encryption
Secure configuration for Impala,
HBase and Cloudera Search
Oracle
Systems
Oracle Engineered
Integrated Solution
Stack
ACQUIRE
39
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
ORGANIZE
ANALYZE
DECIDE
Follow us on
Questions
40
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Oracle Confidential

Download Report