Top 10 Tricks and Tools of an Oracle EPM Administrator

Oct 2014
Top 10 Tricks and Tools of an Oracle
EPM Administrator
Effective EPM Administration
Nicholas King
Database Administrator
Purpose and background
Agenda: Tips & Tricks for...
●
●
●
Improving uptime by building a troubleshooting toolkit
Automation / working effectively / less toil
Security / auditing
Nicholas King
●
●
6 years as Unix sysadmin and software development
8 years Hyperion infrastructure consulting
=14 years administering complex enterprise systems
Tip 1: Manual Troubleshooting
Focus on Uptime
1
#
Tip
Too many logs
Search for *.log on one server
Effective
troubleshooting
depends on getting
useful error messages
from logs.
Too Many Logs!
Types of Logs
●
●
●
●
●
Domains: WebLogic Logs
user_projects\domains\EPMSystem\servers
Diagnostics: Application Logs
user_projects\epmsystem1\diagnostics\logs
Windows Event Viewer
HFM Error Log Viewer
FDM Outbox
1
#
Tip
Demo
Demo: Manual Troubleshooting
Tip 2: Log Analysis Utility
Find Problems Fast
Using the Log Analysis Utility
2
#
Tip
What if you could scan hundreds of logs at once?
●
Step 1: Search folder < 60 min old
loganalysis.bat -system -tmin 60
-d search_directory -o output_filename
●
Step 2: Search both “domains” and “diagnostics”
loganalysis.bat ...
-d d:\Oracle\Middleware\user_projects\ domains
-o domains_report
loganalysis.bat ...
-d d:\Oracle\Middleware\\user_projects\epmsystem1\ diagnostics
-o diagnostics_report
●
Step 3: Run against multiple servers using UNC path
loganalysis.bat ...
\\svr1\d$\Oracle \Middleware\user_projects\epmsystem1
\diagnostics -o node1_diagnostics
Log Analysis Utility on multiple servers
start cmd /C "loganalysis.bat -system -tmin 60 -d
\\hsssvr\d$\Oracle\Middleware\user_projects\domains -o
web1_domains "
start cmd /C "loganalysis.bat -system -tmin 60 -d
\\hsssvr\d$\Oracle\Middleware\user_projects\epmsystem1\diagnostics
-o web1_diagnostics "
start cmd /C "loganalysis.bat -system -tmin 60
-d \\hfmsvr\d$\Oracle\Middleware\user_projects\domains -o
hfm1_domains "
start cmd /C "loganalysis.bat -system -tmin 60
-d
\\hfmsvr\d$\Oracle\Middleware\user_projects\epmsystem1\diagnostics
-o hfm1_diagnostics "
Server 1
Server 2
...
Server x
Server y
Server z
2
#
Tip
Log Analysis Utility output
Each browser tab contains a different report
hss_diagnostics, hss_domains
ra_diagnostics, ra_domains
hfm_diagnostics, hfm_domains
2
#
Tip
2
#
Tip
Demo
Demo: Log Analysis Utility
2
#
Tip
Log Analysis Utility comparison
Advantages
●
●
●
●
●
●
Targeted search for just errors
No experience necessary
Simple, first pass at log analysis
Hundreds of files scanned in
seconds
Easy to read HTML report
Detailed messages can lead
to root cause
Disadvantages
●
●
Scans Oracle log format only
Still need logs to get context
about activities before and after
error message
Tip 3: Validate Utility
Tiny Testing Tool
Introduction to Validate Utility
What does it do?
●
A custom test suite with hundreds of intricate tests
Invocation:
user_projects\epmsystem1\bin\validate.bat
3
#
Tip
Validate Utility internals
3
#
Tip
user_projects\epmsystem1\config\validation\11.1.2.0\vtconfig.xml
hat_tmp\hatInput.vbs > diagnostics\logs\validation\hat.output
----STARTING VALIDATION SCRIPTS---LOGGING IN HFM....
CREATING APPLICATION....
OPENING APPLICATION....
LOADING SECURITY....
LOADING METADATA....
3
#
Tip
Validate Utility comparison
Advantages
●
●
●
●
Specialized test suite
Intricate tests - much more
than just a connectivity test
HTML based output is
easy to read
Pinpoint detailed internal
errors
Disadvantages
●
●
●
Errors can be missed if
they do not fail a specific
test
Failed tests may not
identify the root cause
Must be run on each node
Tip 4: LCM Automation
Duct Tape for Quick Repairs
Lifecycle Management automation
4
#
Tip
Nightly Automated LCM Exports to Date Stamped Folder (versioning)
●
Partial restore without downtime
●
●
●
●
Financial Reports
Shared Services Security
Internal HSS registry
Application artifacts - planning forms, EPMA hierarchy...etc
How to Invoke
●
●
Location: D:\Oracle\Middleware\user_projects\epmsystem1\bin
Utility.bat XML_File
Simple: only need an XML file
●
Every LCM export in SS contains a template import.xml and export.xml.
Just add username, password to XML file.
4
#
Tip
Demo
Demo: Versioned Financial Reports
Tip 5: Copy Provisioning
Establish Security Fast
5
#
Tip
Copy provisioning introduction
Clone/mirror Shared Services security from one app to another
FMAPP2
FMAPP3
user1: Consol Admin
user2: Post Journals
Provisioning
Why Copy Provisioning?
●
●
●
●
Cloning security for Essbase cubes
Quickly apply security to a test app
Migrating security from an old app to newer version
Pushing/Staging large scale security changes
Sync security example
Example:
●
●
Clone FMTEST2 Security to FMTEST3
FMTEST3 is not provisioned
Initial State:
Provisioning Report for FMTEST2
Provisioning Report for FMTEST3
5
#
Tip
Example: Syncing security
Shared Services Application Groups
Source: FMTEST2
FMTEST3 security established in a few mouse clicks
5
#
Tip
Tip 6: Password Changes
Keep the System Secure
Database accounts in Config Tool
6
#
Tip
Steps to Change Typical DB Accounts
1.
Change password for product in the Config Tool
2. Redeploy the application server
2. Change directly in WebLogic
Home>Summary of Security Realms>Summary of JDBC Data Sources> eas_datasource
Tip 7: Working with Oracle Support
How Detailed SRs Save You Time
General Oracle Support tips
7
#
Tip
Adding Details in the SR Saves You Time
●
●
●
●
●
●
Incomplete info will result in lost hours/days
Provide overview, context, step by step process, screenshots
Reproduce the problem - prove within a shadow of a doubt
Simply the problem - can you reproduce with a sample app, basic report,
etc.
Flip side - how can Oracle reproduce the problem outside of your env?
Ask for a WebEx - demonstrate the problem first hand
Tip 8: Oracle SR Tools
Tools Required for Working with SRs
Creating a SR - deployment report
8
#
Tip
Deployment report - Layout of environment
Location: user_projects\epmsystem1\bin
Invocation: epmsys_registry report deployment
Output: diagnostics\reports\deployment_report.html
Database Connections
Product /
Server
Layout
Creating a SR - versions and patch levels
8
#
Tip
Opatch lsinventory - Exact versions of all products on server
Location: D:\oracle\Middleware\EPMSystem11R1\OPatch
Invocation: opatch lsinventory -oh D:\Oracle\Middleware\EPMSystem11R1
-jre D:\Oracle\Middleware\jdk160_29
Output: EPMSystem11R1\cfgtoollogs\opatch\lsinv
Patch 13905088 : applied on Mon Oct 29 16:14:16 PDT 2012
Patch 13936087 : applied on Mon Oct 29 15:51:44 PDT 2012
Patch 14031912 : applied on Mon Oct 29 15:07:44 PDT 2012
support.oracle.com Patch Search
Creating a SR - more tools
8
#
Tip
Zip logs - Collect all relevant logs on system
Invocation: ziplogs
Output: diagnostics\ziplogs\EPM_logs_hss-node_11.1.2.2.0.zip
Registry report - HTML view of internal EPM HSS Registry
Invocation:epmsys_registry (no arguments)
Output: diagnostics\reports\registry.html
Validate (shown earlier)
Invocation: validate
Output: diagnostics\reports\instance_report*.html
Script Locations: user_projects\epmsystem1\bin
Tip 9: Auto Restart Failed Services
Hands Free Problem Solving
Restart failed Windows services automatically
9
#
Tip
● Error causes service
stop/crash
● Not frequent, but may save
some headaches
● Avoid repeated restarts
● Custom settings reset every
time you rerun config tool
Command line
sc \\localhost failure HyS9FRReports
actions= restart/60000/restart/60000/""/60000 reset= 0
Tip 10: Exploring Audit
Identifying System Usage
Exploring Audit
10
#
Tip
Goal: View the number of logins v. time per product from the audit logs
Enable Audit in Shared Services: Administration - Configure Auditing
Audit tables: basic query
Basic query for Essbase logins v. time
To Do: Add unified time series for graphing
10
#
Tip
Audit tables: building a useful query (1)
Create uniform time sequence for graphing
10
#
Tip
Audit tables: building a useful query (2)
Join time series with original query
10
#
Tip
Audit tables: building a useful query (3)
Results show uniform time with number of logins
10
#
Tip
10
#
Tip
Visualizing the results
Logins vs. Time by Product
Essbase
Workspace
10
#
Tip
Recap
Tips and Tricks:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Manual Troubleshooting
Log Analysis Utility
Validate Utility
LCM Automation
Copy Provisioning
Password Changes
Working with Oracle Support
Oracle SR Tools
Auto restart failed services
Exploring Audit
Objectives:
●
●
●
Uptime
Less toil / Effective EPM
Administration
Security
Thank You!
Q&A
Nicholas King
[email protected]

Download Report