OBUG JDE14 November 2014

CNC Best Practices
OBUG JDE14
November 2014
Niall Baker
► Technology Director @ Forza
► Responsible for CNC Managed Services/Consulting
► Joined JDE in 1999
► 15 years CNC experience
Marcelo Tamassia
► Founding partner @ EmeraldCube Solutions
► 18 years of tech industry experience
► 13 years of EnterpriseOne consulting in South and North America
► Planned, designed, executed, and managed over 90 E1
implementations and upgrades worldwide
Agenda
 Intended Audience
 Best Practices
– Performance
– Database Monitoring
– JDE Monitoring
– Log Monitoring
– JDE CNC
 Packman
 Prism
Intended Audience
 JDE System Administrators
 Folks in Technical Roles
 Geeks, Nerds, Techies, Trekkies, etc…
BEST PRACTICES?
PERFORMANCE BEST PRACTICES
#1 Performance – Track Enterprise Queries
 Configure JDE.INI
– Set QueryExecutionTimeTreshold > 0 (start at 30 seconds)
 Search logs for “dbperfrq.c”
 JDE Log
6373/1 MAIN_THREAD
Thu Jan 30 00:00:33.704811
dbperfrq.c770
doQueryDiagnostics: The following SQL query took 10 seconds which is equal to or greater than
QueryExecutionTimeThreshold (10 seconds) for E1User(MARK) with DBProxyUser(JDE).
6373/1 MAIN_THREAD
Thu Jan 30 00:00:33.705162
dbperfrq.c782
SELECT * FROM PRODTA.F41021 WHERE ( LIPQOH <> 0.000000 OR LIPCOM <> 0.000000 OR LIHCOM <> 0.000000 OR
LIPREQ <> 0.000000 OR LIQOWO <> 0.000000 OR LIQTTR <> 0.000000 OR LIQTO1 <> 0.000000 OR LIFCOM <>
0.000000 OR LIMCU <> '
1001' OR LIITM > 1.000000 ) ORDER BY LIITM ASC,LIMCU ASC
#1 Performance – Track Enterprise Queries
 Research Issue
– Determine type of process (JDE.LOG)
– Talk to the user (shown on log message)
 Be mindful of row security impact on queries
 Involve your DBA (if you have one) to determine why the query is taking
to long
– Bad Query/Inquire
– Missing Index(es)
– Slow I/O
– Locking/Blocking
#2 Performance – Tune JDE Kernels
 Make sure you have enough kernels of at least these:
– Call Object: 6 to 12 users / multithread kernel
– Security: 40 to 60 users / kernel
– Metadata: 150 users / kernel
– Network: 1 process + 200 users / kernel
– UBE: 60 users / kernel
 Make sure you don’t have too many
– Memory & Performance considerations
 E1: KER: EnterpriseOne Kernels Overview (Doc ID 961823.1)
#3 Performance – Compress Data
 Use database compression
– SQL Enterprise 2008+
– Oracle Enterprise 11g + Feature
 Results
– Data reduction of 60% to 90%
– Reduced memory requirements
– Reduce backup/restore time
– Fewer I/O
– 40% to 50% performance improvement
– 5% to 10% higher CPU consumption
#3 Performance – Compress Data
 Sample compression rates
Table
Name
Compression
F4111
Item Ledger File
76%
F0911
Account Ledger
73%
F0411
Accounts Payable Ledger
69%
F43121
Purchase Order Receiver File
69%
F4311
Purchase Order Detail File
68%
F9000
Task Master
62%
F0101
Address Book Master
61%
F4229
Sales Summary History File
56%
F0005
User Defined Code Types
54%
#4 Performance – Consider Solid State
 Most JDE E1 environments are I/O bound, not CPU bound
– Faster I/O most likely will result in faster overall E1 performance
– Faster CPU might/ might not result in faster overall E1 performance
 Most JDE E1 environments are 2/3 reads and 1/3 writes
 Performance Comparison (IOMeter Database – Average IO/s)
– HDD 15K RPM: 445 IOPS
– SSD: 2300 IOPS
 Cost / GB
– HDD: 0.05 USD
– SSD: 0.80 USD
#5 Performance – Adjust JVM Heap
 JVM Process Memory
 OS memory limits per single process
– 32-bit, roughly 2GB
– 64-bit, theoretically 2,000 PB
#5 Performance – Adjust JVM Heap
 Garbage Collection
– Lock (lock involved objects so they don’t mutate)
– Mark (iterate through all objects)
– Sweep (remove marked objects / reclaim memory)
 Why do we care?
– Garbage Collection relationship with performance
 Baseline for 50 users
– 32-bit: 1.5 Gb (max = min)
– 64-bit: 2 Gb (max = min)
#5 Performance – Adjust JVM Heap
 GCViewer
M
E
M
O
R
Y
Slow Performance Region
Better Performance Region
Point of Crash
70% of Max Heap Threshold
Performance Horizon
TIME
https://github.com/chewiebug/GCViewer
#6 Performance – Track ER Loops
 Event Rules are interpreted by JAS in real time
 ER loops can cause high CPU and memory consumption
 How to find out?
– Set Long Loop Warning Threshold (jas.ini) to 1000
– If a process performs more than 1000 iterations a message is logged
#6 Performance – Track ER Loops
 Message on JAS log
 Typical causes
– Code issue/bug
– End-user mistake
DATABASE MONITORING
#7 Database Monitoring - I/O Latency
 Measure Latency
– SQL DMVs: Join sys.dm_io_virtual_file_stats and sys.master_files
• Read Latency: CEILING((io_stall_read_ms / ( 1.0 +
num_of_reads)))
• Write Latency: CEILING((io_stall_write_ms / ( 1.0 +
num_of_writes)))
– Oracle: AWR / Foreground Events
 “Acceptable” Latency
Type
Read
Write
Sequential
< 25ms
< 10ms
Random
< 30ms
< 20ms
Total
< 30ms
< 20ms
#8 Database Monitoring - Blocking
 Blocking vs Locking
 What causes blocking
– Concurrency? Poor I/O? Code bug? DB Misconfiguration?
 Identify
– SQL DMVs
• dm_exec_requests + dm_os_waiting_tasks + dm_exec_sql_text
– Oracle
• v$lock + v$session
#9 Database Monitoring - Long running SQL
 Monitor
– SQL: dm_exec_query_stats + dm_exec_sql_text
– Oracle: v$session_longops + v$session
 Research
– Find source process
 Potential fixes
– Application Query Security
– Data Selection Security
– Create Index(s)
JDE MONITORING
#10 JDE Monitoring – Zombie Kernels
 Use Server Manager for alerting
 Research
– Type of kernel
– Call stack
– ESUs?
 Clear zombie after all needed data collected
#11 JDE Monitoring – Call Object Stats
 What are your most executed BSFNs?
 What are the most time consuming BSFNs?
 What BSFNs timeout the most?
 How
– Manually export data from Server Manager JAS Instance
– Prism (TBD)
#12 JDE Monitoring – Batches
 Anomalies
– Errors
– Queuing
 Critical Jobs completion
 Volume
– Most frequent vs. Longest running
 How
– Custom queries/scripts
– Prism
#13 JDE Monitoring – Scheduler
 Make sure JDE scheduler is up and running
 Are batches being submitted?
 How
– Custom queries/scripts
– Prism
#14 JDE Monitoring – Response Time
 Monitor end-user performance
 How long is it taking to logon?
 How long does it take to open P4210?
 How
– Oracle Real User Experience Insight
– Prism Virtual User
LOG MONITORING
#15 - Log Monitoring
 Monitor all tiers
– Operating System logs
– Database logs
– Enterprise Server logs
– HTTP logs
– Weblogic / Websphere logs
– JAS logs
 Why?
– Spot issues proactively
– Reduce troubleshooting time
– Identify patterns and frequency of errors
#15 - Log Monitoring
 Know what to ignore and what is an issue
 Key is to be proactive
 Tools
– Loggly
– Prism
– Splunk
– PaperTrail
– Manual
CNC BEST PRACTICES
#16 – JDE CNC – Compare Specs vs DB
 Execute R9698711 once a quarter against Production
 Generate missing tables via OMW (R98403G if many)
 Generate all missing indexes via OMW (R9698713 if many)
 Fix any column mismatches
#17 – JDE CNC – Users & Roles
 All users should have:
– Valid Address Book number
– At least one role
– Valid system users
 All roles should have:
– At least one environment assigned to it
– At least one user assigned to it
#18 – JDE CNC – Technical Purges
 Database
– Job Master + Dependencies
– Subsystem
– Workflow
– Workcenter
– Scheduler History
– Package History
– Security History
– Orphan Security Entries
– Orphan User Overrides
 Files
– Log files
– PrintQueue
– Temporary files
#19 – JDE CNC – OMW Backup Rule
 Create a Production Save Location to back up production objects before
they get replaced
 Create new status code and activity rule prior PD
 Make sure users/developers cannot bypass the PD save on promotion
 Use Row exit “Backup Rule”
PACKAGE AUTOMATION
#20 – JDE CNC – Packages
 Retention
– No more than 2 full / pathcode
– No need to keep updates for older full packages
– Full package for each pathcode should be built at least every 6
months or when excessive number of updates package
 Be consistent with your name convention
 If you built more than one package / week you should automate it
http://packman.forzaconsulting.eu/
Automate your JDE Object Life Cycle Management
Typical Change Mangement Process
DV
Developer
finishes
project
Package build
request
Assemble
projects
Create
Package
Assembly
Build Package
Deploy
Package
Development
Test
PY
Package build
request
Promote
project
Assemble
projects
Create
Package
Assembly
Build Package
Deploy
Package
Prototype
Test
PD
Approve
Production
Release
Promote
project
Assemble
projects
Create
Package
Assembly
Build Package
Deploy
Package
In Prduction
Automated Change Mangement Process
DV
Developer
finishes
project
Package build
request
Assemble
projects
Create
Package
Assembly
Build Package
Deploy
Package
Development
Test
PY
Package build
request
Promote
project
Assemble
projects
Create
Package
Assembly
Build Package
Deploy
Package
Prototype
Test
PD
Approve
Production
Release
Promote
project
Assemble
projects
Create
Package
Assembly
Build Package
Deploy
Package
In Prduction
Automate your JDE Object Life
Cycle Management
 Decrease time spent on change management
 Minimize manual administration tasks
 Minimize errors
 Automate periodic update packages
 Receive email notifications
 Completely integrated with JD Edwards
PRISM
Prism
 Modules
– Job Monitor
– Log Monitor
– Virtual User
– Database Monitor
– Data Integrity Monitor
– Record Reservation Monitor
– Technical Purges
– Monthly Report
– Auto Updater
Want to know
more?
Come and Visit
Us at the Forza
Stand
Want to know
more?
Come and Visit
Us at the Forza
Stand
Questions?

Download Report