...

Part 3 - Performance: How to Fine-tune Your ODM Solution Sponsored by

by user

on
Category: Documents
6

views

Report

Comments

Transcript

Part 3 - Performance: How to Fine-tune Your ODM Solution Sponsored by
Part 3 - Performance:
How to Fine-tune Your ODM Solution
An InformationWeek Webcast
Sponsored by
Webcast Logistics
Today’s Presenters
David Granshaw
WODM Performance Architect (Events)
Pierre-André Paumelle
WODM Performance Architect (Rules)
Demo Series: Best Practices for Implementing
Operational Decision Management Solutions
Part 3: Performance: How to Fine-tune Your ODM
Solution
April 3rd, 2012
WebSphere Operational Decision Management
Relations between components
WebSphere Decision Center
Ruledocs
Decision Center Console
Decision Center
for Business Space
Rule Solutions
for Office
Session 1 focus
Session 2 focus
Session 3 focus
Decision Center
Repository
Deploy
Synchronize
Deploy
Synchronize
Deploy
Rule Execution
Server
5
Deploy
Rule Designer
Event Designer
WebSphere Decision Server
Event Execution
Runtime
Agenda
► WebSphere Operational Decision Management
► 1. Tuning Decision Server Events
► 2. Tuning Decision Server Rules
► 3. Tuning Decision Center
►4. Performance Improvements in V7.5
Note the performance figures given in this presentation are just examples based on a
typical workload. The performance impact of specific design and runtime changes will vary
depending upon the details of the workload.
Decision Server Events – Runtime Overview
WODM Server
DB Server
Events
Actions
JMS (durable)
Event Destination
Technology
Connectors
Technology
Connectors
JMS (durable)
Action Topic
Events
Runtime
Events
Repository
Database
All events and actions ultimately end up on the JMS event and action
destinations (except events delivered via Extreme Scale).
Technology Connectors can be used to send actions and receive events from a
range of sources including JMS, File, HTTP, JDBC etc
The Events Repository can be used to store information about the occurrence of
events, history and any data that may be shared across events (CSBOs).
© IBM 2012
Event Designer : Durability of Events and Actions
Non Durable processing can be 2.5 times faster than durable
processing so if actions and events do not need to survive a
server restart use non durable processing
For further information on durability see the WODM Infocenter : “Configuring
durable and nondurable events and actions”
© IBM 2012
Event Designer : Simple and Complex Event Rules
• Simple event rules
– By definition, Simple event rules can be evaluated without reference to
previous events or actions and are therefore significantly faster
• e.g. “if the deposit amount is > $500 then …”
• Complex Event Rules
– By definition, Complex event rules have a dependency on previous events
or actions with the same context ID (e.g. customer reference number)
• e.g. “If this customer has withdrawn money 3 times in the last week then …”
• For optimal performance do not add a Context Id to a Simple event rule
unless required.
Simple rules should only have a context ID defined if:
• Information about the occurrence of the events and action processed by this
event rule are actually required by other, Complex event rules.
• Context Scoped Business Objects are being used
© IBM 2012
Event Designer : Simple Rules versus Complex Rules
Simple event rules are faster than Complex rules by:
Non Durable Processing: 87%
1.87
Relative Throughput (evts/s)
1.8
1.6
Simple Rule
Simple Rule with Ctx ID
Complex Rule
2
1.8
Relative Throughput (evts/s)
Simple Rule
Simple Rule with Ctx ID
Complex Rule
2
Durable Processing 22%
1.6
1.41
1.4
1.4
1.2
1.22
1.2
1
1
0.8
1.11
1
1
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
Non Durable
Durable
Don’t define a context ID on a Simple rule unless required
Consider using Simple rules for filtering
© IBM 2012
Event Designer : Ordering Multiple Expressions/Filters in an Event Rule
• Rules will evaluate expressions intelligently from the top down, so
place Simple expressions first
– For example, the second part of an ‘and’ expression will only be
evaluated if the first part is true.
Complex expression first :
More costly Complex expression is always evaluated
Simple expression first :
More costly Complex expression is only evaluated
when the Simple expression is true
© IBM 2012
Event Designer : Ordering multiple filters in a rule
Placing Simple filters first may be faster by:
Non Durable Processing: 51%
Simple Filter First
1.51
Complex Filter First
Relative Throughput (evts/s)
1.4
Simple Filter First
1.4
1.2
1.2
1
1.6
Relative Throughput (evts/s)
1.6
Durable Processing 11%
1
0.8
Complex Filter First
1.11
1
1
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
Non Durable
Durable
Note: The performance difference will depend how often the first filter evaluates to
true. In the graphs above the first filter evaluated to true 1 in every 100 events.
© IBM 2012
Event Designer : Ordering multiple filters in a rule
•
For optimal performance:
Always place Simple filters ahead of Complex filters.
–
The same principle applies to any filters that require more processing, for
example, simplistic filters should come before filters that involve database
fetches or Context Scoped Business Objects
Also, if an event triggers multiple Complex rules to be evaluated, group
the rules together into one event rule group.
© IBM 2012
Event Designer : Technology Connectors
These examples show an example of the performance impact of using the new JMS
and HTTP connectors shipped in V7.5.
Event Connector
No Connector
HTTP Connector
JMS Connector
1.2
1
No Connector
HTTP Connector
JMS Connector
1.2
1
1
0.8
Relative Throughput (evts/s)
Relative Throughput (evts/s)
1
Action Connector
0.9
0.8
0.7
0.7
0.6
0.8
0.6
0.4
0.4
0.2
0.2
0
0
Event Connector
Action Connector
For optimal performance, send events in the correct format directly to Decision
Server JMS event or durable event destinations. Also receive actions directly from
the action or durable action topic
© IBM 2012
Event Runtime Performance : Top Ten Recommendations
1. Monitor system resources (cpu, disk, network, memory)
•
This will ensure you target your performance tuning effectively
2. Check you are not writing large amounts of data to log, error log, ffdc or
trace files
•
It will significantly impact performance and may indicate an underlying
problem in the project or configuration, which will degrade throughput.
3. Ensure you are not recording history unless you intend to.
•
•
History is needed for user defined charts or reports of events, actions, filters
and data. It is not needed for Complex rule evaluation or Context Scoped
Business Objects.
History records large amounts of data. It can impact performance by 50 to
80%.
© IBM 2012
Event Runtime Performance : Top Ten Recommendations
4. Tune the JVM
•
•
A heap size of 1280MB is a good starting point for 32 bit systems; 4096MB
for 64bit.
Consider using generational garbage collection
5. Ensure you have a sufficient number of rule processing threads
•
•
You should consider increasing the number of rule processing threads if
events are building up on the event destination
If you increase the number of rule processing threads you also need to
consider increasing the connection pools for database connections, JMS
Connections and data connection objects
6. Periodically (either manually or automatically) prune unnecessary
context data from the Decision Server Event Repository to improve the
database performance
© IBM 2012
Event Runtime Performance : Top Ten Recommendations
7. Ensure the repository database is tuned (or auto tuned) for the workload
–
use separate, fast disk subsystems (SAN/RAID) for all logs and tables.
8. When using persistent Context Scoped Business Objects, the following
database tuning may improve performance up to a factor of 3:
–
For db2 consider using the statement concentrator:
•
–
UPDATE DB CFG USING STMT_CONC LITERALS IMMEDIATE;
For Oracle consider using cursor sharing:
•
ALTER SYSTEM SET CURSOR_SHARING=FORCE SCOPE=BOTH;
9. Tune the JMS provider and monitor the queue/topic message depths
10. Use the Performance Monitoring Infrastructure to monitor Decision
Server Events
•
This will show statistics about events, actions, rules and other aspects of the
runtime. See WODM Infocenter for more details
© IBM 2012
Note the next 4 slides were not presented at the webinar but provide
additional detail on some of the top ten tuning recommendations.
© IBM 2012
Runtime : Minimising Disk Access - logging
WODM has a number of log and error files. If these are growing significantly
over time it can have a large impact on performance and may indicate an
underlying problem.
Check that the following files and ensure they are not growing
significantly over time. If they are, read the logs to find the cause and
rectify the problem:
• <was_install_dir>\profiles\ <profileName> \logs\SystemOut.log
• <was_install_dir>\profiles\ <profileName> \logs\SystemErr.log
• <was_install_dir>\profiles\ <profileName> \ffdc\ (all files)
Also confirm that large trace files are not being generated in
• <was_install_dir>\profiles\<profileName>\logs\trace.out
• <DecisionServer_install_dir>\director\logs\connectors.log (standalone
connectors only)
For further information on trace see the WODM Infocenter : “enabling trace” and “Configuring
the technology connectors log file”.
© IBM 2012
Runtime : JVM Tuning
• The tuning of the JVM and the garbage collection policy can have a
significant impact on performance.
• Settings will depend upon the WODM project and the OS but here are
some recommended starting configurations
32 bit
WODM installation
64 bit
WODM installation
Minimum heap size
1280 (MB)
4096 (MB)
Maximum heap
size
1280 (MB)
4096 (MB)
-Xgcpolicy:gencon
–Xmn1024M
-Xgcpolicy:gencon
–Xmn2048M
Generic JVM
arguments
For further information on JVM tuning see the WAS Infocenter : “JVM Tuning”
© IBM 2012
Runtime : Decision Server Events Tuning
• If there is spare cpu on the WODM server and events are building up
on the event destination this may indicate an insufficient number of
rule processing threads. The number of threads can be increased in
the WAS administrative console:
– Resources > Resource Environment > Resource environment entries >
WbeSrv01 > Custom properties
– In a cluster it is recommended to use a prime number for the number of
threads. This will provide the greatest concurrency.
© IBM 2012
Runtime : Decision Server Events Tuning
• If you do increase the number of rule processor threads:
– As each rule processor may use a database connection, ensure there are
a sufficient number of JDBC threads
• Resources > JDBC > Data sources > Event Runtime Datasource > Connection
pools
– Set “Maximum connections” to “rule processor threads + history MDB concurrency +
10”.
– As each rule processor thread may use a JMS connection, ensure there
are a sufficient number of JMS connections
• Resources > JMS > Topic connection factories > WbeTopicConnectionFactory
> Connection pools
– Set “Maximum connections” to “rule processor threads + 10”
– When using database enrichment ensure there is a connection available
for each rule processor thread. This is set in event designer data
connections page.
– Set the connection pool limit to at least the number of rule processor threads
© IBM 2012
HA, Scalability and Performance
• When using WebSphere Default Messaging for JMS
– There are two key clustering architectures mentioned in the WODM
Infocenter, the “Silver Topology” and the “Golden Topology”.
– The Golden topology has separate messaging and WODM clusters
– The Silver topology contains a messaging engine and a Decision Server
Runtime in each cluster member
– There is also a choice of messaging engine configurations
– The scalability options will offer the opportunity to have multiple
messaging engines and will provide the greatest throughput.
For optimal performance use the Silver Topology with a “Scalability” or
“High availability with Scalability” messaging engine configuration.
© IBM 2012
HA, Scalability and Performance – Golden versus Silver
Topology
The Silver topology can out perform the Golden topology by 40 – 80% due
to the co-location of the messaging engine and events runtimes
Non Durable Processing
Durable Processing
1.8
Relative Throughput (evts/s)
1.6
1.4
1.8
Golden Topology
1.6
Silver Topology
Golden Topology
1.4
1.4
1.2
1
Silver Topology
Relative Throughput (evts/s)
1.8
1.2
1
0.8
1
1
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
Non Durable
Durable
© IBM 2012
Agenda
► WebSphere Operational Decision Management
► 1. Tuning Decision Server Events
► 2. Tuning Decision Server Rules
► 3. Tuning Decision Center
►4. Performance Improvements in V7.5
© IBM 2012
Decision Service architecture
Rule Engine Execution
IN
Ruleflow
IN
Rules
OUT
BOM to XOM
XOM
•
•
26
© IBM 2012
BOM: Business Object Model
XOM: eXecutable Object Model
Decision Service performance best practices
• The performance cost for a Decision Service may look
something like
Execution Time
XOM
Ruleflow
Functions
B2X
Misc. Rule engine
JVM (GC)
27
© IBM 2012
Decision Service performance best practices
• XOM type choices:
– JAVA XOM better performance
– XML XOM
• Dynamicity
• Useful in case of XML model
• Ruleflow:
– Limit the size and the complexity of the ruleflow, it is
interpreted.
– Always use the same engine algorithm to save memory
© IBM 2012
Decision Service performance best practices
• Choose the correct engine algorithm depending on
your Decision Service.
– RetePlus (The default mode)
• Stateful application
• Rule chaining application
• May be useful in the case of many objects
– Sequential
• Application with many rules and few objects
• Most of the customer cases.
• Really efficient in multi-thread environment.
– Fastpath
• Application with rules implementing a decision structure and
many objects
• May have longer compilation but faster at run time.
• Really efficient in multi-thread environment.
© IBM 2012
Decision Service performance best practices
 Choose your architecture depending on your Decision Service
characteristics:
– Smallest memory consumption
• Engine alone
– Integration with JEE animals
(EJB, MDB, POJO)
• Rule Execution Server
JEE
– No concurrent execution
• Engine alone
• JSE Rule Execution
Server
– Call as a Web Service
• HTDS
– Concurrent executions
• MTDS
• Rule Execution Server
JSE
• Rule Execution Server
JEE
– Call from COBOL application
• zRES
30
© IBM 2012
Decision Server Rules tuning
• The log level in the Rule Execution Server should be set to level
Severe or Warning in the production environment to increase
performance.
– This property (TraceLevel) is accessible in the resource
adaptor of the Rule Execution Server or in the ra.xml.
• Tune the GC and memory size.
– Starting configuration 64bits
– -Xgcpolicy:gencon –Xmn2048M –Xmx4096M –Xms4096M
• Tune the RES pool size. A sizing methodology is available at :
http://www-01.ibm.com/support/docview.wss?uid=swg21400803
© IBM 2012
Decision Server Rules tuning
• Tune the usage of the execution trace and the
Decision Warehouse by filtering.
• A ruleset on an XML XOM should be configured to
run in multiple simultaneous.
– This is configured using the ruleset property
ruleset.xmlDocumentDriverPool.maxSize.
• Use the ruleset caching information (XU dump) from
the Rule Execution Server console to get the Rule
Execution Server status.
© IBM 2012
Decision Server Rules Performance
TPS
Impact of the execution trace
© IBM 2012
Decision Server Rules Performance
TPS
Impact of the XOM type
© IBM 2012
Decision Server Rules Performance
TPS
Remote Web Service call vs. Local call
© IBM 2012
Decision Server Rules Performance
TPS
Fastpath Algorithm vs. Sequential Algorithm
© IBM 2012
Agenda
► WebSphere Operational Decision Management
► 1. Tuning Decision Server Events
► 2. Tuning Decision Server Rules
► 3. Tuning Decision Center
►4. Performance Improvements in V7.5
© IBM 2012
Decision Center tuning
• Limit memory consumption
– Parsing a ruleset consumes memory so you can disable the
“Archive parsing flag” option (Project > Edit project options >
Check the ruleset archive).
– Use Automatic build to avoid a huge ruleset generation cost
at “first ruleset generation”.
– Set in the installation manager the configuration parameter
teamserver.build.archive.storage to file instead of memory.
• Build performance
– Disable “Archive parsing flag”.
– Disable “Rule analysis checks”.
© IBM 2012
Decision Center tuning
• Memory consumption estimation on a repository of
10000 business rules
– Memory foot print of a session is ~10 MB
– Ruleset generation : ~150 MB of short-lived objects created
• If you believe the number of users connected
simultaneously to a single server will not fit into the
memory allocated for a single VM, you should deploy
Decision Center to a cluster of servers and use loadbalancing, so that HTTP sessions are dispatched
according to the server’s available memory.
© IBM 2012
Agenda
► WebSphere Operational Decision Management
► 1. Tuning Decision Server Events
► 2. Tuning Decision Server Rules
► 3. Tuning Decision Center
►4. Performance Improvements in V7.5
© IBM 2012
Decision Server Rule
Benchmark Scenarios (relative to 7.1.1.1 GA)
• Rule Execution Server execution improvement (TPS)
Between 1.01x to 3.97x faster than 7.1.1.1
Average improvement factor = 25%
•
41
The new Web Service stack (HTDS) for rulesets based
on java XOM offers performance improvements
Improvement factor 2x to 4x versus 7.1.1.1
© IBM 2012
Decision Center
(relative to 7.1.1.1 GA)
• Decision Center delivers a new ruleset build chain.
Performance gains from 50% to 150% versus
Rule Team Server V7.1.1.1.
This optimization offers a better scalability of
Decision Center as performance gains increase with
rule project size.
42
© IBM 2012
Customer Based Scenario 1 : Credit Limit Check
• This scenario processes withdrawals from a bank and looks for
customers who repeatedly exceed their credit limits. The majority of
processing is with Simple event rules.
Withdrawal
Events
Enrich Event:
Get current
balance and
credit limit from
DB
Simple rule:
Check if limit
exceeded
Synthetic event
Generated 1 in
every 100 events
Complex rule:
Has balance
been exceeded
before?
Warn
Customer
Action
Performance in V7.5 improved by (relative to V7.0.1.1):
18% for non durable processing
10% for durable processing
© IBM 2012
Customer Based Scenario 2 : Missed Payments
• This scenario looks for customers who have repeatedly missed payments
on multiple accounts : credit card, overdraft and loan. It also uses CSBOs to
keep track of the total of all missed payments. All event rules are Complex.
Credit card,
overdraft, loan
missed
payment
events
Enrich Event:
Get current
customer details
from DB
Complex rule:
Complex rule:
Synthetic event Has this occurred
Have payments
been missed on all Generated 1 in more than once and
every 100 events is the total debt >
accounts
$5000
Performance in V7.5 improved by:
31% for non durable processing
14% for durable processing
© IBM 2012
Warn
Customer
Action
Customer Based Scenario 3 : Rules and Events
• Scenario 3 is a customer based scenario, which looks for customers who
are eligible for a discount as part of a new marketing campaign. The
scenario involves DSE and DSR.
Quote
Requested
Event
Decision Server Events (DSE)
Simple rule:
determine
discount from
Decision Server
Rules
Action
Event
Complex rule:
Has this
customer applied
before
SOAP Technology Connector
Decision Server Rules (DSR)
• Performance in V7.5 improved by:
• 27% for non durable processing
• 16% for durable processing
© IBM 2012
Offer Customer
Discount Action
Where to go for further information
•
WebSphere Operational Decision Management
– www.ibm.com/operational-decision-management
•
Redbook on BRMS Tuning
– Proven Practices for Enhancing Performance: A Q & A for IBM WebSphere ILOG
BRMS 7.1
•
Whitepaper on WebSphere Business Events Tuning
– Getting the best performance from your complex event processing system with
WebSphere Business Events
•
DeveloperWorks tool (System Integration Bus Performance) that
automatically engages and displays PMI statistics relevant to Decision Server
Event runtime performance
– WODM Performance Monitoring Tool
© IBM 2012
Q&A Session
David Granshaw
WODM Performance Architect (Events)
Pierre-André Paumelle
WODM Performance Architect (Rules)
Resources
To view this or other events on-demand please visit:
http://www.informationweek.com/events/past
For more information please visit:
WebSphere Operational Decision Management
www.ibm.com/operational-decision-management
Fly UP