...

TurboIntegrator Process and Cube Logging Troubleshooting

by user

on
Category: Documents
11

views

Report

Comments

Transcript

TurboIntegrator Process and Cube Logging Troubleshooting
Troubleshooting
TurboIntegrator Process and
Cube Logging
Product(s): TM1
Area of Interest: Analytics
TurboIntegrator Process and Cube Logging
2
Copyright
Copyright © 2008 Cognos ULC (formerly Cognos Incorporated). Cognos ULC
is an IBM Company. While every attempt has been made to ensure that the
information in this document is accurate and complete, some typographical
errors or technical inaccuracies may exist. Cognos does not accept
responsibility for any kind of loss resulting from the use of information
contained in this document. This document shows the publication date. The
information contained in this document is subject to change without notice.
Any improvements or changes to the information contained in this document
will be documented in subsequent editions. This document contains
proprietary information of Cognos. All rights are reserved. No part of this
document may be copied, photocopied, reproduced, stored in a retrieval
system, transmitted in any form or by any means, or translated into another
language without the prior written consent of Cognos. Cognos and the
Cognos logo are trademarks of Cognos ULC (formerly Cognos Incorporated)
in the United States and/or other countries. IBM and the IBM logo are
trademarks of International Business Machines Corporation in the United
States, or other countries, or both. All other names are trademarks or
registered trademarks of their respective companies. Information about
Cognos products can be found at www.cognos.com
This document is maintained by the Best Practices, Product and Technology
team. You can send comments, suggestions, and additions to
[email protected] .
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
3
Contents
1
INTRODUCTION ............................................................................................ 4
1.1
1.2
1.3
PURPOSE .............................................................................................................. 4
APPLICABILITY ....................................................................................................... 4
EXCLUSIONS AND EXCEPTIONS .................................................................................... 4
2
TURBOINTEGRATOR PROCESS LOGGING..................................................... 4
2.1
TURBOINTEGRATOR PROCESS LOGGING EXAMPLE............................................................. 4
3
BUILDING TURBOINTEGRATOR PROCESS LOGGING ................................... 5
4
STEP 1 - CREATE THE ZPROCESSLOGS AND ZPROCESSLOGMEASURES
DIMENSIONS............................................................................................................ 5
4.1
ZPROCESSLOGS ...................................................................................................... 5
5
STEP 2 - CREATE THE PROCESSLOGS CUBE.................................................. 6
6
STEP 3 - CREATE THE STARTLOGGING PROCESS ......................................... 7
7
STEP 4 - CREATE A TEST PROCESS ............................................................... 7
7.1
7.2
7.3
7.4
STEP 4A - EDIT THE PROLOG PROCEDURE ...................................................................... 8
STEP 4B - EDIT THE DATA PROCEDURE ......................................................................... 8
STEP 4C - EDIT THE EPILOG PROCEDURE ....................................................................... 9
STEP 5 - CREATING THE PROCESSLOGS RULE ................................................................. 9
8
EXECUTE THE LOGTEST PROCESS............................................................... 10
9
CUBE LOGGING IN TURBOINTEGRATOR .................................................... 10
10
DEFAULT TURBOINTEGRATOR CUBE LOGGING STATEMENTS ................... 10
10.1
10.2
PROLOG ............................................................................................................. 10
EPILOG .............................................................................................................. 10
11
UPDATING YOUR TURBOINTEGRATOR PROCESS....................................... 11
11.1
11.2
SIMPLE .............................................................................................................. 11
ADVANCED .......................................................................................................... 11
12
NOTES FOR USING LOCKOFF AND BATCH UPDATES .................................. 12
12.1
12.2
PROLOG ............................................................................................................. 12
EPILOG .............................................................................................................. 13
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
4
1 Introduction
1.1
Purpose
This article illustrates how to build a cube to store result data from
TurboIntegrator process runs, and how to configure TurboIntegrator
processes to populate the cube.
1.2
Applicability
IBM Cognos TM1 9.1 and prior
1.3
Exclusions and Exceptions
No exclusions identified
2 TurboIntegrator Process Logging
The example given is intended to provide a framework that can be reused and
altered to suit different requirements.
2.1
TurboIntegrator Process Logging Example
The image below shows how a logging cube (named ProcessLogs) stores the details
of all TurboIntegrator process runs. If a TurboIntegrator process run is successful
than all the details are recorded. If a process run is unsuccessful then there is no
finish time, elapsed time, or number of records processed, and the Status is reported
as ‘Not Complete’.
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
5
The following image shows that the most recent run of each process is stored at the
C: level for easy identification.
The example is purposely generic. Once the ProcessLogs cube is built, any existing or
new TurboIntegrator process can be configured to write process execution data to
the ProcessLogs cube.
3
Building TurboIntegrator Process Logging
The following steps describe how to set up process logging on your TM1 server.
4
Step 1 - Create the zProcessLogs and zProcessLogMeasures
Dimensions
The zProcessLogs and zProcessLogMeasures dimensions will be used to create the
ProcessLogs cube.
4.1
zProcessLogs
The zProcessLogs dimension contains the names of all TurboIntegrator processes
that are configured to write process execution data to the ProcessLogs cube. This
dimension initially contains a single consolidated element, AllLogs, but the dimension
is updated with a new element every time the StartLogging process is executed. The
StartLogging process is described later in this article.
Create the zProcessLogs dimension as shown in the following image:
4.1.1.1 zProcessLogMeasures
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
The zProcessLogMeasures dimension contains the measures you want to track for
each executed TurboIntegrator process.
Create the zProcessLogMeasures dimension as shown in the following image:
Note that all elements with the exception of Records are string elements.
5
Step 2 - Create the ProcessLogs Cube
Create a simple two-dimensional cube named ProcessLogs, using the zProcessLogs
and zProcessLogMeasures dimensions.
zProcessLogMeasures should be the last dimension in the cube definition, as this
dimension contains string elements.
IBM Cognos Proprietary Information
6
TurboIntegrator Process and Cube Logging
6
7
Step 3 - Create the StartLogging Process
Create a TurboIntegrator process called StartLogging with datasource type None.
Add two parameters called processName and elementName, both strings.
You will send the values for these parameters from your main process to the
StartLogging process.
Place the following code in the Prolog tab of the StartLogging process. The code
adds the given processName with the elementName as a child element in the
zProcessLogs dimension.
#### Process Logging
DimensionElementInsert ('zProcessLogs', processName, '', 'N');
DimensionElementComponentAdd ('zProcessLogs', 'AllLogs',
processName, 1);
DimensionElementInsert ('zProcessLogs', elementName, '', 'N');
DimensionElementComponentAdd ('zProcessLogs', processName,
elementName, 1);
#### Process Logging
7
Step 4 - Create a Test Process
Create a test TurboIntegrator process named LogTest that copies data from one
element to another using a cube view datasource. If it is a very fast process (i.e. < 1
second) then add a loop to the Data procedure as follows:
iCount=0;
WHILE (iCount <1000);
iCount = iCount + 1;
END;
The LogTest process is used for illustration purposes in this article. The remainder of
the steps described in this article modify the LogTest process to write data to the
zProcessLogs cube. You can apply the general principles used to modify the LogTest
process to your own TurboIntegrator processes.
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
7.1
8
Step 4a - Edit the Prolog Procedure
The process logging code for the Prolog procedure goes before the
#****GENERATED STATEMENTS START**** statement so that you get an accurate
start time. Wrap any code in # characters as shown below so that the logging code
is easy to identify.
The process logging code for the Prolog sets up the variables used (such as start
time, element name, process name and status), then executes StartLogging with the
correct parameters.
The status is set to ‘Not Complete’ in the Prolog so that if the process exits before
completion the status will be accurately reported in the ProcessLogs cube. The status
is changed to 'Complete' in the Epilog procedure.
The CELLPUTS statements write the initial logging information to the ProcessLogs
cube.
#### Process Logging
#Variables
processName='LogTest';
timeStart=NOW;
dateStart = TIMST(timeStart,'\Y \M \d - \h:\i:\s');
elementName = processName | ' - ' | dateStart;
countRecords = 0;
status = 'Not Complete';
#Send parameters to StartLogging
return_value = ExecuteProcess('StartLogging','processName',
processName,'elementName',elementName);
#Write status information for Start
CELLPUTS(status,'ProcessLogs',elementName,'Status');
CELLPUTS(dateStart,'ProcessLogs',elementName,'StartTime');
CELLPUTS(processName,'ProcessLogs',elementName,'ProcessName');
CELLPUTS(status,'ProcessLogs',processName,'Status');
CELLPUTS(dateStart,'ProcessLogs',processName,'StartTime');
CELLPUTS(processName,'ProcessLogs',processName,'ProcessName');
#### Process Logging
7.2
Step 4b - Edit the Data Procedure
The only process logging code required in the Data tab is for incrementing
countRecords to count the number of times the Data procedure is executed.
#### Process Logging
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
9
countRecords = countRecords + 1;
#### Process Logging
7.3
Step 4c - Edit the Epilog Procedure
The process logging code for the Epilog procedure should go at the end of the
Epilog, after all other statements.
This code time-stamps the finish, calculates elapsed time, changes the status to
'Complete', and writes final logging information to the ProcessLogs cube
#### Process Logging
#Variables
timeFinish=NOW;
elapsedTime=TIMST(timeFinish - timeStart,'\h:\i:\s');
dateFinish = TIMST(timeFinish,'\Y \M \d - \h:\i:\s');
status = 'Complete';
#Write status information for Finish
CELLPUTS(status,'ProcessLogs',elementName,'Status');
CELLPUTS(dateFinish,'ProcessLogs',elementName,'FinishTime');
CELLPUTS(elapsedTime,'ProcessLogs',elementName,'Elapsed
Time');
CELLPUTN(countRecords,'ProcessLogs',elementName,'Records');
CELLPUTS(status,'ProcessLogs',processName,'Status');
CELLPUTS(dateFinish,'ProcessLogs',processName,'FinishTime');
CELLPUTS(elapsedTime,'ProcessLogs',processName,'Elapsed
Time');
#### Process Logging
7.4
Step 5 - Creating the ProcessLogs Rule
The final step required to enable TurboIntegrator process logging is to create a the
following rule for the ProcessLogs cube:
['All Logs']=C:0;
['Records']=C:DB('ProcessLogs',ELCOMP('zProcessLogs',!zProcessLogs,ELCOMPN('zProcessLogs',!zProcessLogs)),'Records');
Nothing is required in AllLogs so it is set to 0.
The consolidated Records needs to be overridden with a rule so that it does not add
up the Record values of each process run. The second statement sets the Records
value to the last element in the consolidation (the most recent process run) by
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
10
counting the elements (ELCOMPN) and then returning the nth element name
(ELCOMP) within a DB statement.
8
Execute the LogTest Process
After all the required code is in place and the rule for the ProcessLogs cube has been
created, execute the LogTest procedure. After the process finishes running, open the
ProcessLogs cube to view the logged details of the process execution.
9
Cube Logging in TurboIntegrator
Cube logging is vital in TM1 for maintaining a full history of data changes to cubes,
both for audit purposes and for crash recovery. When logging for a given cube is
enabled (‘on’) all data changes that are entered manually are automatically saved to
the log files.
By default, however, TurboIntegrator toggles cube logging to ‘off’ when running a
process that updates a cube, and then sets the cube logging property back to it’s
initial state at the end of the process. (It is possible to enable cube logging during a
TurboIntegrator process by selecting the Enable Cube Logging option on the Maps
tab, but the default behavior in TurboIntegrator is to run with cube logging
disabled.)
If a TurboIntegrator process is aborted in the middle of a data load (due to too many
errors, PROCESSQUIT statement, etc) the logging property for the cube is never
reset to its initial state and remains ‘off’. Consequently, TurboIntegrator will reset the
logging to ‘off’ at the end of all subsequent processes.
10 Default TurboIntegrator Cube Logging Statements
The following statements are automatically generated for all TurboIntegrator
processes that update cube values. In the statements, <cubename> is the name of
the cube being updated.
10.1 Prolog
OldCubeLogChanges = CUBEGETLOGCHANGES('<cubename>');
CubSetLogChanges ('<cubename>', 0);
10.2 Epilog
CubSetLogChanges ('<cubename>', OldCubeLogChanges);
Assuming that cube logging is enabled at the beginning of the TurboIntegrator
process, the following sequence occurs when the TurboIntegrator process executes
successfully:
•
OldCubeLogChanges is set to 1 (on)
•
The cube logging property of <cubename> is set to 0 (off)
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
•
The TurboIntegrator process runs
•
The cube logging property of <cubename> set back to 1 (on)
11
Assuming that cube logging is enabled at the beginning of the TurboIntegrator
process, the following sequence occurs when the TurboIntegrator process aborts
during the Data procedure:
•
OldCubeLogChanges set to 1 (on)
•
The cube logging property of <cubename> set to 0 (off)
•
The TurboIntegrator process aborts
•
The cube logging property of <cubename> remains set to 0 (off)
Any future process execution will always reset cube logging property to 0 (off).
11 Updating Your TurboIntegrator Process
The recommended practice for all TurboIntegrator processes that update cube
values is to manually set the cube logging property to the desired state at the end of
the Epilog procedure.
There are two methods for this: simple and advanced
11.1 Simple
Add the following TurboIntegrator statement to the end of the Epilog procedure
(assuming that the desired state is 1 (on):
CubSetLogChanges (‘<cubename>’, 1);
This statement is automatically generated when you select the Enable Cube Logging
option on the Maps tab.
11.2 Advanced
Set up a cube (zLogging in the example below) to hold the desired logging properties
of all your cubes – these property values can then be altered by TurboIntegrator ,
rules, or manually as desired. Use the }Cubes dimension (so that you always have all
cubes in the dimension) and a simple measures dimension with two N elements as
below:
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
12
1. Add the following rule to the cube:
['default'] = n: IF(['overide'] = 0,1,stet);
This rule sets the desired cube logging property to 1 (on) for all cubes, but
allow this value to be overridden. When any value is entered into the ‘overide’
column the rule is ‘stet’ and the value becomes zero.
Remember – this does not alter the cube logging property for a cube! The
rule merely provides TurboIntegrator with a default logging state within a
process, independent of the current cube logging property value.
2. Add the following TurboIntegrator statements to the end of the Epilog
procedure:
DesiredState = CELLGETN(‘zLogging’,’<cubename>’,’default’);
CubSetLogChanges (‘<cubename>’, DesiredState);
12 Notes for Using LockOff and Batch Updates
Special consideration must be given when batch updates and the LockOff/LockOn
functions are used in a TurboIntegrator process. In this case it is important NOT to
disable logging during the TurboIntegrator process because user input (allowed
while the cube locks are off) are not logged during the TurboIntegrator process.
You should adhere to the following practices when LockOff/LockOn and batch
updates are used in a TurboIntegrator process.
12.1 Prolog
Do not allow logging to be turned OFF in the Prolog. (The only exception is when the
logging property for the cube, as defined in the }CubeProperties control cube, is set
to NO. In this case, cube changes are not expected to be logged.)
Add the following command after the Generated Statements in the Prolog:
IBM Cognos Proprietary Information
TurboIntegrator Process and Cube Logging
13
CubSetLogChanges (‘<cubename>’, 1);
The CubSetLogChanges function must be called before the LockOff and
BatchUpdateStart functions.
12.2 Epilog
Turn cube logging off before the batch update is committed. After the LockOn
statement and before the BatchUpdateFinish function, insert the following command
in the Epilog:
CubSetLogChanges (‘<cubename>’, 0);
IBM Cognos Proprietary Information
Fly UP