19
Version 1.2 11/4/2008 Page 1 of 19 Oracle ® Hyperion Financial Management, Fusion Edition Release 11.1.1 Financial Management Application Copy Utility This document will cover the most commonly asked questions and issues, as well as covering the basics of the utility’s usage and functionality. All the utility screens will be shown and all options explained for the most current utility version, which as of this writing comes from the Oracle® Hyperion Financial Management, Fusion Edition, 11.1.1 release. Common Questions, Issues and Recommendations ...................................................................................................2 Preparing to Run the Utility .......................................................................................................................................4 Running the Utility .....................................................................................................................................................4 Welcome screen..................................................................................................................................................4 Source UDL selection.........................................................................................................................................5 Source Application selection ..............................................................................................................................5 Target UDL file selection ...................................................................................................................................5 Select Destination application ............................................................................................................................6 Options ...............................................................................................................................................................7 Advanced Options ..............................................................................................................................................8 Data Options .......................................................................................................................................................9 Database Options ..............................................................................................................................................10 Confirmation screen .........................................................................................................................................11 Copying ............................................................................................................................................................11 Task Entry Log Screen .....................................................................................................................................12 Completed Tasks ..............................................................................................................................................12 Failed Tasks ......................................................................................................................................................12 Restart Task screen ...........................................................................................................................................13 Finished ............................................................................................................................................................13 Command Line Interpreter .......................................................................................................................................15 Log Review ..............................................................................................................................................................17

HFM Copy Utility

Embed Size (px)

DESCRIPTION

Hfm copy utility - all about it. How to do and also covers the command line interface.

Citation preview

Page 1: HFM Copy Utility

Version 1.2 11/4/2008 Page 1 of 19

Oracle® Hyperion Financial Management, Fusion Edition

Release 11.1.1 Financial Management Application Copy Utility This document will cover the most commonly asked questions and issues, as well as covering the basics of the utility’s usage and functionality. All the utility screens will be shown and all options explained for the most current utility version, which as of this writing comes from the Oracle® Hyperion Financial Management, Fusion Edition, 11.1.1 release.

Common Questions, Issues and Recommendations ...................................................................................................2 Preparing to Run the Utility .......................................................................................................................................4 Running the Utility .....................................................................................................................................................4

Welcome screen..................................................................................................................................................4 Source UDL selection.........................................................................................................................................5 Source Application selection ..............................................................................................................................5 Target UDL file selection...................................................................................................................................5 Select Destination application ............................................................................................................................6 Options ...............................................................................................................................................................7 Advanced Options ..............................................................................................................................................8 Data Options.......................................................................................................................................................9 Database Options..............................................................................................................................................10 Confirmation screen .........................................................................................................................................11 Copying ............................................................................................................................................................11 Task Entry Log Screen .....................................................................................................................................12 Completed Tasks ..............................................................................................................................................12 Failed Tasks......................................................................................................................................................12 Restart Task screen...........................................................................................................................................13 Finished ............................................................................................................................................................13

Command Line Interpreter .......................................................................................................................................15 Log Review ..............................................................................................................................................................17

Page 2: HFM Copy Utility

Version 1.2 11/4/2008 Page 2 of 19

Common Questions, Issues and Recommendations The copy utility is useful when you need to copy a single application either in its entirety or just a portion between environments, such as between Production and Test, QA or Development. The primary use of these copied applications is to allow the Financial Management administrator to thoroughly test new application changes in a controlled environment before applying changes to Production. The Source environment refers to the application you want to copy and the Target environment refers to the location you are copying the application to. The Source and Target environments may or may not refer to the same physical RDBMS environment. The following bullet points cover the most common questions generated about using this utility:

• The copy utility allows for application renaming so the copied application can have the same or different name in the Target environment. When the Target and Source environment are the same, then you must enter a new name for the copied application.

• The copy utility from a newer Financial Management release is backwards compatible to as far back as version 3.4. It is typically recommended to use the most current commercial release of the utility to take advantage of fixes and new features.

• The copy utility can be run on any server or workstation that has properly configured UDL file(s) with access to source and target databases.

• The utility will work with an encrypted UDL.

• The utility can copy between RDBMS versions and vendors: Oracle, MS SQL and IBM DB2.

• The utility can be used to copy an older versioned application into a newer versioned environment to allow the Schema Upgrade utility to upgrade the newly copied application:

o Example: Copying an application from an older release in Production to a newer release in a Test environment. The application can be refreshed from production into test and schema upgrade utility run again.

• For System 9 and 11 versions, the copied application must be registered with Shared Services

in the Target Environment after the application has been successfully copied. A Security extract must be made in the source environment and loaded in the newly copied application before user provisioning is properly set up. For cases where the Target application is being replaced, meaning the copied application uses the same name of an existing application in the Target environment, it is not required to re-register the application but security should still be loaded to ensure user provisioning is properly updated. If Native users or groups are used for provisioning, then these must be created in the Target environment if they do not already exist before loading security. The CSSImportExportUtility can be used to ensure these are created as they exist in the Source environment . See the Oracle EPM System Security Administration Guide for details on using the CSSImportExportUtility utility.

• A new feature in the 11.1.1 release is the inclusion of a command line interpreter called Hfmcopyappcmd.exe. The command line interpreter allows for scheduling of copy jobs using operating system or third-party scheduling utilities.

Page 3: HFM Copy Utility

Version 1.2 11/4/2008 Page 3 of 19

• Task Flows are not copied along with the Financial Management application; Task Flows are stored in the Shared Services database. Use Lifecycle Management introduced in 11.1.1 to export and import Task Flows between environments. Task Flows must be manually created in earlier releases.

• Do not use this utility to copy an EPMA-enabled application.

• Do not copy over an existing application when the Target application is currently running. o Example: Copying an application from Production to Test and replacing an application in

Test while the application is open. This will cause the application tables to be out of sync with the metadata and data loaded in memory in the HsvDatasource process, resulting in any number of application-related errors. When this happens, the copied application in the Target environment must be shut down and the copy run again.

• It is not required that you delete an application from the UI in the Target environment before running the utility to copy over an existing application. The utility provides options to drop all tables related to an application being copied in the target environment. However, deleting an application from the UI in the target environment is an effective way to ensure the application is shut down in the target environment while the application is being copied.

• The log file created during the application is now created in the %HYPERION_HOME%\logs\hfm folder.

• Do not copy an application when users are active in the Source application. The utility copies a number of tables at a time, one table for every thread configured in Advanced tab. Active users updating tables in the source application while the copy is running will result in the copied application having tables that are not in a consistent state.

• Do not attempt to copy between environments separated by a WAN. The utility processes multiple tables at a time and tables are copied row by row. This generates a great deal of network traffic and the added latency of a WAN often makes the entire copy process take an excessively long time. Use the RDBMS vendor’s recommended backup and restore utilities to restore into a temporary database or schema and then run the utility to copy an application into the desired target environment.

• No assertions are made about the utility’s performance. Due to the nature of the utility copying many tables, typically with millions of rows per table, high database loads are expected on source and target environments and performance will be directly related to the infrastructures’ ability to handle the load.

• The copy utility is not meant to replace the RDBMS vendor backup and restore utilities to recover from catastrophic failures. The RDBMS backup and restore software will always be faster and more secure and should be used whenever possible.

• The copy utility will cause a great deal of transaction log / redo log activity. Each row copied will generate a transaction record. For MS SQL databases, the DBA can change the Target database recovery model to Simple and create a full database backup after the copy is complete. Another consideration for MS SQL is to shrink the transaction logs after the copy is complete. For Oracle, if the instance is configured for Archiving, the DBA should consider the impact on the redo logs and the archive process. The amount of redo generated by the copy will impact the Mean Time To Recover.

Page 4: HFM Copy Utility

Preparing to Run the Utility The Financial Management Application Copy Utility is a standalone utility and can be run from any machine that has valid UDL file(s) to access the Source and Target RDBMS environments. The database user configured in the UDL files must have the same database privileges as documented in the Oracle EPM System Installation and Configuration Guide. If the source and target environments are the same database, then the amount of network traffic generated will be much lower than when the source and target databases are separate*. The location of the machine running the copy utility should be as close as possible to the source and target RDBMS environments. The added latency introduced when running over a WAN or over multiple network hoops will have a negative impact of the overall time to complete the copy. Also, the number of CPUs on the machine running the utility will have a direct impact on the copy process. The utility will spawn a number worker of threads, one thread copies one table, and the default number of threads is four times the number of CPUs. This value can be increased or decreased from the Advanced Options tab. *New to the 11.1.1 version is the ability to perform direct database copies. If the target application is in the same database for the same schema (user), then the data will not be brought out prior to copying to the new tables. Instead, a database INSERT INTO … SELECT FROM… will be executed. This will increase performance for the application copy. The log file created during the application is now created in the %HYPERION_HOME%\logs\hfm folder by default. If the %HYPERION_HOME% environment variable is not set, then the log will be created in “C:\Documents and Settings\%Username%\Local Settings\Temp”, where %Username% is the name of the user logged into the workstation running the utility.

Running the Utility The Financial Management Application Copy Utility is installed on the Financial Management Application server by default in the %Hyperion_Home%\FinancialManagement\Server folder and is named HfmCopyApplication.exe. The following shows the screens presented when this utility is run:

Welcome screen

Version 1.2 11/4/2008 Page 4 of 19

Page 5: HFM Copy Utility

Source UDL selection

Select the >> button to navigate to the proper UDL file for the Source environment.

Source Application selection

The list of applications is retrieved from the Hsx_Datasources table in the Source environment.

Target UDL file selection

Version 1.2 11/4/2008 Page 5 of 19

Page 6: HFM Copy Utility

Select the >> button to browser to the UDL file configured to access the Target environment. This UDL file can be the same as the Source UDL or it can point to a different UDL.

Select Destination application

For the Target destination, you may select an existing application or manually enter a new application name. The application name must adhere to the naming convention as covered in the Oracle Hyperion Financial Management Administrator’s Guide.

Version 1.2 11/4/2008 Page 6 of 19

Page 7: HFM Copy Utility

Options

Copy Application Data: This box is typically checked to generate an identical copy of the source application. Clear this check box to create a “shell” application. Application metadata tables will be copied and populated, while data tables are only created but are not populated. Copy Audit data: This box can be cleared if Task and Data audit trails are not required to be maintained in the Target environment. Often the Task and Data audit tables are very large tables and should not be copied unless required. Copy Cluster Settings: This check box should normally not be checked. This will copy the Financial Management Cluster-related tables only if the Source environment has a cluster defined. Copying this information between Production and Test can cause crossover connections where Test environments connecting to a cluster actually connect to the Production application servers. Overwrite existing Application (if it exists): In general, always select “Drop All Application tables prior to copy”. It is possible for the Target application to have Scenario and Year data populated that the Source application does not have, and not dropping these tables would introduce variances in the newly copied application compared to the source application. The circumstances in which “Only drop tables that are being copied” would be a valid option would be rare. *Note that the HFM_Errorlog table is not copied by this utility regardless of the options selected.

Version 1.2 11/4/2008 Page 7 of 19

Page 8: HFM Copy Utility

Advanced Options The Advanced Options dialog is broken up in to three pages. It would typically not be required to make any changes to any selections on these tabs unless directed by Oracle Support to address a specific situation.

Use Client-Side Cursors: This option will bring all row data to the client from the RDBMS server. The server will cache no data. All caching will occur on the client, meaning the client will require more memory and the server will require less memory. Reduce the number of threads to 2 x Number of CPUs if this option is selected. Use this option if Server-Side cursors fail. Use Server-Side Cursors: This option will run faster and require less memory on the client. The data set will be cached on the server (requiring more resources on the server). This is the default option. SQL Binding: This option specifies how the SQL statement is executed on the server. The default is to use SQL binding. Change this option to not use SQL binding if excessive errors occur or “Multi-Step OLE DB Errors” occur. It is best to leave this option on and then restart any failed processes with this option disabled (if the error is a “Multi-Step…” error). Thread Usage: This option allows the user to specify the number of processing threads. The minimum is one and the maximum is sixteen. The default is the number of processors times 4. This option basically controls how many current tables are being processed and has an impact on the amount of network traffic and the load placed on source and target RDBMS servers. Log SQL Errors: This option specifies whether to output every SQL error to the log file. Please note, some errors are expected. You may see errors for the attempt to drop tables that do not exist or the defaulting of sequence values. This is acceptable. Do not use the presence of SQL errors in the log to gauge whether the processing succeeded. This option is not checked by default to reduce confusion when reviewing the log.

Version 1.2 11/4/2008 Page 8 of 19

Page 9: HFM Copy Utility

Number of Task retries: This option specifies the number of times a query should be re-executed in the event of a failure. This is to resolve possible deadlock issues when inserting data in the database.

Data Options The Data Options tab allows you to control what data is actually copied. The Copy Application Data check box must be checked on the Options screen before any data will be copied. The screen shot below shows non-default selections where the utility would only copy data in years 2006 and 2007 for the Actual Scenario:

Year Options

Limit Data to one or more Years: Select this option when you need to copy a subset of historical data. Be careful to consider the impact Rules may have when previous Year data is not copied.

Scenario Options

Limit data to one or more Scenarios: Select this option to copy only a subset of the overall application. Be careful to consider the impact Rules may have when Scenarios are not copied

Data Options

Do not export Numeric Data: When this option is selected, the numeric data is not copied, but cell text and line item detail information is copied. Do not export User data (grids, forms, reports, etc): This is information maintained in the USERPARAMS.

Version 1.2 11/4/2008 Page 9 of 19

Page 10: HFM Copy Utility

Invalid Records Options Note the warning that these options will impact performance and require more database sort area. The utility does a join on the DCN and DCE tables with the metadata ITEM tables.

**Filter invalid Calc Status Records: Invalid Calc status records are the result of Meta Data changes. This option selects only the rows for which the entity and value exist in the database. **Filter invalid Data Records: Invalid Data status records are the result of Meta Data changes. This option selects only the rows where the entity, account, ICP, custom1-4 exist. Filter Zero value data records: Zeros in the application can impact application performance. The utility does not copy rows found to have zeros for all periods.

**Do not use the two Invalid records options until the next release of the copy utility. The 11.1.1.2013 and lower releases will not properly remove all Invalid records.

Database Options The Database Options pane allows for control over where the data is stored in the database. Multiple tablespaces can be configured based on database table types.

Data tables: consist of all Scenario/Year tables (DCN, DCE, CSN, CSE, LID, TXT, RTD, etc). These tables are generally written to and read from quite frequently. They generally have large amounts of relatively small rows. Metadata tables: consist of all of the metadata-based tables (DESC, LAYOUT, HEADER, ITEM). These tables are read from and written to fairly infrequently. They have small numbers of small rows. System tables: are read from rather frequently (HSV, HSX, HFM). Excluding the HFM_ERRORLOG table (which is not read or written to by the Copy App utility), these tables have small row counts.

Version 1.2 11/4/2008 Page 10 of 19

Page 11: HFM Copy Utility

Audit tables: generally have large numbers of rows of medium-sized data. They are written to rather frequently and read from infrequently. BLOB tables: (the current option shows LOB tables but will be changed to read BLOB in the next release) have BLOB columns that are broken up into 2Kb chunks. These tables are read from more frequently than written to.

Confirmation screen

This screen allows you to verify all settings before selecting “Next” to start the copy process.

Copying

The processing status screen displays the current status of each task. To see the log entries for any given task at any time during processing, simply double-click on the task row to display the Task Entry Log screen. Note that when the source and target are the same database and the same schema (user), the utility does not process each table row by row; instead an INSERT INTO…SELECT FROM statement is used. As a result of this, the utility will not display a row count showing the number of rows that have been processed, but the log still will show the number of rows copied.

Version 1.2 11/4/2008 Page 11 of 19

Page 12: HFM Copy Utility

Task Entry Log Screen

Completed Tasks

Failed Tasks

To see the log entries for any task, running or completed, double-click on the entry in the Task Status window. If the Display SQL Errors option has been selected, the error generated by the RDBMS will also be displayed in the log. If the Task Entry Log screen is displayed after all processing has completed, you will have the option of restarting the task.

Version 1.2 11/4/2008 Page 12 of 19

Page 13: HFM Copy Utility

Restart Task screen

Once all processing has completed, you can go to any task in the list, double-click on the entry and activate the Restart Task page. From this page, you can change task options and select the Restart Task button. The task will then be re-executed with the new options (if any). The task entry in the Processing Status screen will be updated with current Task progress.

Finished

If all tables have successfully copied, then select “OK” to see the Finish screen.

Version 1.2 11/4/2008 Page 13 of 19

Page 14: HFM Copy Utility

From the Finished screen, you can review the log file.

Version 1.2 11/4/2008 Page 14 of 19

Page 15: HFM Copy Utility

Version 1.2 11/4/2008 Page 15 of 19

Command Line Interpreter As part of the 11.1.1 installation, there is a new utility available that accepts command line input. This utility allows for scheduling of copy operations using third-party scheduling utilities such as the built in Windows Scheduler. The HFM Copy App Command Line Utility is installed on the Financial Management Application server by default in the %Hyperion_Home%\FinancialManagement\Server folder and is named HfmCopyAppCmd.exe. The command line utility has the same options as the GUI based utility with the exception being that there is no progress window to view competed tasks and no option to retry a failed task. In order to run the command line utility, the machine running the utility must have installed the Microsoft Visual C++ 2005 Redistributable package. This is installed during the 11.1.1 installation, but it may also be downloaded from Microsoft directly: http://www.microsoft.com/downloads/details.aspx?familyid=32bc1bee-a3f9-4c13-9c99-220b62a191ee&displaylang=en for x86 http://www.microsoft.com/downloads/details.aspx?familyid=EB4EBE2D-33C0-4A47-9DD4-B9A6D7BD44DA&displaylang=en for x64. Usage: HfmCopyAppCmd.exe -S="<source UDL>" -s=<source app> -D="<destination UDL>" -d=<destination app> Optional Parameters:

-o="<log file path>" Default is HYPERION_HOME\logs\hfm\HfmCopyApp.log -cd = Y/N Copy data (default is Y) -ca=Y/N Copy Audit data (default is N) -cc=Y/N Copy Cluster information (default is N) -r=Y/N Replace destination app if it exists (default is Y) -da=Y/N If Replace Existing is true, this flag specifies whether to drop all tables prior to copy (default Y) or only drop tables being copied (N) -e=Y/N Log and SQL errors generated during copy (default is N) -ss=Y/N Use server-side cursors (default is Y) -sb=Y/N Use SQL binding (default is Y) -fn=Y/N Filter numeric data (default is N) -fu=Y/N Filter user data (default is N) -ic=Y/N Filter invalid Calc Status records (default is N) -id=Y/N Filter invalid data records (default is N) -z=Y/N Filter zero value data records (default is N) -t=<Num> Number of processing threads (default is num CPUs x 2) -c=<Num> SQL Command Timeout setting (default is zero. no timeout) -Y="<years>" Semi-colon delimited list of one or more years -dts="<data tablespaces>" Semi-colon delimited list of data tablespaces to use (see notes) -its="<index tablespaces>" Semi-colon delimited list of index tablespaces to use (see notes) -f="<parameter file>" Path to parameter file to use instead of command line settings. Parameter file settings override command line settings

Page 16: HFM Copy Utility

Version 1.2 11/4/2008 Page 16 of 19

Notes: Place quotes around all parameters that contain spaces. Tablespaces (index and data) must be listed in the following order. If you do not want to provide a tablespace for a given type, simply omit it but leave its semi-colon delimiter. -dts="DEFAULT;DATA;METADATA;SYSTEM;AUDIT;LOB;" DEFAULT = The tablespace where all unspecified tables or indexes will reside DATA = The tablespace where Data table (DCN, DCE, CSE, CSE, etc. ) data or indexes will reside METADATA = The tablespace where all Metadata tables or indexes will reside SYSTEM = The tablespace where all system tables or indexes will reside AUDIT = The tablespace where all audit table data or indexes will reside LOB = The tablespace where the BINARYFILES and USERPARAMS tables or indexes will reside Parameter file usage: All parameters must be listed on a separate line. Parameters can be in any order. Use command HfmCopyAppCmd.exe -cf="<filename>" to create a blank parameter file. Example command string run from directory where HfmCopyAppCmd.exe resides (following is type on 1 line):

HfmCopyAppCmd.exe -S="C:\Hyperion\FinancialManagement\Server Working Folder\hfm.udl" -s=comma -D="C:\Hyperion\FinancialManagement\Server Working Folder\hfm.udl" -d=comma2 -r=Y -da=Y

The output from the above command should look like the following:

Parsing command line parameters... Command line parameter parsing completed. Starting application copy process...Log File: C:\Hyperion\logs\hfm\HFMCopyAppCmdLog.log Application copy process completed. Review the process log for results. (C:\Hyperion\logs\hfm\HFMCopyAppCmdLog.log) Processing completed.

Page 17: HFM Copy Utility

Log Review Starting with the 11.1.1 release, the log file created by HfmCopyApplication.exe during the application is now created in the %HYPERION_HOME%\logs\hfm folder. The log file name of the last copy will be HFMCopyApp.log. When the utility has been run multiple times, the existing HFMCopyApp.log file is renamed using the format HFMCopyApp-MMDDYYYYHHMMSS.log where MMDDYYYYHHMMSS is based on the date and time the original file was created. The log created by HfmCopyAppCmd.exe is named HFMCopyAppCmdLog and follows the same renaming rules. The following shows what the top section of a log looks like:

Version 1.2 11/4/2008 Page 17 of 19

Page 18: HFM Copy Utility

The following shows the next section in the log, which is the initialization:

One step in the initialization is the creation of two cache tables in the source environment named HFMCAU_COLUMN_INFO and HFMCAU_INDEX_INFO. These tables contain the table and index Data Definition Language (DDL) to use when creating the new tables in the target environment. The cache tables are dropped when the copy utility successfully completes. An error in the initializing section related to these two cache tables usually either means the RDBMS user in the source UDL file does not have the rights to create tables, or that these two cache tables already exist but were created from an older version of the utility. The DBA can check and drop these two tables if they are found to exist when the copy utility is not running. The information contained in these two tables allows the utility to create the target tables faster, as the DDL can be retrieved faster from the tables rather than generating the DDL one table at a time while the utility is running. If the cache tables are not able to be created for any reason, the target application can still be successfully copied. Next is an example of an error caused when a newer version of the copy utility tries to update pre-existing cache tables left over from a failed copy attempt started from an older utility version. Manually dropping these two tables prevents the error from happening the next time the utility is run:

Version 1.2 11/4/2008 Page 18 of 19

Page 19: HFM Copy Utility

The next log snippet shows the entries for one table that has been successfully copied:

The next log snippet shows an error occurring while copying a table when the database ran out of space:

Copyright © 2008, Oracle and / or its affiliates. All rights reserved. http://www.oracle.com

Version 1.2 11/4/2008 Page 19 of 19