Vault Background Processes Whitepaper

OverviewAutodesk has added several new features to the 2009 release of Productstream and Vault, to improve performance and functionality. While some of these features are apparent, there are a few processes that aren’t as visible. Autodesk uses several background processes to perform many common housekeeping chores to keep the system running optimally. This document will describe the way these processes behave, and methods to modify their behavior.

Data Management Server Behavior Most processes ran by the data management server are scheduled events. The data management server uses a job dispatch service to look at a list of scheduled events once every minute. It then compares the scheduled times with the current time and determines if it is in the past or in the future. If the scheduled time is in the past it will start the process. If a process needs to reoccur the data management server will schedule the next execution time based on the time interval stored in the event list. This time is called the period. For example if a process is supposed to run once a day its period would be 24 hours.

The following illustration shows “Process1” is scheduled to run at 1:00 AM. At 1:01 AM the data management server checks the list of scheduled processes and sees there was a process scheduled at 1:00 AM and starts it.

Autodesk® Productstream® 2009 and Autodesk® VaultBackground Processes

Contents

Data Management Server Behavior 1

Configuring Processes 2

Process Groupings 4

Process Definitions 5

Backing Up 12

Summary 12

AUTODESK® MANUFACTURING WHITE PAPER


2

Notice the process starts if the time the data management server polls is greater than the scheduled time recorded in the data base. After the process is complete the data management server schedules the next job based on the process period. When you have a data management environment with one Autodesk data management server and one file store, the single data management server handles all of the processes. If the environment is a multi-site environment with replication, each data management server will run its own processes unless it is process that affects the entire data management environment, in which case each data management server competes for ownership of the process. The first data management server to run the process becomes the owner of the process until it is finished executing. Once the process is finished it is scheduled to run again based on its time period. When it is time to run the process again each data management server will again compete for ownership. The first server to start the process will own the process for that execution. The following figure illustrates an environment process.

Notice Server 1 in the illustration polls .01 seconds before Server 2. Since Server 1 polls slightly before Server 2, Server 1 wins, will start the process, and own it for this execution. When Server 2 polls it sees that the process is currently running by another server and ignores the process. Once the process is complete Server 1 will schedule the process for its next execution.Server 1 and Server 2 continue polling when the new time to run the process passes the first server to start the process will own the process for that execution. If Server 2 starts the process first then server 2 will run the process and schedule the next process time after completing its execution.

If something interrupts a process while it is running, for example the SQL server gets reset or disconnected, the process will keep trying to complete. All processes will expire after trying to complete for 30 days.

Configuring Processes Most of the processes can be enabled, disabled or adjusted in one or more of the following ways. When making adjustments you should consider what effects the adjustment will have on the overall performance for the users.

AutomaticSome events will run automatically. Processes that run during the migration of a vault are a prime example. These events are scheduled as soon as the vault is attached; therefore the data management server sees the event scheduled and starts the process on its next poll.


3

User InterfaceSeveral of the processes have a user interface which is found in either the server console or in the client. It is recommended that you try to change settings through the user interface before editing any configuration files.

Web.configThe Web.config file is an XML configuration file where many of the Vault and Productstream settings are stored. This file is as critical to the data management server as the registry is to Microsoft® Windows. Editing the Web.config incorrectly can cause Vault and Productstream to run incorrectly or not at all. Before editing this file you should make a backup copy. If there is no backup of the Web.config there is no way to recover if a mistake is made while editing the file. If after editing the Web.config, Vault or Productstream does not function correctly you should replace the edited Web.config with your original Web.config.

The Web.config can be found in the default path of “C:\Program Files\Autodesk\Data Management Server 2009\Server\Web\Services”. If you are in a multi-site environment you will have to make the changes to the Web.config on each site.

Some of the changes made in the Web.config will not take effect unless you run a b2bmigrate. This is a console command which reruns all of the migration scripts. Use the following command to run b2b migrate.

Connectivity.admsconsole.exe -Ob2bmigrate -VU<administrator> -VP<password> -DBU<dbusers> -DBP<dbuser password>

If your Vault or Productstream were installed with the default settings you can leave The DBU and DBP options blank as follows.

Connectivity.admsconsole.exe -Ob2bmigrate -VU<administrator> -VP<password>

Editing Web.configYou can use any text editor or XML editor to edit the Web.config file. A simple tool which can be downloaded is Microsoft® XML Notepad 2007. You can use the following link to download this tool.

http://www.microsoft.com/downloads/details.aspx?familyid=72d6aa49-787d-4118-ba5f-4f30fe913628&displaylang=en

Disabling a ProcessWhen disabling a process simply, comment out the “eventclass name” line by adding a start tag of “” . The figure below shows the Filestore.Vacuum process disabled.


4

Changing a Process SettingUnder the “connectivity.settings” tag in the xml you will find all of the settings which can be adjusted. When making changes to the settings, only change the value of the setting and nothing else. The following illustration shows the start time for the Item Lock Cleanup changed from 3 AM to 9 PM.

Any time a setting is changed you must run b2bMigrate. In a multi-site environment the setting change needs to be done along on all sites, but you will only need to run b2bMigrate from one of the sites. It is important to remember the settings from the last b2bMigrate will be stored in the database, which is why the change should be done to all sites. This will prevent a server with old settings from overwriting the new settings at a later date.

Process GroupingsThe process behaviors can be grouped into several different categories. This section will define the categories and describe the properties of each one. It is necessary to point out that one process may fall into more than one group.

EnvironmentProcesses which run and affect the entire data management environment are environment processes. In a multi-site environment each data management server competes for the ownership of these processes every time they are executed.

NightlyNightly processes are processes that are scheduled to run after business hours. The times set for these processes can be changed in the Web.config file.

Per SitePer site are processes run on each data management server in a multi-site environment. Per site processes will run on all enabled vaults in one session. This means each data management server in a multi-site environment could potential run one process involving all of its enabled vaults at same time as another server.

Per VaultPer Vault processes are processes which are ran against each vault in a data management environment.


5

Per Site and Per VaultPer Site and Per Vault processes are similar to Per Site processes; the process is run on all enabled vaults on a particular site. The difference is each vault will have its own process. The number of sessions running will be equal to the number of enabled vaults.

ImmediateImmediate processes run when a vault is migrated, restored, attached, backed up, or detached. These processes run once after the associated activity and are not rescheduled. If the data management server running the process is interrupted, the process will restart after the interruption. If the SQL server is disconnected during the process, the process will be rescheduled.

User EnabledUser enabled processes are processes which can be enabled and disabled through the user interface of the server console or the client software. These processes should have their settings changed from the user interface.

Process DefinitionsThis section describes each process and provides information to aid in understanding of what each process is doing, what they effect and why they are ran. The following table indicates each process and which group or groups they are classified into.

Environment Nightly Per Site Per Vault Immediate User Enabled

Content Indexing Service

Content Indexing Service - Update

ECO Email Notification

ERP Packaging

File Store Cleanup

File Store Replication

File Store Validation

Search Index Optimization

Item Effectivity Update

Item Lock Cleanup

Search Indexing

Property Re-Indexing

Update Statistics

Each process has an impact on the server and network environment. This impact is determined by several factors, for example, speed of the network speed of the CPU, total amount of system memory. In the following sections a simple guide of the amount of impact each process has on the data management environment is described as low, medium, or high. This rating is determined by looking at three main items SQL processing, communication with SQL, and communication with the file store. Placing SQL on a separate server may increase the performance in some environments.


6

Content Indexing Service (Index.RedoContent)

Potential Server Impact Default Time Default Settings Occurrence

High When Enabled Disabled One Time

This process extracts full text contents from existing files in the vault. This allows full content searching. This process only runs if full content searching is enabled. If full content is enabled and you attach a vault, this process will start. If anything interrupts this process it will continue from where it left off.

To enable full content searching, right click on a vault and select Content Indexing Service.

If this process is enabled and a vault is attached the process will be scheduled to run and it will run on the data management server’s next poll.


7

Content Indexing Service (Index.UpdateFullcontent)


Medium Full Content Enabled Disabled 20 minutes

This process runs every 20 minutes once full content searching is enabled. The process searches newly added and updated content to add their full text content to the data management server’s search index. This keeps the full content up to date to provide fast and accurate content searches.

To disable this process, you should follow the steps from above to disable Content Indexing Service. To change the 20 minute wait time, you need to edit the Web.config. The key name is “FullcontentIndexing”. The value should be set in the format “h:mm:ss (hours, minutes, seconds).

ECO Email Notification (ECO.EmailNotification ~ Productstream Only)


Low When Enabled Disabled 10 minutes

This process runs when Automatic Email notifications are turned on. Notifications are saved in a queue and sent by this process. Once the Email is sent the notification is deleted from the queue.

To enable ECO Email Notification, select Administration from the Tools pull down menu.


8

In the Administration dialog box under the Advance Settings tab select the Email button. In the Email dialog box, check the Enable Email Notifications and fill in the appropriate information.

To disable this process, you will need to go back and disable email notifications.

ERP Packaging (ERP.Packaging ~ Productstream Only)


Low Disabled Disabled On Demand

This process communicates with an existing ERP system. Productstream must be configured and the process enabled before it will run. To enable this process click Tools > EPR Integration and fill in the appropriate information in the ERP Administration dialog box.


9

File Store Cleanup (Filestore.Vacuum)


High 2:00 AM Enabled 24 Hours

During file replication files are only copied from one server to another. If a file is deleted from one server it may not be deleted from a second server. The file store cleanup process compares each file store against the SQL database and removes files which have been deleted from the data management environment. If the environment has only a single site or, if the site runs a backup this process may be disabled. Scheduling a backup during this process should be avoided because the backup process runs its own file store cleanup before backing up the file store.

To change the time the file store cleanup process runs you need to edit the Web.config file. The key name is “FilestoreVacuumTime”. The Value should be set in 24 hour format time. For example setting the value to 2:30:45 will schedule the process 45 seconds past 2:30 AM.

File Store Replication (Filestore.Replication ~ Productstream Only)


Medium User Defined Disabled User Scheduled

This process copies files that exist on one site’s file store to another site’s file store where the specific files do not exist. This process is scheduled by the user and is only available in Productstream.

The user can start a replication of a single vault or a single site on demand, or they can schedule each vault to replicate at different times, using the server console menu. To start a replication, right click a single site from the site list and select the replication option.

File Store Validation (Filestore.Validation)


High Migrating / Attaching Enabled (Can’t disable) One time

This process compares database metadata against the file store to determine if all the files in the database exist in the file store.


10

Search Index Optimization (Index.Optimize)


High 1:00 AM Enabled 24 Hours

This process updates Productstream’s and Vault’s indexing engine. This engine enables Productstream and Vault to return search results quickly. When the process is running the indexer is locked. This process runs during off hours to prevent user down time. If this process does not run, the data management server would have to index on demand. This will cause slower performance for most users during peak times.

To change the time of the index optimization process you need to edit the Web.config file. The key name is “OptimizeIndexesTime”. The Value should be set in 24 hour format time. For example setting the value to 2:30:45 will schedule the process 45 seconds past 2:30 AM.

Item Effectivity Update (Effectivity.Update ~ Productstream Only)


Low Enabled Enabled 1 Hour

This process checks the effective date of an item and determines if it should be active or expired at that particular site. If the environment is entirely one time zone, this could be changed to once every 24 hours.

To change the time of the item effectivity process you need to edit the Web.config file. The key name is “EffectivityUpdateTime”. The Value should be set in 24 hour format time. For example setting the value to 2:30:45 will schedule the process 45 seconds past 2:30 AM.

Item Lock Cleanup (Lock.Vacuum)


Low 3:00 AM Enabled 24 Hours

When the data management server and the SQL server perform different actions on the database it will apply locks to prevent collisions of data. If something interrupts these processes the locks may never get removed. The item lock cleanup process runs through the database and removes any lock that left behind.

To change the time of the item lock index process you need to edit the Web.config file. The key name is “LockVacuumTime”. The Value should be set in 24 hour format time. For example setting the value to 2:30:45 will schedule the process 45 seconds past 2:30 AM.

Search Indexing (Index.RedoProperties)


High Migrating / Attaching Enabled (Can’t disable) One time

This process takes properties from the database into the indexer for the first time. This indexing yields faster results when the user searches the database on specific properties.

Property Re-Indexing (Filestore.PropertyReindex)


High On Demand Enabled On Demand

This process extracts properties from a bulk of files and updates the database with the new information. This operation can be started and stopped by the user. If the operation is stopped and restarted later, it will start with the last file processed.


11

This process can be launched by right clicking a vault in the server console and selecting Re-Index Properties. The number of files that will be processed can be determined by using the Calculate button before launching the Re-index.

If the Filestore.PropertyReindex is already running the following dialog box will appear with the appropriate percent complete.

Autodesk, Productstream, and Vault are registered trademarks or trademarks of Autodesk, Inc., in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product offerings and specifications at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document. © 2008 Autodesk, Inc. All rights reserved.

Update Statistics (Database.Statistics)


Medium 4:00 AM Enabled 24 Hours

This process runs Update Statistics on the SQL database. This function improves query performance, which in turn improves Vault and Productstream performance.

To change the time of the database statistics process you need to edit the Web.config file. The key name is “DatabaseStatisticsUpdateTime”. The Value should be set in 24 hour format time. For example setting the value to 2:30:45 will schedule the process 45 seconds past 2:30 AM.

Backing UpWith Productstream 2009, hot backup is available. This means that the database is not locked during backup process and is available to other processes and users. This process will still use system resources when running. If your backup takes an extended amount of time it may overlap some of the maintenance processes listed in this document. If necessary use the information provided to reschedule some of the process so they do not overlap.

SummaryEach background process will consume some of the system resources of the server they run on. Several of the background processes may run immediately and in parallel after a server upgrade or after a vault is attached. Depending on the size of the vault and the resources of the server it may take some time for each process to complete its first run. The system should be allowed to finish all of the processes without rebooting or stopping these processes. As shown in this document if the processes are interrupted they will be rescheduled or restarted at different times which could cause the initial vault setup to take longer than necessary.

Documents

Vault Background Processes Whitepaper