View
224
Download
0
Category
Preview:
DESCRIPTION
3 Session Chapters 1.Introduction to ARC 2.Getting Started 3.ETL 1.Architecture 2.License Mechanism 3.ARC in Multi-ADM environment 4.Maintenance Part one Part two
Citation preview
ARC Basic Training
Idit Shachar-AbutbulMarch 2011
Part one
2
Copyright Statement All of the information and material inclusive of text, images, logos, product names is either the property of, or used with permission by Ex Libris Ltd. The information may not be distributed, modified, displayed, reproduced – in whole or in part – without the prior written permission of Ex Libris Ltd.
TRADEMARKS Ex Libris, the Ex Libris logo, Aleph, SFX, SFXIT, MetaLib, DigiTool, Verde, Primo, Voyager, MetaSearch, MetaIndex and other Ex Libris products and services referenced herein are trademarks of Ex Libris, and may be registered in certain jurisdictions. All other product names, company names, marks and logos referenced may be trademarks of their respective owners.
DISCLAIMER The information contained in this document is compiled from various sources and provided on an "AS IS" basis for general information purposes only without any representations, conditions or warranties whether express or implied, including any implied warranties of satisfactory quality, completeness, accuracy or fitness for a particular purpose.
Ex Libris, its subsidiaries and related corporations ("Ex Libris Group") disclaim any and all liability for all use of this information, including losses, damages, claims or expenses any person may incur as a result of the use of this information, even if advised of the possibility of such loss or damage.
© Ex Libris Ltd., 2009
3
Session Chapters
1. Introduction to ARC2. Getting Started3. ETL
1. Architecture 2. License Mechanism3. ARC in Multi-ADM environment4. Maintenance
Part one
Part two
4
Chapter one:Introduction to ARC
5
Chapter Objectives• What is ARC• ARC relationship with the Aleph 500 ILS• ARC Packages and Reports• ARC Studios
6
What is ARC• ARC = ALEPH Reporting Center• ARC is a web-based reporting, querying and
statistical environment.• ARC integrates the Cognos ReportNet™
software
7
The ARC Relation To The ALEPH 500™ ILS
ARCARCDatabaseDatabase
ALEPH ALEPH DatabaseDatabase
ARC Tables
ALEPH Data
ALEPH Tables
The ARC takes periodic “Snap Shots”
of the ALEPH 500™ALEPH 500™ set of tables and arranges
them into a completely separate database.
8
Cognos ReportNet Components
• Cognos Connection Web presentation layout/portal
• Query Studio Ad hoc reporting tool
• Report Studio Professional reporting tool
9
ARC Predefined Reports
• ARC comes complete with a selection of standard pre-defined reports and statistics
• These reports and statistics draw on information from data sources within the ALEPH system modules.
10
ARC Predefined Reports
11
ARC Packages
• Predefined Reports are organized into packages
• The packages are based on subjects related to various areas of library management.
12
To create a report about…
Choose Package Main Subjects for Analysis
Loans Loans Information about Loans and Loans History
Cash Cash Summary of Fines information and Patrons Cash Payments, based on Fines data
Requests Requests Information about Hold Requests and Booking Requests
Circulation Activities Events Circulation information and statistics about circulation activities, such as: Loans, Returns, Renewals, Lost and Claimed Returned Items - Based on circulation transactions and circulation events
Orders Acquisition Orders Information about Order Budgets, Expenditures, Vendors, and Claim and Acquisition statistics in general, based on Order data.
Patron information Patrons Local and global patron information
13
Items Inventory Information about Items and Items history and Circulation activities based on Item data.
Course Reading Course Reading Information from course reading data and its related Items.
Budgets Acquisition Budgets Information about Budget transactions, Orders, and Invoice Header based on Budget Transaction data.
Web OPAC OPAC OPAC Statistics
Cataloging Information Bibliographic Data Bibliographic and Catalogers Information
ETL Process ETL ETL process
ARC Usage Audit User Activities within ARC
14
ARC Studios – Query studio
• The Query Studio allows you to:
• Create ad hoc, simplified reports and statistics using simple “drag and drop” into the report authoring window
15
ARC Studios – Report studio
• The Report Studio allows you to:
• Create new sophisticated reports that analyze library data according to specific information needs.
• Report Studio provides many methods of formatting, presenting, and distributing your findings.
16
Chapter two:Getting Started
17
Chapter Objectives• Logging into ARC• ARC portal structure• My Folders vs. Customized Reports• Personalizing ARC• Actions on reports• Searching in ARC• Query Studio quick overview
18
Logging into ARC• The ARC URL syntax is
http://<server_name>:<port*>• Login for support is
administrator/administrator999
* Usually 8081
• This is the default login and is not used by customers.
• More about this later in part two.
19
ARC Portal
Tabs for Public folders and My folders
The connected user
and his scheme
Main tool bar
Main Menus
20
My Folders vs. Customized Reports
My Folders Customized ReportsCan only be accessed by you when you log-in
All users can access
Reports saved here can be modified
Reports saved here can be modified
Present in a separated tab- My Folders tab
Present under Public Folders tab
21
Personalizing ARC• Click on “My Preferences”
to Personalize the ARC portal for your user
• You can determine the following:• the amount of entries in a
list• the product and content
language • other defaults
22
Personalizing ARC
23
Personalizing ARC• For Content Language we support the
following languages:English, German, French, Italian, Czech, Danish, Swedish, Simplified & Traditional Chinese
24
Actions on reports• Main actions are:
Run, Open, Schedule and Create a Shortcut or a Report View
25
Actions- Create a report view
• A report view shares the same report specification as the source report, but other properties such as prompt values, schedules, languages, and output formats are independent of the source report.
26
Actions- Create a shortcut
• A shortcut is a pointer to another entry such as a report, report view, folder, job, or URL.
• Updating the source automatically updates all shortcuts to the entry.
• You can change access permissions for a shortcut entry, but it does not change the access permissions for the source entry.
27
Actions – Create a Schedule
• Reports can be scheduled individually or by using a job.
• Reports can be scheduled to run • by minute, hourly, daily, weekly, monthly or
yearly • for different output formats • saved or distributed to mailing lists
• A schedule can be associated with each report or job only once.
28
Actions – Create a Schedule
29
Searching in ARC• You can search for entries whose name,
description, or both match the string entered in the search criteria.
30
Searching in ARC
• You can find types of entries by leaving the search string empty and selecting a specific entry type in the advanced search criteria.
• The search is not case sensitive.
31
Query Studio quick overview• Use Query Studio to:
1. Create ad hoc reports that can be viewed by others.
2. Customize predefined reports created in Query Studio.
32
Query Studio quick overview
33
Query Studio quick overview
34
Report Example
35
Chapter three:ETL
36
Chapter objectives• Basic outlines of ETL process and architecture• ETL and ARC Database Layers• Extract in detail• Transform and Load in detail• ETL configuration and running• Related Documents
37
Basic outline of ETL process & architecture• The ETL process Extracts data from ALEPH,
Transforms it to a different physical structure, and Loads it into the data warehouse database.
Source system - ALEPH
Extraction, Transformation and Load tool
Data Warehouse
Client / server
38
ETL and ARC Database Layers
DWH
ARC schema
Extract Transform
Load
MRR STG
ALEPHDatabase
andConfiguration
39
• STG - The Staging area layer transforms data from the MRR layer, and builds the fact and dimension tables in a de-normalized structure (star schema). In every loading process this layer is truncated and filled again. At the end of this step the data from STG tables is loaded into the DWH layer.
ARC Database Layers (cont.)• MRR – A mirroring area. Includes tables which are
extracted from ALEPH but not necessary in the ALEPH table structure. This layer is static, i.e. no processing on MRR tables is done.
MRR STG DWH
ALEPH ARC schema
Database and
Configuration
MRR STG DWH
ALEPH ARC schemaDatabase
andConfiguration
40
ARC Database Layers (cont.)• DWH – Data Warehouse- The end-user layer
which includes tables with the same structure as the STG tables. This layer is fully indexed to support efficient retrieval.
MRR STG DWH
ALEPH ARC schemaDatabase
andConfiguration
41
ETL and ARC Database Layers (cont.)
MRR STG
ALEPH ARC schema
Extract Transform
STG layer is synonymous with the DWH layer
Database and
Configuration
Load
DWH
42
ARC ETL Architecture• The three ARC components – MRR, STG and DWH
– must reside on the same machine.
• It is recommended that ARC and ALEPH not be on the same machine.
• The Load phase is much heavier on IO, hence it will significantly affect the ALEPH online performance
43
Extract
• The base of the extraction is the administrative library.
• Extraction of all additional information, such as bibliographic data, is confined to records directly related to administrative records.
44
Extract
• The parallel processing is managed by a cycle table which is very similar to the cycle tables used for index building in ALEPH.
• Each cycle relates to a specific MRR table; as such it may gather information from different Oracle and configuration tables in ALEPH.
45
Extract
• p_arc_01.cycles example:
0001 + + BIB-INFO 000000101-000048621 $0002 + + CATALOG $0003 + + ITEMS 000000001-000048621 $0004 + + COURSE $0005 + + EVENT 000000001-000050000 $0006 + + EVENT 000050001-000100000 $0007 + + EVENT 000100001-000140816 $
46
Extract
• Each cycle creates a sequential file to be loaded into the respective MRR table.
• The extraction is done by a standard ALEPH dump procedure.
47
• It is possible to limit the amount of data that is extracted: • How many history records (items, loans and requests)
will be extracted• How many OPAC and Events transactions will be
extracted• If patron ID’s will be extracted
• This is done by configuring the file arc_conf_table.conf dlib <USM50> dt vi arc_conf_table.conf [ARC-HISTORY] from year = 2004 [OPAC] from date = 20050101
[EVENTS]From date = 20070101[PATRON-DATA]eliminate_data=N
Extract
48
• Following the extraction from ALEPH to the MRR tables, the following steps are performed:
• Truncate staging area tables. • Inserting data from the MRR layer into the STG
layer (relevant for STG tables whose structure is equal to the MRR tables).
• Building STG layer (mainly the fact tables and calculated dimensions like the periods and time dimensions).
Transform and Load
49
• The transformation phase is composed of stored procedures that run PL/SQL statements.
• These stored procedures reside in an Oracle table.
Transform and Load
50
• Changes related to the ETL (extract part) are done on the ALEPH side and distributed as part of the ALEPH Service Pack.
• In addition to the regular ALEPH Implementation notes, a special document is distributed that lists mandatory implementation notes that should be done on the ALEPH side by ARC users.
ALEPH SP
51
• Changes related to the reports and ETL (transform part) are done on the ARC side and distributed as an ARC Service Pack.
• In addition to the regular ARC Implementation notes, a special document called “ARC Service Pack” describes how the ARC service pack should be implemented.
• The latest ARC service pack should be installed as part of each ARC installation.
ARC SP
52
• Before running ETL, you must verify that you have the latest service packs installed on both the ALEPH and the ARC servers.
• Do all required configuration, according to the “ALEPH configuration for ARC” document.
Running ETL for the first time
53
Configure ETL
This configuration allows the connection between the ARC and the ALEPH DBs.
Thumb rules:Use ARC_TWO_TASK when ARC and ALEPH are installed on different servers.Use ARC_ORACLE_SID when they are installed on the same server.Exceptions:When ARC and Aleph are installed on the same sever, but have different Oracle version then also use ARC_TWO_TASK
54
exlarc.arc=)description= (address= (port
On ARC server type:cd $ORACLE_HOME/network/admincat tnsnames.ora
# Reporting Center Definitionssetenv ARC_TWO_TASK exlarc.arc#setenv ARC_ORACLE_SID aleph5
setenv ARC_ADMIN ARC_ADMIN [ Will usually be ARC_ADMINset arc_user = “USMR0” [ ARC schema name]
Configure ETLARC_TWO_TASK Example: dlib
<usm50>drvi prof_libraryOnly one should stay
unmarked
55
# Reporting Center Definitions#setenv ARC_TWO_TASK exlarc.arcsetenv ARC_ORACLE_SID aleph5
Configure ETLARC_ORACLE_SID Example:
For For ARC_ORACLE_SID:ARC_ORACLE_SID:From crn user type:echo $ORACLE_SID aleph5
dlib <usm50>drvi prof_libraryOnly one should stay
unmarked
56
Additional configuration• You can define the size of the cycles in the extract
phase.• This is done by configuring $data_root/prof_libraryidits@il-aleph07(a20_3) USM50> grep p_arc_01 prof_library setenv p_arc_01_loop_length 50000 setenv p_arc_01_time_loop_length 002 setenv p_arc_01_bor_loop_length 3000
57
• Test Oracle connection with ARC DB by running csh -f p_arc_01_test as defined in prof_library.
Test the connection
58
csh –f p_arc_01 <usm50>,<ALL>,<Y>,<4> >& $data_root/report/p_arc_01.log.20110320_1113 &
parameters: <usm50> = ALEPH ADM library name <ALL> = all tables or one (i.e.- bib-info). If using 1 table - can run on debug mode. <Y> = Y/N lock library <4> = number of processes = number of processors-1. To know number of processors: on Sun – uname –X On Linux – top.data_root/report… = for log file. Make sure directory exists.
Run ETL
59
• ETL process creates a few log files:• $alephe_scratch/p_arc_01.log.<date> -
Main ETL log file (as any Aleph batch job)• $data_scratch/p_arc_01.log – log of records
loaded.• $data_scratch/p_arc_01_transform.log - log
of transform process.• $data_scratch/p_arc_01.cycles – log of the
cycles progress
ETL Log Files
60
ETL Log Files (cont.)• Example of info given by p_arc_01.log:Starting Process Mon Jan 31 13:58:26 IST 2011Table mrr_acquisition_method Successfully Loaded 3 of 3Table mrr_approval_department Successfully Loaded 2 of 2Table mrr_bib_info Successfully Loaded 31989 of 31989Table mrr_cash Loaded 126520 of 739484 - Not all records were loadedTable mrr_unfied_cataloging Successfully Loaded 0 of 0 End Extract Process Mon Jan 31 14:02:45 IST 2011TWO_TASK : exlarc.arcORACLE_SID : aleph9ARC_ADMIN: ARC_ADMINStarting Transform Process Mon Jan 31 14:02:45 IST 2011End Transform Process Mon Jan 31 14:16:57 IST 2011
61
ETL Log Files• Another way to view errors of the transform process is via
the ARC portal under Public Folders ETL ETL status:
62
Running ETL in Debug Mode• Sequential files created by the EXTRACT phase are not
deleted. They remain in the ADM $data_files directory.
• TRANSFORM phase is not applied.
• When executing ETL on a single table. The parameters noted above remain the same, except for the table name, replacing the ALL parameter.
• setenv ARC_DEBUG Yapcsh –f p_arc_01 USM50,BIB-INFO,N,3
63
ETL in debug mode- p_arc_01.log
Starting Process Thu Jun 22 18:50:41 EST 2006Table mrr_fact_items Successfully Loaded 8750976 of 8750976End Extract Process Thu Jun 22 20:29:16 EST 2006Debug Mode - No Transform
64
• ETL can be set to run on a regular basis from the job_list.
• To set it, you need to add corresponding records to two tables: job_list and job_list.conf.
• Both tables reside under $alephe_tab
ETL Configuration in the job_list
65
• Suppose you want your ETL to run: Once a week, on a Sunday morning at 07:00 apcsh -f p_arc_01 ABC50,ALL,N,3 and
• 2) job_list! Running ETL once a week (Sunday) for ALL tables!W8 07:00:00 Y ABC50 p_arc_01 ABC50,ALL,N,03
• 1) Job_list.confW8 W YNNNNNN - run on Sunday
ETL Configuration in the job_list
66
Related Documents• 1. “ALEPH Configuration for ARC”
• Explains configuration steps that should be done on the ALEPH side before ETL activation.
• 2. “ALEPH to MRR Mapping”• Provides mapping between ALEPH Z Oracle tables and ARC
MRR layer tables. • 3. “MRR to DWH Mapping”
• Provides mapping between ARC MRR layer tables and ARC DWH layer tables.
• 4. “ARC Logical Model”• Provides mapping between ARC presentation layer packages
and ARC DWH layer tables.
Thank You!Idit.Shachar-Abutbul@exlibrisgroup.com
Recommended