Embed Size (px)
Citation preview
Assignment 11 SQL Integration Services
1. SQL 2005 Samples A. Get Samples AND
1. http://www.microsoft.com/downloads/details.aspx?FamilyId=E719ECF7-9F46-4312-AF89-6AD8702E4E6E&displaylang=en
2. B. Get the BICI (business intelligence: case insensitive)
C. D. Attach the AdventureWorksDW database
E. F. Security . Logins .ASP/Net properties . Map to AdventureWorksDW as
datareader and datawriter
G.
Page 1
Assignment 11 SQL Integration Services H. Installing the Samples should give you the following
I.
2. SQL Integration Services A. Part of new SQL 2005 BI Services B. Successor to SQL 2000 DTS (data transformation services)
1. But all new… not an upgrade C. Its throughout SQL… SQL Maintenance Plan is an SSIS package
3. The BI “Platform”: Integrate; Analyze; Report A. Integration Services to acquire and transform data B. Analysis Services to enrich and discover (data mining) with hierarchical
views C. Report Services to present data…. Into information.
4. DTS -> SSIS… limited migration path A. Recommend rewrites B. BUT DTS still runs C. DTS uses scripting; SSIS uses components D. SSIS is much faster with complex logic…(X7)
1. ie change casing… removing bad data etc.
5. So what problems are we trying to solve A. Data from many diverse sources
1. Text, Binary (legacy), Current OLTP database 2. These need to be mined, transformed and cleaning . 3. Then compile into a data warehouse. 4. This used to required many stages and applications
B. One data pipeline that sends data directly to warehouse. 1. Includes data mining to consolidate data
Page 2
Assignment 11 SQL Integration Services
6. Example 1 Creating a ETL (Extract – Transfer – Load ) Package A. Open the following text file.
B. 1. This has AverageRate (1) 2. Currency Type 3. DateTime 4. End of Day Rate
a) If not told otherwise the flat file conversion makes all columns strings(50)
C. Now check the destination in AdventureWorksDW .
D. E. Both CurrencyKey and TimeKey are foreign keys related to “lookup” tables
DimTime and DimCurrency. 1. So we will have to “transform” the data from the flat file into “keys”
F. AverageRate and EndofDayRate are floats.
Page 3
Assignment 11 SQL Integration Services
7. Use Visual Studio (BI Development Studio) A. File . New Project BIProject Integration Services Project: SSIS_1
B. C. Rename package to CurrencyImport..
D.
8. Connection Managers: Templates for future connection tasks
9. New Flat File Connection (location and mapping) A. Right click on connections at the bottom. New Flat File B. Open the sample you viewed earlier… accept defaults…Name as shown.
C.
Page 4
Assignment 11 SQL Integration Services D. Advanced. Change names as follows
E. Click on Suggested Data Types.
1. Based on first 100 rows of data. F. The two rates it got right…float. G. But the alternate keys in lookup talbes DimCurrency and DimTime require
TimeStamp and nChar (Unicode)
H.
I.
10. New AdventureWorksDW OLEDB connection A. Right click on connections tab on the bottom
B.
Page 5
Assignment 11 SQL Integration Services
11. Add a Data Flow Task A. Does the heavy lifting… from source to destination
1. Transform, clean, modify 2. ETL or Load data into source.
B. Note… Data Flow is NOT Flow Control… 1. One has to do with data and the other with the entire project.
C. Drag onto Control Tab a Data Flow task and rename as follows
D.
12. Add Flat File Source A. Drag on a Flat File Source from tool box B. Rename as shown… ensure the columns are correct from connection
manager.
C.
Page 6
Assignment 11 SQL Integration Services
13. Add Lookup task to transform foreign key data. A. From transform toolbox drag on Lookup Task B. Rename as shown
C. D. Properties: Reference Table
E. F. Columns tab use drag from left to right to link fields
G. 1. This says “Link CurrenyID from the flat file to CurrencyAlternateKey in
DimCurrency and return the CurrencyKey into a new column
Page 7
Assignment 11 SQL Integration Services H. Drag on another Lookup for Currency Date
1. I. Link to DimTime as follows
J.
14. Add AdventureWorks table as destination A. From the DESTINATIONS tab drag on a OleDB Destination task B. Rename as shown
C. D. Drag green arrow from lookup to destination E. Double Click on destination and Select the FactCurrencyRate Table F. Check Mappings…
Page 8
Assignment 11 SQL Integration Services
15. Test A. Add data viewers to green arrows below flat file and above the database
B.
16. F5 to Run… Screen print the following from the first Viewer
A. B. Note this is prior to the transformation C. Green arrow to continue
Page 9
Assignment 11 SQL Integration Services D. Screen print the second viewer
E. F. Note the addition of the keys from the lookup table. G. Green arrow to complete H. All turns green. I. Click on Progress Tab….This is GREAT information!
J. K. ***
L. Finally when done: Stop Debugging…
Page 10
Assignment 11 SQL Integration Services
17. Example 2: Looping
18. Create the Looping Control A. Goal … loop through a folder of files and import each using the Dataflow
you have already created. B. Open the following folder and note the files C. C:\Program Files\Microsoft SQL Server\90\Samples\Integration
Services\Tutorial\Creating a Simple ETL Package\Sample Data
1. D. Copy . Paste and rename the current package as follows
E. F. On Control Flow Tab… drag on a ForEachLoop from the toolbox
1. Rename as shown
2. G. Double click and set collection as shown
H. I. In Variable Mappings
1. Create a new variable
2. 3. Name varFileName… accept defaults 4. OK back to Control Flow
Page 11
Assignment 11 SQL Integration Services J. Drag Data Flow into the Loop as shown
K.
19. Modify the Flat File connection to accept variable A. Open properties of CurrencySourceFlatFile B. Click on … of Expressions C. Assign the variable @varFileName to the ConnectionString D. As this datasource is called the value in varFileName will be assigned to
the connectstring.
20. Testing Loop A. In SQL Management get a row count for FlatCurrencyRate
B. C. Run…Note that with each file opened it opens the two GridViews\
Page 12
Assignment 11 SQL Integration Services
D. E. Continue to step through each of the files…. F. When done note the Progress Tab…shows each file
1. \ G. And note the new rowcount after the import
H.
21. Example 3… Using configurations to change folders A. Copy and paste package as shown
B. C. Click on Control Flow Tab surface and select SSIS . Variables from menu
Page 13
Assignment 11 SQL Integration Services
D. E. Add variable varFolderName
F. G. As Shown
H. I. In the ForEachLoop Double click and open Collections and Add an
Expression variable for the Directory Property J. Tip… you can drag and drop this from the list of variable in the expression
builder.
Page 14
Assignment 11 SQL Integration Services
Page 15
22. Enable Package Configurations… A. Click on Control Tab back ground to set scope B. Select Package configurations from the SSIS menu
C.
Assignment 11 SQL Integration Services D. Enable and Add as shown
E. F. This fires the wizard G. Confirm XML and browse to default project folder H. Type: SSISTutorial and save… it will create the filename.
I. J. Creates the following.
K. L. NEXT
Page 16
Assignment 11 SQL Integration Services M. To expose a property to set with this configuration file.
N. O. Next and Finish P. This creates the following file in the project root
Q.
23. Make a folder with a copy of the sample data. A. Copy the sample data folder and paste into the root (ie drive c:\) (or any
location) on your computer
Page 17
Assignment 11 SQL Integration Services
Page 18
B. Rename the copied folder to More Sample Data
C.
24. Open SSISTutorial.dtsConfig with Visual Studio and format document for easy editing. A.
B. C. Change the ConfiguredValue as shown for Properites[Value]
Assignment 11 SQL Integration Services D. Save and Test… Note the progress tab… E. It should display the new file path.. F. Step through and finish
G.
25. Just to prove it all works… Get another row count on the destination table
A. B. Note yours will be different but it should have changed.
26. Hand in all screen prints to prove this all works… A.
Page 19
Assignment 11 SQL Integration Services
Page 20
