Generating Letters on the WebGenerating Letters on the WebUsing ASP.NET, XSLT and WordMLUsing ASP.NET, XSLT and WordML
Architectural Background & Major Enhancements in Architectural Background & Major Enhancements in Version 2.0Version 2.0
By: By: Ben AminniaBen Aminnia
President, L.A. SQL Server Professionals Group President, L.A. SQL Server Professionals Group www.sql.la Database Architect, Pointer Corporation Database Architect, Pointer Corporation www.pointercorp.com
22
AgendaAgenda
1. Introduction and Background1. Introduction and Background2. Architectural Overview2. Architectural Overview3. From Hello Word to a Production System3. From Hello Word to a Production System4. New Features in Version 2.04. New Features in Version 2.0
Managers CornerManagers Corner Four Storage Options for Generated LettersFour Storage Options for Generated Letters Multi-lingual Letters in Foreign LanguagesMulti-lingual Letters in Foreign Languages
5. Questions and Answers5. Questions and Answers
33
1. Introduction & Background1. Introduction & Background
Where it all started:Where it all started: My Presentation at SQL Server 2005 Launch My Presentation at SQL Server 2005 Launch
Event – December 2005Event – December 2005 Missing Piece of the PuzzleMissing Piece of the Puzzle
Opportunity Knocks!Opportunity Knocks! In Office 2003, Word becomes WordMLIn Office 2003, Word becomes WordML Web-based System to Generate, Archive, and Web-based System to Generate, Archive, and
Retrieve Template LettersRetrieve Template Letters Store Generated Letters in an XML Column of a Store Generated Letters in an XML Column of a
SQL Server TableSQL Server Table
44
2. Architectural Overview2. Architectural OverviewThe Old Method The New Method
Letter Template Development
Word Document with Bookmarks
XML Document with Tags
Composition OLE Automation to Create a Word Document and Replace Bookmarks with Runtime Values
XML Document and Replace Tags with Runtime Values
Storage Word Document on the File System
XML Column in a SQL Server Table
Content Search Windows Search through the File System
Full-text Index in SQL Server
Attribute Search Not Available (unless a loosely connected table points to the file system)
SQL Server WHERE Clause
Bookmark / Tag Search
Not Available XML Search
55
2. Architectural Overview2. Architectural Overview
Hello World Hello World oror Hello Word! Hello Word! Step 1 – Just the XML Tags: Step 1 – Just the XML Tags: HelloWorld1.xmlHelloWorld1.xml Step 2 – Open it with Word: Step 2 – Open it with Word: HelloWorld2.xmlHelloWorld2.xml Step 3 – Make it Updatable: Step 3 – Make it Updatable: HelloWorld3.xslt HelloWorld3.xslt
(Programming Shell for the WordML Document)(Programming Shell for the WordML Document)
66
3. From Hello Word to a 3. From Hello Word to a Production SystemProduction System
The Site: The Site: www.vipletters.com LoginLogin Role-Based View / ScopeRole-Based View / Scope Site-Based View / ScopeSite-Based View / Scope Generate a LetterGenerate a Letter ReportsReports Others piecesOthers pieces
77
3. From Hello Word to a Production System3. From Hello Word to a Production System
Goals & ObjectivesGoals & ObjectivesThe Architect’s Perspective:The Architect’s Perspective: Web-based Intranet (fewer than 1,000 users)Web-based Intranet (fewer than 1,000 users) Role-based (not everyone can see everything)Role-based (not everyone can see everything) A General Web Site for All Public Users vs. A A General Web Site for All Public Users vs. A
Specific Web Site for Each Client (with specific Specific Web Site for Each Client (with specific requirement that may not be applicable to the requirement that may not be applicable to the general site)general site)
And the most important of all …And the most important of all …
88
3. From Hello Word to a Production System 3. From Hello Word to a Production System
The WheelThe Wheel
99
3. From Hello Word to a Production System 3. From Hello Word to a Production System
The WheelThe WheelASP.NET Membership:ASP.NET Membership:
ASPNET_Profile------------------------------
UserIdPropertyNamesPropertyValuesString
ASPNET_Users--------------------
UserIdApplicationId
ASPNET_UsersInRoles-----------------------------
UserIdRoleId
ASPNET_Roles--------------------
RoleIdRoleNameDescription
ASPNET_Membership------------------------------
UserIdPasswordEmailIs ApprovedIsLockedOutLastLockoutDate
ASPNET_Applications------------------------------
ApplicationIdApplicationNameDescription
1010
3. From Hello Word to a Production System 3. From Hello Word to a Production System
Other Features and Other Features and RequirementsRequirements
Allow DEV to see more stuff (e.g. connection Allow DEV to see more stuff (e.g. connection strings)strings)
Show / Hide Certain Objects to / from One Client Show / Hide Certain Objects to / from One Client vs. the Public Sitevs. the Public Site
Connect to DEV / Public / Client DBsConnect to DEV / Public / Client DBs Updateable WEB.CONFIGUpdateable WEB.CONFIG Setup requirements by the Web HostSetup requirements by the Web Host
1111
3. From Hello Word to a Production System 3. From Hello Word to a Production System
Developing a New LetterDeveloping a New LetterDeveloping a New Letter – From A to ZDeveloping a New Letter – From A to ZA.A. Receive a Sample Letter and Verify / Create Receive a Sample Letter and Verify / Create
User Data Entry Tags – enclosed in [] User Data Entry Tags – enclosed in [] ((.DOC.DOC or or .DOCX.DOCX))
B.B. Create VIP Template Files Create VIP Template Files ((.XML.XML and and .XSLT.XSLT))
C.C. Create Data Entry Form and the [Generate] Create Data Entry Form and the [Generate] Button on the Form Button on the Form ((.ASPX.ASPX and and .ASPX.CS.ASPX.CS))
How about the How about the ReplicateReplicate function? function?What if there are 125 data entry fields?What if there are 125 data entry fields?
D.D. Add New Letter to VIP System Menu Add New Letter to VIP System Menu ((Web.SiteMapWeb.SiteMap))
E.E. Enter Authorized ASP.NET Membership Roles – Enter Authorized ASP.NET Membership Roles – allow / deny allow / deny ((Letters\web.configLetters\web.config))
1212
Developing a New Letter – From A to Z (Continued)Developing a New Letter – From A to Z (Continued)
If a new membership role needs to be created:If a new membership role needs to be created:
F.F. Create the new role in ASP.NET MembershipCreate the new role in ASP.NET MembershipG.G. Add menu access for the new role – allow / deny Add menu access for the new role – allow / deny
((web.configweb.config and and MembershipAdmin\web.configMembershipAdmin\web.config))
And Finally …And Finally …H.H. Add the XML tags of the new letter to the Add the XML tags of the new letter to the
DocumentMetadataDocumentMetadata table, which populates two table, which populates two DDList controls on the DDList controls on the Letters ReportLetters Report page. page.
3. From Hello Word to a Production System 3. From Hello Word to a Production System
Developing a New LetterDeveloping a New Letter
1313
Developing a New Letter – From A to Z (Continued)Developing a New Letter – From A to Z (Continued)
Some Additional Tricks:Some Additional Tricks:
• Letters with / without a Company LogoLetters with / without a Company Logo• Letters with / without a SignatureLetters with / without a Signature• Adding a Unique ID to the bottom of each Adding a Unique ID to the bottom of each
generated lettergenerated letter
3. From Hello Word to a Production System 3. From Hello Word to a Production System
Developing a New LetterDeveloping a New Letter
1414
• Many detailed areas can be covered here, which Many detailed areas can be covered here, which are applicable to developing reports and charts are applicable to developing reports and charts FOR ANY APPLICATION FOR ANY APPLICATION (not just VIP Letters)(not just VIP Letters)
• I have created separate presentations for I have created separate presentations for Developing Reports and ChartsDeveloping Reports and Charts
• One thing that I’d like to emphasize here …One thing that I’d like to emphasize here …• Since we are storing letters and their metadata Since we are storing letters and their metadata
in XML columns, you need to be fairly familiar in XML columns, you need to be fairly familiar with XML column retrieval techniques.with XML column retrieval techniques.
3. From Hello Word to a Production System 3. From Hello Word to a Production System
Developing Reports and ChartsDeveloping Reports and Charts
1515
• Similar to Reports and Charts, Managers Corner Similar to Reports and Charts, Managers Corner is also part of enhancements for managing the is also part of enhancements for managing the website, which is applicable to administration of website, which is applicable to administration of ANY APPLICATION ANY APPLICATION (not just VIP Letters)(not just VIP Letters)
• The Challenge: The Challenge: Coordinating creation of new Coordinating creation of new users among three people:users among three people:
• Manager: Identify new users and their system rolesManager: Identify new users and their system roles• New User: Specify Password , Security Question and New User: Specify Password , Security Question and
AnswerAnswer• Administrator: Help them out as neededAdministrator: Help them out as needed
• Solution:Solution: A simple form which is email enabled A simple form which is email enabled
4. New Features in Version 2.0 4. New Features in Version 2.0
Managers CornerManagers Corner
1616
4. New Features in Version 2.04. New Features in Version 2.0
Architectural Challenge for the DBAArchitectural Challenge for the DBA
Each record is about 100 KB large;Each record is about 100 KB large; So it takes ONLY about ten thousand records to So it takes ONLY about ten thousand records to
reach one GB in DB size;reach one GB in DB size; There’s no physical deletion; deleted records There’s no physical deletion; deleted records
are only marked for deletion (with are only marked for deletion (with [isdeleted]=1[isdeleted]=1););
1717
4. New Features in Version 2.0 4. New Features in Version 2.0
Reasons for Multiple Archival OptionsReasons for Multiple Archival Options Increasing cost of storage / hardwareIncreasing cost of storage / hardware Performance Degradation / Response TimePerformance Degradation / Response Time Legal and Regulatory RequirementsLegal and Regulatory Requirements Search and Retrieval TechniquesSearch and Retrieval Techniques Managers’ ConcernsManagers’ Concerns
1818
4. New Features in Version 2.0 4. New Features in Version 2.0
What is the management asking now?What is the management asking now? So where are you storing the generated letters? So where are you storing the generated letters?
INSIDE THE DATABASE?INSIDE THE DATABASE? Is it a good idea to have SQL Server as our Is it a good idea to have SQL Server as our
document repository medium?document repository medium? Some decision makers aren’t used to this, so Some decision makers aren’t used to this, so
they may ask: they may ask: “Can we have a file-system “Can we have a file-system repository instead?”repository instead?”
Is the answer Is the answer yesyes or or nono?? How about using BOTH SQL Server and File-How about using BOTH SQL Server and File-
System as our document repository? System as our document repository? What are What are the Pros and Cons of such approach?the Pros and Cons of such approach?
1919
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
1. Save documents in a database table1. Save documents in a database table Word Documents (VIP System utilizing Word Documents (VIP System utilizing
XML / XSLT)XML / XSLT) Other Documents (using Filestream or other Other Documents (using Filestream or other
methods)methods) See also See also Allen BerezovskyAllen Berezovsky’s blog: ’s blog:
http://www.harborobjects.com/AllenBerezovsky/post/2009/03/04/FILESTREAM-in-SQL-Server-2008.aspx
2020
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
2. Save documents on file system and 2. Save documents on file system and create a link in a database tablecreate a link in a database table
Historically, this used to be the Historically, this used to be the recommended way to archive documentsrecommended way to archive documents
In my RKCM table, over 40,000 letters have In my RKCM table, over 40,000 letters have been archived since 2002;been archived since 2002;
2121
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
2b. Save documents on file system – 2b. Save documents on file system – without a link on a database tablewithout a link on a database table
Write your own search / retrieval mechanismWrite your own search / retrieval mechanism For example, using NoSQL as described in For example, using NoSQL as described in
IEEE article;IEEE article; It’s amazing that despite its pros and cons, It’s amazing that despite its pros and cons,
it’s still offered as a viable option!it’s still offered as a viable option!
2222
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
3. Save documents both on file system 3. Save documents both on file system and in a database tableand in a database table
One prospect has actually asked about this One prospect has actually asked about this for an upcoming project;for an upcoming project;
What are the pros and cons?What are the pros and cons?
2323
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
4. Do NOT save documents anywhere 4. Do NOT save documents anywhere (neither on the file system nor on the (neither on the file system nor on the database)database)
• Think about it: Think about it: What does this mean?What does this mean?• Just save the original parameters which Just save the original parameters which
were used to generate the documentwere used to generate the document• When “retrieval” is needed, just “Re-When “retrieval” is needed, just “Re-
generate” the whole documentgenerate” the whole document• PROS and CONSPROS and CONS
2424
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
In the VIP System – Version 2.0, we now In the VIP System – Version 2.0, we now have all four archival / storage options:have all four archival / storage options:
On the databaseOn the database On the file systemOn the file system Both on the database and the file systemBoth on the database and the file system Neither on the database nor on the file Neither on the database nor on the file
system (just re-generate the letter when system (just re-generate the letter when user is trying to retrieve the archived user is trying to retrieve the archived document)document)
2525
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
In the VIP System – Version 2.0, we now In the VIP System – Version 2.0, we now have all four archival / storage options:have all four archival / storage options:
Configuration of archival option is per letter Configuration of archival option is per letter templatetemplate
Each letter template has a record in Each letter template has a record in DocumentConfigDocumentConfig table table
There are two places where There are two places where DocumentConfigDocumentConfig record is utilized:record is utilized:
• When [When [GenerateGenerate] is clicked to create a letter] is clicked to create a letter• When When SelectSelect is clicked to retrieve a letter is clicked to retrieve a letter
2626
4. New Features in Version 2.0 4. New Features in Version 2.0
Document Storage & Archival OptionsDocument Storage & Archival Options
In the VIP System – Version 2.0, we now In the VIP System – Version 2.0, we now have all four archival / storage options:have all four archival / storage options:
BUT WAIT! BUT WAIT! What if the record in What if the record in DocumentConfigDocumentConfig table is subsequently table is subsequently changed?changed?
DocumentArchiveDocumentArchive table keeps track of each table keeps track of each individual letter’s archival status as of the individual letter’s archival status as of the time it was originally generated.time it was originally generated.
A change in A change in DocumentConfigDocumentConfig table is NOT table is NOT RETROACTIVE!RETROACTIVE!
2727
4. New Features in Version 2.04. New Features in Version 2.0 Foreign Language & Multi-lingual LettersForeign Language & Multi-lingual Letters
Let’s clarify the terminology here:Let’s clarify the terminology here: A letter in a foreign language is entirely in A letter in a foreign language is entirely in
one (non-English) language;one (non-English) language; A multi-lingual letter has more than one A multi-lingual letter has more than one
language in the same letter;language in the same letter; The VIP system supports both foreign The VIP system supports both foreign
language and multi-lingual letters;language and multi-lingual letters;
2828
4. New Features in Version 2.04. New Features in Version 2.0 Foreign Language & Multi-lingual LettersForeign Language & Multi-lingual Letters
Some preliminary design questions:Some preliminary design questions: How about the user data entry form?How about the user data entry form? How about values that are entered on the How about values that are entered on the
data entry form?data entry form? Does the user have a foreign language Does the user have a foreign language
keyboard?keyboard? Are the values going to be copied / pasted Are the values going to be copied / pasted
from another system which is already in the from another system which is already in the foreign language?foreign language?
2929
4. New Features in Version 2.04. New Features in Version 2.0 Foreign Language & Multi-lingual LettersForeign Language & Multi-lingual Letters
Based on answers to design questions, Based on answers to design questions, we’ll need one or more of the following:we’ll need one or more of the following:
Letter template in Word, saved as XML, etc.Letter template in Word, saved as XML, etc. Creation of a separate Creation of a separate xsltxslt file per language file per language Data entry form(s)Data entry form(s) User selection of desired languageUser selection of desired language A foreign language keyboardA foreign language keyboard Code-behind to select the language-specific Code-behind to select the language-specific
xsltxslt file accordingly file accordingly
3131
Contact InformationContact Information
• Emails:Emails: [email protected] [email protected]
• Websites:Websites: www.sql.la www.pointercorp.com www.vipletters.com www.takeatest.net