Walkthrough: Microsoft Sync Framework Power Pack for SQL Azure November CTP

Writer: John Mulhausen

Technical Reviewer: Deepa Choundappan, Sean Kelley, Maheshwar Jayaraman.

Published: November 2009

Applies to: SQL Server 2008 R2 November CTP, SQL Azure, Microsoft Sync Framework 2.0

Summary: This document is a walkthrough of the components, features, and functionality of Microsoft Sync Framework Power Pack for SQL Azure November CTP. This release features the SQL Azure provider for Microsoft Sync Framework, a plug-in for Visual Studio 2008 Professional SP1 and SQL Azure Data Sync Tool for SQL Server, all of which simplify using Sync Framework and SQL Azure together.

PurposeMicrosoft Sync Framework was built with the goal of creating an extensible framework that can synchronize with any kind of data over any protocol. Since shipping Sync Framework 2.0 in October 2009, the team has focused on expanding Sync Framework’s functionality to include a free provider that can synchronize with Microsoft’s cloud-based relational database service built on SQL Server technologies, named SQL Azure.

While it was possible with some tweaking to use the existing ADO.NET provider for synchronization with SQL Azure, Microsoft has created a synchronization provider that performs efficiently, lowers the barrier to entry, and ensures reliability when synchronizing with SQL Azure.

Microsoft Sync Framework Power Pack for SQL Azure November CTP contains the new provider, SqlAzureSyncProvider, and tools that automate its use.

SQL Azure Data Sync Tool for SQL Server offers users a wizard that provisions and configures a local SQL Sever database and SQL Azure database for synchronization with one another. With this tool, enterprise-to-cloud synchronization can be achieved in a few mouse clicks.

A new Visual Studio 2008 Professional SP1 item template called SqlAzureDataSyncClient has been created to make it easy to cache data from SQL Azure in order to provide offline capabilities for applications that want to consume data from the cloud.

ComponentsMicrosoft Sync Framework Power Pack for SQL Azure November CTP is comprised of the following components:

1. SqlAzureSyncProviderThe power of the Sync Framework engine has been extended by the new synchronization provider for SQL Azure, written by Microsoft. Named SqlAzureSyncProvider, this is a new and free provider, which will enable the Sync Framework engine to understand and synchronize with data that is being exchanged between SQL Azure and any database that can be synchronized with Sync Framework, using efficient and reliable algorithms to handle the complexities of cloud synchronization.

2. Visual Studio Plug-InThis plug-in extends the functionality of Visual Studio 2008 Professional SP1 by providing a new Visual Studio item template called SqlAzureDataSyncClient. Projects that use the item are pre-populated with both the appropriate assembly references for SQL Azure-to-client synchronization, and a generated class which contains a method named Synchronize that the developer can call in an application to automatically synchronize with data from the cloud.

3. SQL Azure Data Sync Tool for SQL ServerThis tool contains a wizard that walks users through the SQL Azure connection process, automating both the provisioning and synchronization of data between SQL Server and SQL Azure. When complete, the tool saves the synchronization configuration as a job in SQL Server Agent, which is ready to be run at any time.

SqlAzureSyncProvider FeaturesSqlAzureSyncProvider is a new database provider for Sync Framework 2.0 that makes it easier than ever to create cloud-aware applications. Key features include:

1. Derived from SqlSyncProvider, Optimized for SQL AzureIn addition to leveraging the efficiency and maturity of SqlSyncProvider, the provider intelligently handles some SQL Azure-specific complexities that are required on a multi-tenant system. SqlAzureSyncProvider also features SQL Azure-optimized implementations of Sync Framework’s support for batching of change application, as well as conflict resolution and scope-based synchronization. In addition, the provider decreases the number of round-trips to the server by using table-valued parameters (TVPs) to apply changes.

SQL Azure uses a throttling mechanism to minimize the impact of runaway operations on other users by controlling log growth and limiting transaction size. When synchronization triggers this throttling mechanism, SqlAzureSyncProvider responds by using a “back-off algorithm” which automatically reduces batch sizes from the default of 5,000 rows during synchronization. A helpful side-effect of the use of this algorithm is that changes are viewable before synchronization is complete, and synchronization progress can be displayed in real time.

2

2. New SQL Azure EventsWhen writing an application that uses synchronization, developers rely on events to automatically detect and handle necessary synchronization operations. Existing events provided by Sync Framework, such as ApplyChangeFailed, ApplyMetadataFailed, and all change enumeration events are here, but so are new events like AzureBatchApplied, which is fired when a batch of changes is successfully applied to the SQL Azure database.

3. Automated ProvisioningGetting off the ground has largely been automated by the SqlAzureSyncScopeProvisioning and SqlAzureSyncTableProvisioning classes. Everything that needs to happen cloud-side and on-premise is taken care of by detecting the presence of, and (if necessary) creating, all of the appropriate metadata tables. This is taken a step further by the new SqlAzureDataSyncClient Visual Studio plug-in and SQL Azure Data Sync Tool for SQL Server, both of which automate the setup of SQL Azure connectivity, provisioning, and filtering.

WalkthroughThis section describes the preparation, setup, and use of Microsoft Sync Framework Power Pack for SQL Azure November CTP.

1. Preparationa. Ensure that the Sync Framework 2.0 SDK has been installed.

i. The Sync Framework 2.0 SDK can be downloaded from the following page: http://www.microsoft.com/downloads/details.aspx?FamilyID=89adbb1e-53ff-41b5-ba17-8e43a2e66254

b. Ensure that an account at SQL.Azure.com exists. i. Sign up with a Windows Live ID at:

http://www.microsoft.com/windowsazure/getstarted/#sql c. Ensure that Visual Studio 2008 Professional SP1 has been installed and is not running.d. Ensure that SQL Server 2008 has been installed and is not running.

i. For best results, use the November CTP release of SQL Server 2008 R2 or later. The November CTP ads native support for SQL Azure to SQL Server Management Studio. The November CTP release of SQL Server 2008 R2 can be downloaded from: http://www.microsoft.com/sqlserver/2008/en/us/R2Downloads.aspx

e. Ensure that the SQL Server Agent service is running. i. To check the status of this service, click the “Start” button, then click “Run.”

1. If using Windows 7, type “services” in the text field labeled “Search programs and files”, then click on the program called “Services.”

2. Otherwise, type “services” in the text field labeled “Open” and hit return.

ii. In the “Services” window, scroll to the entry for “SQL Server Agent.” Right-click the entry and then click “Properties.”

3

iii. In the “SQL Server Agent Properties” window, click the drop-down menu labeled “Startup type” and select “Automatic,” then click the “Start” button located under the label “Service Status.” Finally, at the bottom of the window, click “Apply,” then “OK.”

iv. Close the “Services” window.

2. Installationa. Download Microsoft Sync Framework Power Pack for SQL Azure November CTP, then

double-click the MSI file and run the installation wizard using the default options. This will set up the SQL Azure Sync plug-in for Visual Studio 2008 Professional SP1 as well as SQL Azure Data Sync Tool for SQL Server.

i. This download is available at the Sync Framework page on MSDN Developer center: http://go.microsoft.com/fwlink/?LinkID=168920&clcid=0x409

b. Installation to a custom file path is not supported. If the tool is installed to a custom path, any SQL Server jobs generated by these tools will need to be manually updated.

3. Usage

1. SQL Azure Data Sync Tool for SQL Server WalkthroughThis walkthrough demonstrates the tool SQL Azure Data Sync Tool for SQL Server, which utilizes SqlAzureSyncProvider to automate both the provisioning and synchronization of data between an on-premise SQL Server and SQL Azure.

1. Click “Start,” then “All Programs,” “Microsoft Sync Framework,” and “SQL Azure Data Sync Tool for SQL Server.”

2. This launches a wizard that will configure the local SQL Server instance and automatically provision an Azure database for synchronization. Click “Next” to proceed.

3. Enter a valid server name, database name, username, and password associated with a SQL.Azure.com account. Click “Next” to proceed.

4

Figure 3.1.1: Enter valid SQL Azure credentials.

a. SQL Azure Firewall settings can affect connectivity. For more information on how to configure SQL Azure Firewall settings, see http://go.microsoft.com/fwlink/?LinkId=166665.

4. Enter the name of the SQL Server instance and the database that will be synchronized with SQL Azure.

Figure 3.1.2: Enter the names of the local SQL Server instance and local database.

5

5. In the select box, highlight tables in the database associated with the entered SQL.Azure.com account to add them to the synchronization scope. Then, set the conflict resolution policy. Click “Next” to continue.

Figure 3.1.3: Create the synchronization scope.

a. When “Process” is clicked, the schema for any tables that were added to the synchronization scope will be created in the SQL Azure database based on the schema in the local database, but the tables will be empty until the synchronization job is run for the first time.

b. This tool assumes that the selected local database has not been set up for synchronization with SQL Azure. If a database with the same name is detected under the selected SQL Azure account, the tool will prompt the user to drop the database from SQL Azure and rerun the tool.

c. The conflict resolution policy determines which version of the data managed by the synchronization relationship will be propagated in the case of the data being updated in both Azure and SQL Server since the last synchronization.

6. In the select box, use the “Move Up” and “Move Down” buttons to list the selected tables in the order in which they should be synchronized during a job. Click “Next” to proceed.

6

Figure 3.1.5: Order the tables by their foreign-key dependencies.

a. Tables must be ordered so that foreign key constraints are not violated during synchronization. If a synchronization job attempts to insert a row containing a foreign key value into a referencing table which does not match any value in the referenced table, the job will fail, even if the foreign key constraint is being violated simply because the synchronization job has not yet completed.

7. The synchronization scope has been defined and must be given a name that is unique within the local database. Use the text field to name the scope, and use the radio button to specify whether the selected SQL Azure database is a 1GB or 10GB type. This information can be retrieved from the “Type” property listed for the database in the SQL.Azure.com portal. Click “Process” to complete the provisioning process.

7

Figure 3.1.6: Name the synchronization scope and select the size of the SQL Azure database.

8. Open SQL Server Management Studio and look under SQL Server Agent to find a job named after the synchronization scope has been created. This job can be run at any time to execute synchronization between the local and cloud copies of the selected data.

Figure 3.1.7: The job, which will use Microsoft Sync Framework and SqlAzureSyncProvider during synchronization, is added to the available list in SQL Server Agent and is now ready to be executed.

8

a. In order for the job to run successfully, the credentials used by the SQL Server Agent service must match the credentials of the account running the job.

2. SQL Azure Offline Plug-In for Visual Studio WalkthroughThis walkthrough demonstrates how easy it is to use the new SQL Azure synchronization provider by adding a new item template for Visual Studio named SqlAzureDataSyncClient. When added to a project, SqlAzureDataSyncClient uses the provider assemblies and automatically creates a class and a local data store to streamline the synchronization process.

Note: There is no limit to the number of locally-defined synchronization scopes that can be created using this process, so long as they are given unique names.

1. In Visual Studio 2008 Professional SP1, create or open up a Console Application project. 2. After the project is loaded, right-click the “Solution Explorer” pane, then hover over “Add,”

and click “New Item.”

Figure 3.2.1: Right-click > Add > New Item

3. Scroll down and select “SqlAzureDataSyncClient” under “My Templates.” This will add the synchronization client to the project. Give the client a name, then click “OK” to continue.

9

Figure 3.2.2: Select SqlAzureDataSyncClient, then enter sync client name.

4. A “SQL Azure Data Sync” wizard will appear with the “SQL Azure Connection” form selected. Enter a valid server name, database name, username, and password associated with a SQL.Azure.com account. Click “Next” to proceed.

Figure 3.2.3: Enter valid SQL Azure credentials.

a. SQL Azure Firewall settings can affect connectivity. For more information on how to configure SQL Azure Firewall settings, see http://go.microsoft.com/fwlink/?LinkId=166665.

10

5. In the select box under “Select Tables to Sync”, highlight tables in the database to add them to the synchronization scope. Click “Next” to continue.

Figure 3.2.4: Select tables to add them to the synchronization scope.

a. The “Filter Columns” button is not functional at this time.6. The data that is to be synchronized has been selected, and the configuration is ready to be

saved as a synchronization scope. Give the scope a name by typing it next to the “Sync Scope Name” field. Click “Next” to continue.

11

Figure 3.2.5: Give the synchronization scope a name.

a. To provision the selected SQL Azure database using this scope at a later time, select “Script out to file,” then specify the file location to use to save the script. This will generate a file with a “.SQL” extension that can be run later. The file would need to be executed before calling the Synchronize method described in step eight.

7. In the next screen, specify the file location for the local data store. The local data store will be in SQL Compact 3.5 SP1 format, which has the extension “.SDF”. Click “Next” to download the schema for the specified scope into the specified SDF file, then click “Finish” to close the wizard.

Figure 3.2.6: Specify the location of the local data store.

a. The tables will be empty until synchronization is run for the first time.b. Selecting an existing SDF file will result in the original file being deleted and

replaced.8. References are added to the Sync Framework, Data, XML and SQL Azure Data Sync

assemblies. A generated class named after the scope that was defined in step 4 is also added to the project. This class contains a method named Synchronize which has been pre-configured for use with the selected scope.

12

Figure 3.2.7: The project in Visual Studio after running the wizard.

9. Write code anywhere in the program that calls the Synchronize method. For example, if a new C# project was created in Step 1, the Main()method could be updated as follows: static void Main(string[] args){ SqlAzureSyncClient_YOURSYNCSCOPENAME_Offline.Synchronize();}

10. Press F5 to build the project and launch the program in debug mode. a. Synchronization progress is provided to the output window by Synchronize.

Allow time for the initial synchronization process to complete; subsequent calls to Synchronize will result in much smaller data sets being exchanged.

b. If running a 64-bit version of Windows, the platform target must be updated to specify “x86.” To ensure the platform target is correct, click “Project,” “<PROJECT NAME> Properties” from the main menu in Visual Studio, then in the Properties window for the project, click “Build” on the left navigation bar, then click the drop down next to “Platform target,” and select “x86.”

4. FeedbackThis release was provided in order to gather feedback from our customers. Now that you have previewed what the Sync Framework team is doing with SQL Azure integration, please let us know what you think of our direction, and tell us about your experiences.

You can send us your thoughts in any of the following ways:

Email syncfdbk@microsoft.com. Send us a note on Twitter @syncfx.

13

Post a comment to our blog at http://blogs.msdn.com/sync. Post a thread to our forums at http://social.microsoft.com/Forums/en-US/category/sync.

ConclusionThe code, features and functionality being used and described here are provided for demo purposes only, are still currently in development, and are subject to change. For terms of use information, visit http://www.microsoft.com/info/cpyright.mspx.

For more information:

http://www.microsoft.com/windowsazure/sqlazure/ Microsoft SQL Azure

http://msdn.microsoft.com/sync Microsoft Sync Framework Dev Center

14