Embed Size (px)
Citation preview
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Getting Your Stuff Into Drupal: Using the Feeds, Migrate, and Migrate DrupaltoDrupal Modules Handout
Objectives
1. Import basic commaseparatedvalues (.csv) data into a Drupal site using the Feeds module.
2. Import basic MySql database tables into a Drupal site using the Migrate module. 3. Migrate Drupal 6 site content into Drupal 7 using the Migrate DrupaltoDrupal module.
Prerequisites Students should have a strong working knowledge of Drupal core, including content types, adding new modules, and using Drush. In addition, students should be familiar with commaseparatedvalues (.csv) files, writing SQL queries, and PHP.
Materials Students should have a new, local Drupal 7 site and have access to a code editor and a MySql database frontend tool (PhpMyAdmin or similar).
Background The basics of building a Drupal site using best practices are well documented and readily available in many workshops, books, and online resources. This is a powerful skill that can be utilized to build various kinds of web sites. But what if an organization already has a web site, fully populated with content? What if content earmarked for a web site is available in a spreadsheet or other flatfile format? How can the process of automating the importing
Page 1 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
(migration) of content into Drupal be accomplished? As with most things Druaply, there are many ways to accomplish this task. In this workshop, we’ll cover three of the most popular methods: the Feeds module (https://drupal.org/project/feeds), the Migrate module (https://drupal.org/project/migrate), and the Migrate DrupaltoDrupal module (https://drupal.org/project/migrate_d2d).
Which Method to Use? Depending on the source data, usually one of these three methods can help automate the process of migrating a virtually unlimited (depending on your hardware) amount of content into your Drupal site. How do you select which one is best?
Data format/scope Method/Module
Not too much content Manual
Simple content in flat file (.xml, .csv, .xls) Feeds module
Complex content in flat files Migrate module
NonDrupal content in a database Migrate module
Content in previous major version of Drupal Migrate DrupaltoDrupal
While this table provides some general guidelines for selecting the best possible method, there are always going to be special cases. For example, if you have a series of .csv files where a record in one .csv file refers to a record in a different .csv file, then it might be best to use the Migrate module instead of the Feeds module.
Organizing a Migration Project Most (successful) migration projects are organized as such:
1. Analyze review the data to be migrated. This should be the very first task: go tablebytable (filebyfile), fieldbyfield through the old data, ensure that every single field is understood. Annotate the data, make note of which fields are to be migrated, which can be ignored and which need modifications during the migration process. Ensure all stakeholders are involved in the process so there is no miscommunication later (when it can be much more timeconsuming to fix).
2. Map using a spreadsheet or similar tool, map each source field to its Drupal counterpart. For example, if the source data is a .csv file of users to migrate, the map might look something like this:
Page 2 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Source field Drupal destination field
xoops_user.uname user.name
xoops_user.email user.mail
xoops_user.user_regdate user.created
At this point, no technical Drupal work has begun this is almost always an exercise in analysis and communication. All project stakeholders should be involved in this process and sign off on the final map. Errors found and fixed at this stage will be quicker to fix here than in the next phase.
3. Iterate set up and perform the migration. This is best done on a development or staging instance of the web site, never on the production instance. Expect to make mistakes, learn new things about the data, and need to rollback and remigrate during this process.
Often, it can be assumed that the following things are true when dealing with source data:
1. The data is never as clean as it is claimed to be. 2. Data will almost always need to be “massaged” during migration. 3. You don’t know half of what you need to know to complete the migration until you’re in the
middle of the migration. In the sections below, we’ll outline how to perform basic migrations using the Feeds, Migrate, and Migrate DrupaltoDrupal modules.
Feeds Module The Feeds module (https://drupal.org/project/feeds version 7.x2.0alpha8 as of the writing of this document) provides a method for importing data from a variety of flat file types (.csv, .rss, .xml, etc…) into Drupal as users, nodes, taxonomy terms, or simple database records. It accepts data via a URL (usually in the case of RSS feeds) or via file upload (.csv or .xml files, commonly). The module can be configured to perform an import just once, or periodically via Drupal’s cron functionality. For simple imports, the entire process can be managed from the Feeds module’s user interface. For more complex imports, custom “tamper” PHP functions (using the Feeds Tamper module https://drupal.org/project/feeds_tamper) can be written to massage incoming data. Feeds is also extensible. Custom “fetchers” (the mechanism that accesses the source data), “parsers” (the mechanism that reads the data), and “processors” (the mechanism that prepares and maps the data to Drupal entities) can be written in PHP. Feeds also has builtin support for the Rules module (https://drupal.org/project/rules) which allows Drupal to react to various Feedsrelated actions.
Page 3 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Feeds Module Example Migration Background and Preparation In this simple example, we’re going to use the Feeds module to migrate a .csv file containing 2,000 gravesite locations for Florida veterans (http://catalog.data.gov/dataset/gravesitelocationsofveteransandbeneficiariesinfloridaasofjuly2013). The original .csv file from data.gov actually has over 325,000 records, but we’re going to use a truncated version of it. To perform this migration, the following setup is necessary:
1. A fresh (“Standard” profile) Drupal 7 site with the following additions: a. The Feeds module (https://drupal.org/project/feeds) downloaded and the Feeds
and Feeds Admin UI modules enabled. b. The Feeds Tamper module (https://drupal.org/project/feeds_tamper) downloaded
and the Feeds Tamper and Feeds Tamper Admin UI modules enabled. 2. The “ngl_florida_5_records.csv” and “ngl_florida_2000_records.csv” source files
downloaded to your local machine (provided by the instructor). Inspecting the first row of the .csv provides us with the source field names. Using this data, a new “Gravesite” content type can created on the Drupal 7 (destination) site. The table below will serve as the map for this migration. For this example not all fields will be migrated.
Source field (from .csv) Source field notes Drupal destination field (“gravesite” content type)
d_first_name text field_first_name
d_mid_name text field_middle_name
d_last_name text title (change label to “last name”), in the future use Automatic Entity Label (https://drupal.org/project/auto_entitylabel) to combine first/middle/last/suffix into this field? Or use the Name (https://drupal.org/project/name) module?
d_suffix text not migrated
d_birth_date mm/dd/yyyy field_birth_date (field type: Date, widget: Text field, only collect year, month, day)
Page 4 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
d_death_date mm/dd/yyyy field_death_date (field type: Date, widget: Text field, only collect year, month, day)
section_id text not migrated
row_num text not migrated
site_num text not migrated
cem_name text (ALL CAPS) field_cemetery_name
cem_addr_one text (ALL CAPS) not migrated
cem_addr_two text (ALL CAPS) not migrated
city text (ALL CAPS) not migrated
state text (ALL CAPS) not migrated
zip text not migrated
cem_url url field_cemetery_url
cem_phone text not migrated
relationship text not migrated
v_first_name text not migrated
v_mid_name text not migrated
v_last_name text not migrated
v_suffix text not migrated
branch text (ALL CAPS) not migrated
rank text (ALL CAPS) not migrated
war text (ALL CAPS) not migrated
Based on the information from the table, a new “Gravesite” content type can be created with the following fields:
1. Default title field change label to “Last name” 2. First name (field type: text) 3. Middle name (field type: text) 4. Birth date (field type: Date, widget: Text field, only collect year, month, day) 5. Death date (field type: Date, widget: Text field, only collect year, month, day)
Page 5 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
6. Cemetery name (field type: text) 7. Cemetery URL (field type: link) 8. The “Gravesite” content type for this example does not need a “Body” field, so we can
delete it.
Importing the .csv using the Feeds module To use the Feeds module to migrate the gravesite data into our new content type, the steps are as follows:
1. Create a new “Feed importer” via Structure | Feed importers | Add new importer (admin/structure/feeds/create). Name it “Gravesite importer”, and provide a reasonable description.
2. The main configuration page for the Feeds importer has four main sections: a. Basic settings this is where information about when the importer will run, and
where (if any) content types it will be attached to. b. Fetcher this defines the mechanism that will fetch the source data. c. Parser this defines the mechanism that will read the data and put it into a format
that the Processor can handle. d. Processor this defines where the source data will end up on the Drupal site,
including the field mappings. 3. For our example, the default values for the Basic settings are fine. The “Attach to content
type” setting can be confusing. It is not the content type our data will eventually be migrated into, rather if you want to provide a method for multiple nodes to define their own source files (of the same format), you can set it here. For example, it may be desired to create a new “State” content type and use a different .csv for each state. In this case, the importer can be attached to the “State” content type, rather than being standalone.
Page 6 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
4. Since we’re going to be uploading our .csv file directly, we need to change the “Fetcher” from “HTTP Fetcher” to “File Upload”. The default settings for the File Upload fetcher are fine for this example.
5. For this example, since our source data is in .csv format, we need to change the Parser to “CSV Parser”. Again, the default settings for the CSV Parser are fine.
6. Finally, since we want to create “Gravesite” nodes from the data in the .csv, we want to use the (default) “Node Processor”.
a. Node Processor Settings: select “Gravesite” as the “Bundle”. The “Update existing nodes” setting is useful when running periodic imports (if your source data is updated fairly often). In this case, we’re doing a onetime migration, so the “Do not update existing nodes” settings is fine (the other settings will increase the processing time). “Skip hash check” is related to “Update existing nodes”, and can be left unchecked for this example. Change the “Author” user to any user on your site who has permission to create Gravesite nodes. Leaving the “Authorize” checkbox selected is usually a good idea, and for this example, “Expire nodes” should be set to “never”.
b. Node Processing Mappings: this is where we will utilize the source/destination map table we created earlier. It is ideal if the source table has a unique (usually an integer) ID field for each record. In this example, one does not exist. When it does exist, it is normally the first field entered in the map, then set as “unique” in the “target configuration” column of the mapping table. In this example, we’ll just go onebyone through the source fields (be sure to enter their names exactly as they appear in the .csv) and map them to our Drupal Gravesite content type fields. In some cases, where the Drupal field has multiple parts (Date and Link fields, for example), be sure to select the proper part to migrate the source data into.
7. Testing our migration it is normally a good idea to create a small (less than 10 records) subset of your .csv file to test the migration with. This way, should (when) you find a mistake, it is fast and easy to rollback and retry. To test the migration, go to /import (or click on the “Import” link in your “Navigation” menu) on your web site and click on the “Gravesite importer”. Click to choose your test .csv, then click “Import”. If all goes well, you’ll see a status message letting you know how many records were imported. Then, navigate over to “Content” (admin/content) a check out your new nodes!
8. If the import didn’t go as planned, delete the imported nodes from the admin/content page (this is why using a very small test .csv is important), make your changes to your importer, and try again.
Feeds Tamper As time permits, we can leverage the Feeds Tamper module (https://drupal.org/project/feeds_tamper) to massage some of our data on its way in to Drupal. For this example, we’re going to use it to change the “Cemetery name” field from ALL CAPS to something a little more reasonable. To do so, first install and enable the Feeds Tamper module (be sure to enable the Feeds Tamper Admin UI module as well). Navigate back to the main Feeds importer page (admin/structure/feeds) and note the new “Tamper” option in the “Operations” column.
Page 7 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Clicking this options will show you a list of mapped fields for your importer, along with a list of any “plugins” (tampers) active, and an option to add a new plugin. Click to add a plugin to the “cem_name > Cemetery name” field. The plugin we’re going to use is “Convert case” (in the “Text” section). In the settings for “Convert case”, set “How to convert case” to “Title Case”. Now, we’re ready to rerun the migration. Be sure to rollback (manually delete) the previously imported Gravesite nodes prior to rerunning the import.
Additional Tips 1. Use the Admin Views (https://drupal.org/project/admin_views) module to gain a
muchimproved default admin/content page that allows you to delete more than just the nodes you can see on the first page. This can be immensely helpful if you need to rollback a large amount of content.
Migrate Module The Migrate module (https://drupal.org/project/migrate version 7.x2.5 as of the writing of this document) is a robust Drupal module designed to provide an extensible method for migrating content of virtually any type and format into Drupal. The Migrate module utilizes custom PHPbased migration classes that allow developers to control virtually every aspect of the migration process. Two of the “killer features” of the Migrate module are
1. The ability to rollback a previously run migration. 2. The ability to run continuous migrations that can automatically update content on the
destination as it is updated/added on the source. A powerful feature of the Migrate module is the fact that the source site can remain virtually untouched during the migration process. This allows the source site to continue functioning normally while the content is migrated and the new Drupal site is built. The ability for the migrate module to periodically migrate new and modified content from the source to the destination (depending on the source configuration) allows the source and destination site to remain insync (as far as content is concerned) during the entire process. This is called “continuous migration”.
Page 8 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
The Migrate module requires knowledge of PHP. The user interface is strictly for running and monitoring migrations. There are plans to make more aspects of the module available in the user interface, but for the time being, you must be comfortable with PHP (and some objectoriented programming) in order to develop migrations with the Migrate module. A typical process for creating a Drupal migration using the Migrate module is as follows:
1. Analyze ensure that you understand every aspect of the data to be migrated, this is crucial in ensuring that data is mapped and transformed properly between the source and the destination.
2. Map create custom migration classes based on the data to be migrated. This process defines the source, destination, migration map, field mappings, and field data processing. Normally, most of this work is done in PHP code.
3. Review using the Migrate module UI, review with stakeholders the migration field mappings, scope of the migration (number of records to be migrated), as well as any unmapped source and destination fields to ensure everything is accounted for.
4. Iterate run the migration class on a development server and review results. As issues are found, rollback the migration, make necessary changes to the migration class and reimport.
The next two sections will provide some details on Migration module classes and the Migrate module user interface (UI).
Migration Classes Most migration projects utilizing the Migrate module begin by creating the necessary migration classes. While this section will document the main areas of a typical migration class, the best documentation for migration classes are the comments and code in actual migration classes. The Migrate module provides excellent sample migration classes, as does this course’s sample code. Typically, all Migrate module migration classes have (at least) the following four components:
1. The migration source this dictates where the incoming data is coming from. The Migrate module provides several builtin source classes, including MigrateSourceSQL, MigrateSourceCSV, MigrateSourceXML, and MigrateSourceMSSQL.
2. The migration destination normally, this is a Drupal entity. Examples include MigrateDestinationUser, MigrateDestinationFile, MigrateDestinationTerm and MigrateDestinationNode.
3. The migration map this defines the relationship between source content and destination content and is used to keep track of migration progress. If a migration needs to be rolledback, the migration map is used to know exactly which content to remove from the destination site. As an example of how the migration map works, assume that 3 user accounts were migrated from the source to the destination. In the source, the unique user IDs were 1, 2, and 3. But, after migration, the same accounts had user IDs of 6, 7,
Page 9 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
and 8 on the destination. The migration map keeps track of the fact that source ID 1 became destination ID 6, source ID 2 because destination ID 7, and source ID 3 became destination ID 8. It is because of this map that continuous migration and rollbacks are possible. In each individual migration class, information about which fields in the source and destination can be used as the source and destination IDs are defined.
4. Field mappings for each field to be migrated from the source to the destination, a mapping must be defined. For example, when the source field containing the email address is named “email”, and the destination field is named, “email_address”, the field mapping defines that source data from the “email” field should end up in the destination “email_address” field. This example is a relatively simple case. Other field mappings could include processing the incoming data prior to it being saved to the destination. Examples of processing functions include:
a. setting a default value if the source value is not defined b. deduplicating source data if values must be unique on the destination (for
example, usernames in Drupal) c. specifying a source migration if the value is related to data previously migrated
(common when dealing with relational data) Some individual migration classes may also need to include custom functions to help massage data during the migration. Examples include:
Converting content from one markup language to another Removing potentially hazardous markup Rewriting URL paths Adding redirect paths
Multiple migration classes can also be grouped, and then run by group. For example, all userrelated migrations (user accounts, avatars, profiles, etc…) can be placed in a single “user” group, then the entire group can be run as a single migration. The official Migrate module documentation can be found at: https://drupal.org/migrate. Several pages are particularly useful:
Commonly implemented migration methods: https://drupal.org/node/1132582 Typical migrate commands using Drush: https://drupal.org/node/1561820
Migrate Module User Interface The Migrate module’s user interface can be found (by default) in Drupal’s management menu in the “Content” area (admin/content/migrate). The main page of the Migrate module interface lists all of the registered migration classes, along with data about the status of each class. Data for each migration includes the current migration status (“Idle”, “Importing”, “Stopping”, “Rolling back”, “Disabled”, or “Unknown”), the total number of source rows, the number of rows imported and unimported, the number of messages
Page 10 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
(normally related to issues reported by the migration module during migration), the speed (throughput) of the migration, and finally the timestamp of last time the migration was run. The main Migrate module user interface page also provides a mechanism to import, stop, rollback, and reset migrations. It is recommended that these mechanisms only be used for small, or test, migrations, and that drush commands provided by the Migrate module be used for the majority of migrations (see the “Running the migrations” sections of the two examples later in this document for more information). A migration is “registered” when it is declared in a custom module’s hook_migrate_api() implementation (there is also a second way of registering migrations using the MigrationBase::registerMigration() method, but it is deprecated, so it will not be addressed here).
Submenu items from the main Migrate module user interface page include:
1. Migrate this is the default submenu item that leads to the main Migrate module user interface page.
2. Registration this provides a manual method to request that the Migrate module register any unregistered migration classes. Assuming that migration classes are registered using an implementation of hook_migrate_api(), the default values for this page are normally fine. More information about migration class registration is available in the Migrate module documentation at https://drupal.org/node/1824884.
3. Handlers in some (rare) instances, multiple handlers for sources, destinations, and fields are declared that can conflict with one another. This pages provides a mechanism for disabling handlers on a casebycase basis. From the Migrate module documentation (https://drupal.org/node/1006990): “Handlers are classes which enable you to add additional behavior to the processing of data being imported. These are usually used to support migration into contributed or custom modules that maintain their own data connected to core entities. Most of the time this will be related to custom field types.”
Page 11 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Each custom migration class provides a summary of its configuration that can be accessed by clicking on it’s class name in the main Migrate module UI. The information in this summary comes from the custom migration class itself, and can be used by migration stakeholders to review the settings of the migration. The summary has a minimum of 4 sections, each one a direct reflection of a portion of the migration class.
1. Overview this section provides basic information about the migration as defined in the migration class including:
a. Team information this is provided by creating new MigrateTeamMember classes. For example: $this>team = array( new MigrateTeamMember('Mike Anello', ‘[email protected]', t('Migrator')), );
b. Dependencies as defined in the “dependencies” or “softDependencies” properties of the migration class. Dependencies are class names of other migrations that the current migration is dependent on. For example, it is typical for a migration that creates nodes to be dependent on a migration that creates users, as often users are utilized as node authors. Standard (“hard”) dependencies prevent a migration from running before the dependent migration has run successfully. A soft dependency ensures that the dependent migration has run first, but will still allow the migration to proceed even if errors occurred in the dependent migration.
c. System of record by default, this is set to “source data”, meaning that data from the source will always take precedence over data on the destination.
d. Description as defined in the “description” property of the migration class. 2. Destination provides a list of all destination fields available to the migration based on the
“destination” property of the migration class. For example, if the destination property of the migration class is the “article” content type, then all properties and fields associated with the “article” content type will be listed.
3. Source provides a list of all fields provided by the source data. This is normally based on the source query (if the source is a database), but can also be based on .csv and .xml fields, depending on the source type.
4. Mapping: (category) there can be multiple “Mapping: (category)“ sections, each one defined by an “issueGroup” property of a field migration. Each one lists all defined field mappings for the particular “issueGroup”, including the source and destination fields, default values, descriptions, and links to external project trackers. The Migrate module automatically creates a “Mapping: Done” issue group for defined field mappings, and a “Mapping: DNM” (“do not migrate”) for unmapped fields, providing that migrators add unmapped source and destination fields to the “addUnmigratedSources” and “addUnmigratedDestinations” properties of their migration classes. Together, the
Page 12 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
“Mapping: (category)“ summaries are extremely valuable for all migration stakeholders as an overview of the status of each migration class. Source and destination fields not included in a “Mapping: (category)” group will be highlighted in the user interface.
As an example of a “Mapping: (category)” summary, considering the following field mapping: $this>addFieldMapping('timezone') >defaultValue('America/New_York') >description(t('Can we translate a timezones like "5" to a D7 timezone string?')) >issueNumber(1234) >issueGroup(t('Tough stuff')); In this case, the destination field (“timezone”) is being set to a default value (“America/New_York”) despite the fact that a source timezone field exists (but is in a different format). Rather than ignoring this, a relevant description, issueNumber (linked to an external issue tracker assuming the “issuePattern” migration class property is set to the URL of the external issue tracker), and issueGroup are added to the field mapping resulting in the following display in the Migrate UI:
Note how all the relevant information from the field mapping class is summarized in the user interface. This is valuable because it allows the migrator to add code to the migration class that is directly available to other migration stakeholders (who may or may not be comfortable with PHP code).
Page 13 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Migrate Example Based on the JeepNewsNow.com migration from XOOPS 2.x to Drupal 7. In this example, we’ll take a database from the XOOPS content management system (http://xoops.org/) and migrate its users, content categories, stories, and story comments into Drupal. The example is based on the JeepNewsNow.com web site; the sample data provided will be a sanitized subset of the actual data.
JeepNewsNow Example Migration Background and Preparation To perform this migration, the following setup is necessary:
3. A fresh (“Standard” profile) Drupal 7 site with the following additions: a. The Migrate module downloaded and the Migrate and Migrate UI modules enabled. b. The BBCode (https://drupal.org/project/bbcode) and Redirect
(https://drupal.org/project/redirect) modules enabled. i. The BBCode module is required because the legacy database allowed for
content to be entered using the BBCode markup format. During the migration, we’ll utilize the BBCode module to “unBBCode” the content.
ii. The Redirect module is required so that we can retain URL paths for articles (stories) between the old and new sites.
c. The custom “JeepNewsNow Migration” module (provided by the instructor) enabled (enable any dependencies as well).
d. A new vocabulary named “Category” (machine name: category) with a new “Category image” (field_category_image) field (path: images/category).
e. A new term reference field on the “Article” content type referring back to the new “Category” vocabulary (machine name: field_category, singlevalued, select list)
4. Create a new, empty “jeepnewsnow_xoops_subset” database on your local MySql server and import the “jeepnewsnow_legacy_db.mysql” file into it. This is the source database that we’ll be migrating from. The custom “DrupalEasy Migration” module assumes that the database name is “jeepnewsnow_xoops_subset”, it is accessible at “localhost”, and that the user “root” (password: “root”) has full permissions. If your system differs, update the database connection information in jnn_migrate_user.inc.
5. (optional) Copy the (provided) legacy images into sites/default/files/legacy. These are images associated with categories, users, and articles (stories). While their path doesn’t have to be inside the Drupal site, we do so here to keep things neat (the path to this legacy directory is hardcoded as part of the migration).
At this point, the destination site should be ready for the migration.
Page 14 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
JeepNewsNow Migration module code walkthrough The Migrate module relies on custom migration classes written by the migrator as part of a custom module. For this example, the “JeepNewsNow Migration” custom module (“jnn_migrate”) contains the necessary migration classes in order to migrate users, user avatars, categories, stories, and comments from the XOOPS database into Drupal. Rather than displaying the entire custom module code in this document, only a highlevel overview will be provided (the jnn_migrate module is welldocumented).
jnn_migrate.info This is a typical Drupal custom module .info file. A list of module dependencies are included to ensure that the destination Drupal site has all the required modules enabled in order to accept all of the incoming data. The jnn_migrate.info file also contains references to the other required files in the module.
jnn_migrate.module This file is empty, but since all Drupal modules must contain a .module file, we must include it.
jnn_migrate.install For this migration, we need a .install file to ensure that user signatures are enabled on the destination Drupal site. This is because, as part of the migration, we’re importing user signatures from the source site.
jnn_migrate.migrate.inc This file contains a single hook implementation that for hook_migrate_api(). When, implemented, this hook informs the Migrate module of the Migrate API version being used and all migration classes defined by the JeepNewsNow Migration module.
jnn_migrate_user.inc This file contains the implementation of the JnnMigration class as well as userrelated JeepNewsNow migration classes. The JnnMigration class extends the Migrate module’s DynamicMigration class. While individual migrations can subclass DynamicMigration directly, by subclassing it this way, we can store common JeepNewsNowspecific information in the JnnMigration class that all individual migrations can access. In this example, the source database connection information, the list of migrators working on the project, and the “issuePattern” are part of the JnnMigration class. The JnnUserMigration class handles the migration of user account information. It is dependent on the JnnAvatarMigration class. The basic parts of the migration class are:
Page 15 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
1. Source: a custom query on the XOOPS database for the user records using the MigrationSourceSql class.
2. Destination: MigrateDestinationUser 3. Map: the XOOPS user table has a user ID field that will be utilized. The
MigrationDestinationUser destination has knowledge of its ID field builtin. 4. Field mappings: 9 fields will be migrated, we will ignore an issue with the timezone field
migration. The JnnAvatarMigration class must be run prior to the JnnUserMigration. This is because each source avatar image file must be migrated using the MigrateDestinationFile class because Drupal only stores a file ID reference to the user profile picture in the users table. The basic parts of this migration class are:
1. Source: a custom query on the XOOPS database for the user avatar records using the MigrationSourceSql class.
2. Destination: MigrateDestinationFile 3. Map: the XOOPS user table has a user ID field that will be utilized. The
MigrationDestinationFile destination has knowledge of its ID field builtin. 4. Field mappings: 6 fields will be migrated, but only 2 will contain data coming from the
source database. The other 4 fields will define the properties related to the source and destination of the actual image files.
For this migration, the source files will come from the sites/default/files/legacy/ directory that we previous moved to the destination site.
jnn_migrate_node.inc This file contains the contentrelated JeepNewsNow migration classes: categories, stories, and comments. The JnnTopicMigration class handles the migration of article categories (called “topics” in XOOPS). The basic parts of the migration class are:
1. Source: a custom query on the XOOPS database for the topic records using the MigrationSourceSql class.
2. Destination: MigrateDestinationTerm when instantiating this class, we pass the vocabulary we want the terms migrated into.
3. Map: the XOOPS topics table has an ID field that will be utilized. The MigrationDestinationTerm destination has knowledge of its ID field builtin.
4. Field mappings: 8 fields will be migrated, but many of these are related to an image associated with each topic/category. This migration class demonstrates an alternate way of migrating images without having a dedicated migration class (as compared to the JnnAvatarMigration class).
The JnnArticleMigration class handles the migration of articles (called “stories” in XOOPS). The basic parts of the migration class are:
Page 16 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
1. Source: a custom query on the XOOPS database for the story records using the MigrationSourceSql class.
2. Destination: MigrateDestinationNode when instantiating this class, we pass the content type we want the stories migrated into.
3. Map: the XOOPS stories table has an ID field that will be utilized. The MigrationDestinationNode destination has knowledge of its ID field builtin.
4. Field mappings: 12 fields will be migrated, most are simple mappings. The user ID (author ID) and topic/category field mappings are interesting because they utilize other migrations via the “sourceMigration” function. This utilizes each migration’s map to ensure that the proper ID values are used. For example, assume that in the source database, there is a user with an ID of 10. After migration into Drupal, the same user’s Drupal user ID is 20. As we learned earlier, the user migration map table keeps track of this. Assume that in the source database, this same user authored a story. In the source story table, that story has an “author ID” of 10. If were were to migrate this story data into Drupal directly, this story would have the incorrect author ID. By leveraging the “sourceMigration” function, the Migrate module automatically looks up the Drupal (destination) user ID of source user ID 10 and uses that value (20) to set the author of the story on the destination (Drupal). The sourceMigration function can be used to keep data insync whenever one migration class depends on another migration class.
5. prepareRow() function this gives the migration class access to the incoming data so that it can massage it prior to it being migrated into Drupal. For this particular migration class, it does the following:
Sets the published status to either 0 (not published) or 1 (published) depending on the source data. The source “published” field uses a timestamp that must be converted into a 0 or 1 on the destination.
Converts story body content from BBCode markup into HTML markup (utilizing a function from the BBCode Drupal module).
Updates image “src” property values in the body content to utilize the new image paths (sites/default/files/legacy/).
6. complete() function after each row is migrated, the complete function is called. For this migration class, it is used to create a Drupal redirect from the legacy URL (of the form modules/news/article.php?id=[story_id]).
The JnnCommentMigration class handles the migration of story comments. The basic parts of the migration class are:
1. Source: a custom query on the XOOPS database for the comment records using the MigrationSourceSql class.
2. Destination: MigrateDestinationComment when instantiating this class, we pass the content type whose comments we want the source comments migrated into.
3. Map: the XOOPS stories table has an ID field that will be utilized. The MigrationDestinationComment destination has knowledge of its ID field builtin.
4. Field mappings: 13 fields will be migrated, including 3 involving the sourceMigration function: the comment author ID, the comment article ID, and the comment parent ID
Page 17 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
(each comment can have another comment as its parent). 5. prepareRow() function this gives the migration class access to the incoming data so
that it can massage it prior to it being migrated into Drupal. For this migration class, it does the following:
XOOPS allows comment subjects up to 255 characters; Drupal comment subjects are limited to 64 characters, so we perform a simple truncate.
There are several comments that we know are orphaned (their story no longer exists), so we skip migrating them by returning “FALSE” for the row.
Convert comment content from BBCode markup into HTML markup (utilizing a function from the BBCode Drupal module).
6. prepare() function this gives the migration class access to the the migrated data immediately prior to it being saved to Drupal. All fieldlevel processing has been completed and prepareRow() has been called. For this migration class, in the case where a comment doesn’t have a created or modified date, we fake them a bit and set them to the created and/or changed date for their associated article node.
Running the migration While it is possible to run the migration from the user interface, it is common to run the migrations using Drush. Since the migrations are dependent on one another, they must be run in a particular order. For example, the JnnUser migration must be run before the JnnComment migration since each comment is associated with a user. Below is the results of this migration using two Migrate drush commands and options: migratestatus (alias: ms) and migrateimport (alias: mi). $ drush migratestatus Group: jnn_nodes Total Imported Unimported Status Last imported JnnArticle 1992 0 1992 Idle Group: jnn_users Total Imported Unimported Status Last imported JnnAvatar 48 0 48 Idle JnnUser 30239 0 30239 Idle Group: jnn_comments Total Imported Unimported Status Last imported JnnComment 13005 0 13005 Idle Group: jnn_taxonomy Total Imported Unimported Status Last imported JnnTopic 11 0 11 Idle $ drush migrateimport JnnTopic Processed 11 (11 created, 0 updated, 0 failed, 0 ignored) in 0.4 sec (1656/min) done with 'JnnTopic' [completed]
Page 18 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
$ drush mi group="jnn_users" feedback="100 items" Processed 24 (24 created, 0 updated, 0 failed, 0 ignored) in 0.3 sec (5141/min) done with 'JnnAvatar' [completed] Processed 100 (100 created, 0 updated, 0 failed, 0 ignored) in 3.4 sec (1759/min) continuing with 'JnnUser' [status] Processed 100 (100 created, 0 updated, 0 failed, 0 ignored) in 3.4 sec (1757/min) continuing with 'JnnUser' [status] Processed 100 (100 created, 0 updated, 0 failed, 0 ignored) in 3.6 sec (1660/min) continuing with 'JnnUser' [status] Processed 100 (100 created, 0 updated, 0 failed, 0 ignored) in 3.8 sec (1575/min) continuing with 'JnnUser' [status] Processed 50 (50 created, 0 updated, 0 failed, 0 ignored) in 2.4 sec (1250/min) done with 'JnnUser' [completed] $ drush mi JnnArticle feedback="500 items" Processed 500 (500 created, 0 updated, 0 failed, 0 ignored) in 16.4 sec (1831/min) continuing with 'JnnArticle' [status] Processed 500 (500 created, 0 updated, 0 failed, 0 ignored) in 17.1 sec (1751/min) continuing with 'JnnArticle' [status] Processed 500 (500 created, 0 updated, 0 failed, 0 ignored) in 15.3 sec (1960/min) continuing with 'JnnArticle' [status] Processed 492 (492 created, 0 updated, 0 failed, 0 ignored) in 17.9 sec (1651/min) done with 'JnnArticle' $ drush mi JnnComment feedback="5000 items" Processed 5000 (5000 created, 0 updated, 0 failed, 0 ignored) in 96.1 sec (3122/min) continuing with 'JnnComment' [status] Processed 5004 (5000 created, 0 updated, 0 failed, 4 ignored) in 121.9 sec (2460/min) continuing with 'JnnComment' [status] Processed 3001 (3000 created, 0 updated, 0 failed, 1 ignored) in 63.7 sec (2824/min) done with 'JnnComment' $ drush ms Group: jnn_nodes Total Imported Unimported Status Last imported JnnArticle 1992 1992 0 Idle 20140408 15:31:33 Group: jnn_users Total Imported Unimported Status Last imported
Page 19 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
JnnAvatar 24 24 0 Idle 20140407 16:33:34 JnnUser 450 450 0 Idle 20140407 16:33:49 Group: jnn_comments Total Imported Unimported Status Last imported JnnComment 13005 13000 0 Idle 20140408 15:37:15 Group: jnn_taxonomy Total Imported Unimported Status Last imported JnnTopic 11 11 0 Idle 20140408 15:29:58 During the course of testing a migration, there are a number of other useful drush migration commands and options that can be utilized:
drush migrationrollback [migration name] this rolls back any previously migrated data for a given migration (alias: mr).
The limit option can be added to any migrateimport command to only migrate a particular number of records. For example: drush migrateimport limit=”100 items” [migration name].
The idlist option can be added to any migrateimport command to only migrate particular source records (by source ID). For example: drush migrateimport idlist=1,2,4,6.
The full list of drush (version 6.x) Migrate module commands can be found at http://www.drushcommands.com/drush6x/migrate.
DrupaltoDrupal Data Migration Example Based on the DrupalEasy.com migration from Drupal 6 to Drupal 7. The DrupaltoDrupal Data Migration module (“migrate_d2d”) is a framework, built on top of the Migrate module, designed to assist in migrating content from Drupal 5.x and 6.x sites into Drupal 7.x. The module extends the Migrate module classes with Drupalspecific source, destination, map, and field mapping information, thus making it much easier to create DrupaltoDrupal migration classes. Note: as of the writing of this document, the current recommended release of the DrupaltoDrupal Migration module is 7.x2.0, released November 26, 2012. The dev version of this module has more features and requires the dev version of the Migrate module. While the dev versions have more features (enhanced UI, support for the “Upload” module, etc…), they are not yet considered stable. DrupaltoDrupal migrations are commonly used instead of Drupal core’s standard upgrade path between major releases. Much like the case with nonDrupal to Drupal migrations, this allows the source site to remain untouched, while the new Drupal 7 site can use continuous migration
Page 20 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
to keep its content uptodate while the rest of the site is rebuilt. This is sometimes called “migrateandrebuild”. When doing a DrupaltoDrupal migration, structural information about both the source and destination sites is already known. Therefore, the migrate_d2d module encapsulates much of the source, destination, map, and fieldmapping information normally added in a custom module by the migrator in migrate_d2d classes that extend Migrate module classes. For example, the DrupalNode6Migration (part of migrate_d2d) extends DrupalNodeMigration (also part of migrate_d2d) which extends DrupalMigration (also part of migrate_d2d) which extends DynamicMigration (part of Migrate). These DynamicMigration subclasses (and many others in migrate_d2d) do much of the heavylifting of DrupaltoDrupal migrations, leaving only sitespecific customizations up to the migrator. Looking at the code of the DrupalNode6Migration class’s constructor, it can be seen that this class specifies the source, destination, and map information, as well as the basic field mappings (title, body, and other common node metadata): public function __construct(array $arguments) parent::__construct($arguments); $query = $this>query(); $this>sourceOptions['fix_field_names'] = $this>fixFieldNames; $this>source = new MigrateDrupal6SourceSQL($query, $this>sourceFields, NULL, $this>sourceOptions); $this>addFieldMapping('language', 'language') >defaultValue($this>defaultLanguage); $this>addFieldMapping('body:summary', 'teaser'); $this>addFieldMapping('body:format', 'format') >callbacks(array($this, 'mapFormat')); $this>addFieldMapping(NULL, 'moderate'); /** @todo Prevent stub creation when tnid == 0 $this>addFieldMapping('tnid', 'tnid', FALSE) >sourceMigration($arguments['machine_name']); */ $this>addFieldMapping('tnid', NULL); $this>addFieldMapping(NULL, 'tnid'); $this>addFieldMapping('translate', 'translate');
Page 21 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
The fact that all of this is encapsulated in classes provided by the DrupaltoDrupal Migration module makes it much easier to create sitespecific migration classes. While a custom module is still required for DrupaltoDrupal migrations, the amount of code is drastically reduced in many instances. For example, consider a content type called “Quicktip” that contains one custom field a term reference field. The entire migration class that needs to be written is: class DrupalEasyQuicktip extends DrupalNode6Migration public function __construct(array $arguments) parent::__construct($arguments); $this>addFieldMapping('field_tags', '3') >sourceMigration('DrupalEasyTags') >arguments(array('source_type' => 'tid')) >description(t('Simple mapping of vocabulary')) >issueGroup(t('DONE')); The class simply extends the Drupal6NodeMigration, calls the base constructor (which takes care of the source, destination, map, and common field mappings), and then adds the field mapping for the single custom field. Content types that don’t have custom fields (or any other types of customizations) are even simpler, in that they don’t even need a custom migration class they can use a migrate_d2d class (such as “DrupalNode6Migration”) directly. For example, here are the arguments for this example’s “Basic page” node migration, as found in the de_migration_migration_api() function (in de_migration/de_migration.migrate.inc): array( 'class_name' => 'DrupalNode6Migration', // Since we don't need to do anything special with Basic Pages, we can just use the default class. 'description' => t('Migration of page nodes from Drupal 6'), 'machine_name' => 'DrupalEasyBasicPage', 'source_type' => 'page', 'destination_type' => 'page', ), Similar to custom migration classes that extend the DynamicMigration class, hook_migrate_api() must be implemented to register the various DrupaltoDrupal migration classes. Since the migrate_d2d classes already extend the DynamicMigration class, information common to all
Page 22 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
migrations, as well as information specific to each migration can be passed in to each migration class via a set of arguments defined in hook_migrate_api(). This makes the hook_migrate_api() code a bit more complex than a typical nonDrupaltoDrupal migration, but it also allows the actual sitespecific custom migration classes to usually be much simpler. An example, including common information in hook_migrate_api(), can be found in the DrupaltoDrupal Migration documentation: https://drupal.org/node/1813498. Finally, it is important to note that all of the same extensibility available to standard Migrate module subclasses is also available to DrupaltoDrupal Migration subclasses, including the prepareRow(), prepare(), and complete() functions. DrupalEasy.com (Drupal 6.x) Example Migration Background and Preparation In this example, we’ll take a subset of the database from the Drupal 6.x version of DrupalEasy.com and migrate its roles, users, tags, quicktips, basic pages, podcast stories, and associated comments into Drupal 7.x The sample data provided is a sanitized subset of the actual site data. To perform this migration, the following setup is necessary:
1. A fresh (“Standard” profile) Drupal 7 site with the following additions: a. The Migrate module downloaded and the Migrate and Migrate UI modules enabled. b. The DrupaltoDrupal Migration module downloaded and enabled. c. The Link, Views, and Chaos Tools modules downloaded and enabled. d. The custom “DrupalEasy Migration” (de_migration) module (provided by the
instructor) enabled. e. Some nodes on the source site use the “Markdown filter” module
(https://drupal.org/project/markdown) to properly render textareas, via a custom “Full HTML Markdown” input (text) format. In order for these nodes to be migrated with the proper text format, install and enable the “Markdown filter” module and create a new text format as follows: i. Name: Full HTML Markdown (machine name: full_html_markdown) ii. Roles: Administrator iii. Enabled filters: Markdown, Convert line breaks into HTML, Convert URLs
into links, Convert line breaks into HTML, Correct faulty and chopped off HTML (be sure the filters are applied in this order).
f. A new “Quicktip” content type (anything not explicitly specified can be left at default values): i. Add “Short summary” (field_short_summary) text field. ii. Add (existing) “Tags” (field_tags) term reference field.
g. A new “Podcast” content type (anything not explicitly specified can be left at default values):
Page 23 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
i. Add (existing) “Tags” (field_tags) term reference field. ii. Add “Audio file URL” (field_audio_file_url) text field, number of values: 2. iii. Add (existing) “Short summary” (field_short_summary) text field.
2. Create a new, empty “drupaleasy6_subset” database on your local MySql server and import the “drupaleasy6_subset.mysql” file into it. This is the source database that we’ll be migrating from. The custom “DrupalEasy Migration” module assumes that the database name is “drupaleasy6_subset”, it is accessible at “localhost”, and that the user “root” (password: “root”) has full permissions. If your system differs, update the database connection information in de_migration.migrate.inc.
Note that for this migration, no file attachments will be migrated. In the Drupal 6.x version of the source site, the Upload module was used to attach images and other related files to content, but the 7.x2.0 version of DrupaltoDrupal Migration does not yet include support for the Upload module (it is included in the latest dev release). At this point, the destination site should be ready for the migration.
DrupalEasy Migration module code walkthrough The DrupaltoDrupal Migration module relies on custom migration classes written by the migrator as part of a custom module. For this example, the “DrupalEasy Migration” custom module (“de_migration”) contains the necessary migration classes in order to migrate roles, users, tags, nodes and comments from three content types from the Drupal 6.x version of DrupalEasy.com into Drupal 7.x. Rather than displaying the entire custom module code in this document, only a highlevel overview will be provided (the de_migration module is welldocumented).
de_migrate.info This is a typical Drupal custom module .info file. A list of module dependencies are included to ensure that the destination Drupal site has all the required modules in order to accept the incoming data. The de_migrate.info file also contains references to the other required files in the module.
de_migrate.module This file is empty, but since all Drupal modules must contain a .module file, we must include it.
de_migrate.install For this migration, we need a .install file to ensure that user signatures are enabled on the destination Drupal site. This is because, as part of the migration, we’re importing user signatures from the source site.
de_migrate.migrate.inc
Page 24 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
This file contains a single hook implementation that for hook_migrate_api(). When, implemented, this hook informs the Migrate module of the Migrate API version being used and all migration classes defined by the DrupalEasy Migration module. Unlike the hook_migrate_api() implementation in the JeepNewsNow Migration example, this implementation contains much more information about each migration class to be instantiated, including source database connection information, source and destination bundle information, migration class dependencies, and more.
de_migrate_user.inc This file contains the implementation of the DrupalEasyRole and DrupalEasyUser migration classes. These classes extend the DrupalRole6Migration and DrupalUser6Migration classes provided by the migrate_d2d module, allowing us to only add information custom to our migration. For example, the entire DrupalEasyUser migration class is: class DrupalEasyUser extends DrupalUser6Migration public function __construct(array $arguments) parent::__construct($arguments); protected function query() // Get the default query and modify it $query = parent::query(); $query>condition('u.name', array('ryanprice'), 'NOT IN'); // Nothing against ryan, but he was UID=1. He'll get a new account. return $query; The class takes advantage of everything that the DrupalUser6Migration class provides (source, destination, map, and field mappings), while only slightly modifying the source query to ensure that UID=1 doesn’t get migrated to the new site.
de_migrate_node.inc This file contains the implementation of the DrupalEasyQuicktip and DrupalEasyPodcast classes each of these extend the DrupalNode6Migration class. Again, by extending existing migrate_d2d classes that already encapsulate information about the source, destination, map, and basic field mappings, these migration classes are quite straightforward. There are no custom migration classes for Basic Page nodes or Quicktip and Podcast comments because these do not have any sitespecific customizations, so we can use the migrate_d2d classes directly.
Page 25 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Running the migration While it is possible to run the migration from the user interface, it is common to run migrations using Drush. Since the migrations are dependent on one another, they must be run in a particular order. For example, the DrupalEasyUser migration must be run before the DrupalEasyQuicktipComment migration since each comment is associated with a user. Below is the results of this migration using two Migrate drush commands and options: migratestatus (alias: ms) and migrateimport (alias: mi). $ drush ms Group: drupaleasy_nodes Total Imported Unimported Status Last imported DrupalEasyBasicPage 23 0 23 Idle DrupalEasyPodcast 107 0 107 Idle DrupalEasyQuicktip 108 0 108 Idle Group: drupaleasy_comments Total Imported Unimported Status Last imported DrupalEasyPodcastComment 312 0 312 Idle DrupalEasyQuicktipComment 92 0 92 Idle Group: drupaleasy_users Total Imported Unimported Status Last imported DrupalEasyRole 3 0 3 Idle DrupalEasyUser 263 0 263 Idle Group: drupaleasy_taxonomy Total Imported Unimported Status Last imported DrupalEasyTags 223 0 223 Idle $ drush mi group=drupaleasy_users Processed 3 (3 created, 0 updated, 0 failed, 0 ignored) in 0.1 sec [completed] (3517/min) done with 'DrupalEasyRole' Processed 263 (263 created, 0 updated, 0 failed, 0 ignored) in 3.6 [completed] sec (4369/min) done with 'DrupalEasyUser' $ drush mi DrupalEasyTags Processed 223 (223 created, 0 updated, 0 failed, 0 ignored) in 3.4 [completed] sec (3916/min) done with 'DrupalEasyTags' $ drush mi group=drupaleasy_nodes Processed 23 (23 created, 0 updated, 0 failed, 0 ignored) in 0.8 sec [completed] (1753/min) done with 'DrupalEasyBasicPage' Processed 107 (107 created, 0 updated, 0 failed, 0 ignored) in 3.1 [completed] sec (2084/min) done with 'DrupalEasyPodcast' Processed 108 (108 created, 0 updated, 0 failed, 0 ignored) in 3.1 [completed] sec (2091/min) done with 'DrupalEasyQuicktip' $ drush mi group=drupaleasy_comments
Page 26 of 27 Copyright DrupalEasy Academy 20112014
Drupal Career Starter Program Getting Your Stuff Into Drupal Using Feeds and Migrate
Processed 312 (312 created, 0 updated, 0 failed, 0 ignored) in 4.5 [completed] sec (4127/min) done with 'DrupalEasyPodcastComment' Processed 92 (92 created, 0 updated, 0 failed, 0 ignored) in 1.3 sec [completed] (4118/min) done with 'DrupalEasyQuicktipComment' When developing migration classes, it is normal to write some code, test the migration (using “drush mi”), rollback the migrate (using “drush mr”), modify the migration class, then test the migration again. While the user interface for the Migrate module is useful (found in Content|Migrate admin/content/migrate), the drush interface has the advantage of not being subject to the web interface’s PHP timeout setting. If a large migration is being run, it is more than likely that if it is being run through the web interface, that it will timeout before it is complete. By running the migration via drush, the main Migrate UI page (admin/content/migrate) can act as a “dashboard” for the migrator to keep tabs on the progress of the migration.
Page 27 of 27 Copyright DrupalEasy Academy 20112014
