Data migration to Drupal using the migrate module

  • View
    2.108

  • Download
    0

Embed Size (px)

DESCRIPTION

Migration is the idea of moving data from A to B (and often, in the middle you transform this data). This presentation focus on the use of Drupal Migrate module. I recommend to use Migrate module for a complex migration. When you have either a complex source or destination. Migrate module is very flexible, powerful and well documented. Summary : - Introduction to data migration - Understanding the migrate module - Organise your Data migration with Migrate - Migrate in Drupal 8 Presentation for Drupalcamp Kyoto, Japan, in April 2014.

Transcript

  • 1.Data Migration with Drupal using Migrate Drupal Data Migration with Drupal using Migrate, Drupalcamp Kyoto. 12th April 2014 - Luc Bzier, @Luukyb Drupalcamp Kyoto, Japan https://www.flickr.com/photos/lifes26/4509331619/

2. Who am I Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 3. Who am I Freelance web, exclusively Drupal projects since ~ 4 years. I moved to Cebu City, Philippines ~ 1 year. Twitter @Luukyb luc@webizat.com Luukyb on drupal.org Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 4. Who am I Contributed to several large data migration to drupal using migrate module. Oracle to drupal : BBC Good Food UK public sector organisation Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 5. http://www.flickr.com/photos/larahsphotography/2795859728/ 1. Introduction to data migration 2. Understanding the migrate module 3. Organise your Data migration with Migrate 4. Migrate in Drupal 8 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 6. Data migration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 7. Data migration Moving data from A to B (and often, in the middle you transform the data.) Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 8. Data migration Simple migration? Feeds module Data mapping using UI. drupal.org/project/feeds Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 9. Data migration Use migrate module for a complex migration. Complex source or destination. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 10. Data migration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/kieranlane/6635502419/ 11. Data migration Using the migrate module can require code, coffee and patience. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/kieranlane/6635502419/ 12. Data migration Good news: Migrate module is very flexible, powerful and well documented. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 13. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 14. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Drupal module Install and enable like any other contrib module drupal.org/project/migrate 15. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Created by Mike Ryan. drupal.org/user/4420 Senior Technical Consultant at Acquia 16. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb framework for migrating content into Drupal from other sources 17. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Whats inside ? - OOP Architecture - An API - Some drush commands - Migrate UI 18. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Built-in support of core items (Nodes, Users, Files, Taxonomy and comments) 19. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Built-in support for PDO, XML, CSV, JSON, and native MSSQL and Oracle API sources 20. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb You can extend all sources, destinations. 21. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Migration: - 100% native drupal - iterative 22. Introduction to Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Related modules: - Drupal-to-Drupal data migration (migrate_d2d) - WordPress Migrate - TYPO3_migrate 23. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 24. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Your migration classes extends the abstract class Migration 25. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Your source. $this->source = new MigrateSourceSQL($query); 26. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Your destination. $this->destination = new MigrateDestinationNode(article'); 27. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Your mapping. $this->addFieldMapping ('field_summary', 'summary'); 28. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Transform your data. $this->addFieldMapping ('field_summary', 'summary) ->callbacks(array($this, summaryFormat')); 29. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Extra transformation and processing can be done using methods like: function prepareRow($row) 30. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Some cool features ! Source dependency definition. $this->addFieldMapping('field_image', imagepath') ->sourceMigration(MediaImages'); 31. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Easy to rollback the data. When you run a migration, migrate module stores the keys. 32. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb If you import into a contributed module: 2 cases. 33. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb First case, my contrib module does not support migrate. 34. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Example: Media module. Migrating content for the media module requires migrate_extras. 35. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Migrate extras module implements extra support. drupal.org/project/migrate_extras 36. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Module maintainers are now recommended to implement migration directly in the modules. 37. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb So second case, my contrib module implements migrate. 38. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Example: field collection. Comes with a file dedicated to migrate: field_collection.migrate.inc 39. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb Use drush ! Limit the number of items to import, Rollback the data, Audit your migration, 40. Understanding Migrate module http://www.flickr.com/photos/zigazou76/9104028602 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb 41. Organise your Data migration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 42. Organise your Data migration Study the source data, keep track of the changes. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 43. Organise your Data migration Use a separate migration environment. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 44. Organise your Data migration Use a script to build this environment from scratch. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bzier, @Luukyb https://www.flickr.com/photos/usace_albuquerque/5547589951/ 45. Organise your Data migration Install your site w