Lean Drupal Repositories with Composer and Drush

  • Published on
    13-Apr-2017

  • View
    816

  • Download
    1

Embed Size (px)

Transcript

<ul><li><p>Lean Drupal Repositories with </p><p> Composer and Drush </p><p> by Greg Anderson</p><p>Photo by Adam Wyles2016</p><p>https://www.flickr.com/photos/the-travelling-bum/5905853321/https://www.flickr.com/photos/the-travelling-bum/5905853321/</p></li><li><p>Pantheon.io 2</p><p>Session Description</p><p>Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for </p><p>using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site </p><p>repositories for optimal use on teams.</p><p>We will answer such gripping questions as:</p><p> How do I avoid placing a copy all of the Drupal core and contrib modules into my repository?</p><p> How do I keep my core and contrib modules up-to-date?</p><p> What if I need to customize .htaccess, or some other file?</p><p> Can I work with a "lean" repository, and still seamlessly use a full repository to deploy?</p><p> How do I share Drush aliases with team members, without using another repository, and without making my alias list too long?</p><p> What is a Drush wrapper script, and how can it help me?</p><p> How does Drupal Console fit in to all of this?</p><p> Should I use test fixtures in my Behat tests, or make a copy of the production site database?</p><p>These techniques will also be helpful for solo site developers--you never know, the next person who needs to check out, build and test your Drupal site </p><p>from scratch might be you!</p></li><li><p>Pantheon.io</p><p>Then create a Git repository:</p><p>cd myproject</p><p>git init</p><p>git add .</p><p>git commit -m My new project</p><p>3</p><p>Getting Started with Drupal and Composer</p><p>One easy step in Composer:</p><p>composer create-project drupal-composer/drupal-project myproject --stability=dev</p><p>Install and run with Drush:</p><p>cd web</p><p>drush qd --db-url=mysql://root@localhost/myprojectdb</p><p>Answer y</p><p>es when C</p><p>omposer </p><p>asks to re</p><p>move VCS</p><p> files, or r</p><p>un </p><p>with --no</p><p>-interac</p><p>tion opti</p><p>on</p></li><li><p>Pantheon.io 4</p><p>Preferred Project Structure</p><p>webcomposer.json</p><p>drupal console</p><p>circle.yml</p><p>external libraries</p><p>behat.yml</p><p>behat drush</p><p>drupal/core</p><p>index.phpcustom-installers</p><p>drupal-scaffoldmodules</p><p>contrib</p><p>custom</p><p>vendor</p></li><li><p>Pantheon.io 5</p><p>Specifying Project Layout</p><p>{</p><p> "name": "drupal-composer/drupal-project",</p><p> "scripts": {</p><p> "drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",</p><p> "post-install-cmd": "sh ./scripts/composer/post-install.sh"</p><p> },</p><p> "extra": {</p><p> "installer-paths": {</p><p> "web/core": ["type:drupal-core"],</p><p> "web/modules/contrib/{$name}": ["type:drupal-module"],</p><p> "web/profiles/contrib/{$name}": ["type:drupal-profile"],</p><p> "web/themes/contrib/{$name}": ["type:drupal-theme"],</p><p> "drush/contrib/{$name}": ["type:drupal-drush"]</p><p> }</p><p> }</p><p>}</p></li><li><p>Pantheon.io 6</p><p>Renaming Document Root</p><p># Ignore directories generated by Composer</p><p>drush/contrib</p><p>vendor</p><p>docroot/core</p><p>docroot/modules/contrib</p><p>docroot/themes/contrib</p><p>docroot/profiles/contrib</p><p># Ignore Drupal's file directory</p><p>docroot/sites/default/files</p><p>.gitignore</p><p>#!/bin/sh</p><p>DOCUMENTROOT=docroot</p><p># Prepare the scaffold files if they are not </p><p>if [ ! -f $DOCUMENTROOT/autoload.php ]</p><p> then</p><p> composer drupal-scaffold</p><p> mkdir -p $DOCUMENTROOT/modules</p><p> mkdir -p $DOCUMENTROOT/themes</p><p> mkdir -p $DOCUMENTROOT/profiles</p><p>post-install.sh</p><p>{ "extra": { "installer-paths": { "docroot/core": ["type:drupal-co "docroot/modules/contrib/{$name} "docroot/profiles/contrib/{$name "docroot/themes/contrib/{$name}" "drush/contrib/{$name}": ["type: } }}</p><p>composer.json</p></li><li><p>Pantheon.io 7</p><p>Scaffold Files</p><p>web</p><p>drupal/core</p><p>scaffold files</p><p>modules</p><p>contrib</p><p>custom</p><p>.htaccess</p><p>autoload.php</p><p>index.php</p><p>robots.txt</p><p>updates.php</p><p>sites</p><p>examples.sites.php</p><p>default</p><p>default.services.yml</p><p>default.settings.php</p><p>Drupal.org</p><p>drupal-scaffold</p></li><li><p>Pantheon.io 8</p><p>Custom Scaffold Configuration</p><p>{</p><p> "name": "pantheon-systems/drupal-project",</p><p> "scripts": {</p><p> "drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",</p><p> "post-install-cmd": "sh ./scripts/composer/post-install.sh"</p><p> },</p><p> "extra": {</p><p> "drupal-scaffold": {</p><p> "source": "https://github.com/pantheon-systems/drops-8/archive/{version}.tar.gz",</p><p> "includes": [</p><p> "sites/default/settings.php",</p><p> "sites/default/settings.pantheon.php"</p><p> ]</p><p> }</p><p> }</p><p>}</p></li><li><p>Pantheon.io 9</p><p>Document Root at Project Root</p><p>composer.json</p><p>drupal console</p><p>external libraries</p><p>drush</p><p>core</p><p>index.phpcustom-installers</p><p>drupal-scaffoldmodules</p><p>contrib</p><p>custom</p><p>vendor</p><p>circle.yml</p><p>private</p><p>Privatesubfolder</p></li><li><p>Pantheon.io 10</p><p>composer.json Relocated to Private Directory</p><p>{</p><p> "name": "pantheon-systems/drupal-project",</p><p> "scripts": {</p><p> "drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",</p><p> "post-install-cmd": "sh ./scripts/composer/post-install.sh"</p><p> },</p><p> "extra": {</p><p> "installer-paths": {</p><p> "../core": ["type:drupal-core"],</p><p> "../modules/contrib/{$name}": ["type:drupal-module"],</p><p> "../profiles/contrib/{$name}": ["type:drupal-profile"],</p><p> "../themes/contrib/{$name}": ["type:drupal-theme"],</p><p> "../drush/contrib/{$name}": ["type:drupal-drush"]</p><p> }</p><p> }</p><p>}</p></li><li><p>Pantheon.io 11</p><p>Updating Code</p><p>web</p><p>drupal/core</p><p>scaffold files</p><p>modules</p><p>contrib</p><p>custom</p><p>.htaccess</p><p>autoload.php</p><p>index.php</p><p>robots.txt</p><p>updates.php</p><p>sites</p><p>examples.sites.php</p><p>default</p><p>default.services.yml</p><p>default.settings.php</p><p>Drupal.org</p><p>drupal-scaffold</p><p>composer update</p><p>composer drupal-scaffoldgit pull</p><p>Project FilesOnly</p><p>Scaffold FilesOnly</p><p>Composer and Scaffold</p><p>Files</p></li><li><p>Pantheon.io 12</p><p>Customizing Scaffold Files</p><p>{</p><p> "name": "pantheon-systems/drupal-project",</p><p> "scripts": {</p><p> "drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",</p><p> "post-install-cmd": "sh ./scripts/composer/post-install.sh",</p><p> "post-drupal-scaffold-cmd": "cat htaccess-append.txt &gt;&gt; ../.htaccess"</p><p> }</p><p>}</p><p>To patch</p><p> non-sca</p><p>ffold files</p><p>, use:</p><p>cweaga</p><p>ns/com</p><p>poser-</p><p>patche</p><p>s</p></li><li><p>Pantheon.io 13</p><p>Checking for Security Updates</p><p>With Drush</p><p>With Composer</p><p>$ drush pm-updatestatusChecking available update data ... [ok]Checking available update data for Drupal. [ok]Checking available update data for Token (token). [ok] Name Installed Version Proposed version Message Token (token) 7.x-1.2 7.x-1.6 SECURITY UPDATE available</p><p>$ composer require roave/security-advisories:dev-master$ composer require drupal-composer/drupal-security-advisories:7.x-dev./composer.json has been updatedLoading composer repositories with package informationUpdating dependencies (including require-dev)Your requirements could not be resolved to an installable set of packages.</p><p> Problem 1 - drupal-composer/drupal-security-advisories 7.x-dev conflicts with drupal/token[7.1.2]. - Installation request for drupal-composer/drupal-security-advisories 7.x-dev -&gt; satisfiable by drupal-composer/drupal-security-advisories[7.x-dev]. - Installation request for drupal/token 7.1.2 -&gt; satisfiable by drupal/token[7.1.2].</p><p>n.b. Requires git_deploy module if using Composer with --prefer-source, or if dev modules are used.</p></li><li><p>Pantheon.io 14</p><p>Manage Custom Modules with Composer</p><p>webcomposer.json</p><p>drupal console</p><p>circle.yml</p><p>external libraries</p><p>behat.yml</p><p>behat drush</p><p>drupal/core</p><p>index.phpcustom-installers</p><p>drupal-scaffoldmodules</p><p>contrib</p><p>custom</p><p>vendor</p><p>Variant</p><p>WHY?</p><p>Use private modules in multiple projects.</p></li><li><p>Pantheon.io 15</p><p>Lets Make a Module!</p><p>$ drupal generate:module $ drupal generate:controller</p></li><li><p>Pantheon.io 16</p><p>Customize Modules composer.json</p><p>{ "name": "greg-1-anderson/snazzy", "type": "drupal-module", "description": "A Snazzy Module", "keywords": ["Drupal"], "license": "GPL-2.0+", "homepage": "https://github.com/greg-1-anderson/snazzy", "minimum-stability": "dev", "support": { "issues": "https://github.com/greg-1-anderson/snazzy/issues", "source": "https://github.com/greg-1-anderson/snazzy" }, "require": { }}</p></li><li><p>Pantheon.io 17</p><p>Add Our Custom Module to Drupal Project</p><p>{ "repositories": [ { "type": "vcs", "url": "https://github.com/greg-1-anderson/snazzy.git" } ], "require": { "composer/installers": "^1.0.20", "drupal-composer/drupal-scaffold": "^1.0", "greg-1-anderson/snazzy": "dev-master" }, "extra": { "installer-paths": { "web/modules/custom/{$name}": ["greg-1-anderson/snazzy"], "web/modules/contrib/{$name}": ["type:drupal-module"] } }}</p></li><li><p>Pantheon.io 18</p><p>Generate require Section from Existing Site</p><p>composer create-project drupal-composer/drupal-project myproject --stability=dev</p><p>cd myproject</p><p>drush composer-generate @remote</p><p>Look up t</p><p>he modul</p><p>es and the</p><p>mes </p><p>used on @</p><p>remote</p><p> and add</p><p> them to </p><p>the requi</p><p>re section</p><p> of the </p><p>compos</p><p>er.jso</p><p>n in the c</p><p>wd.</p></li><li><p>Pantheon.io 19</p><p>Lean Repository Deployment</p><p>Push</p><p>Request Test</p><p>Request Build</p><p>Behat</p><p>Deploy</p><p>Pull Lean Repo</p><p>Install</p><p>Update</p></li><li><p>Pantheon.io 20</p><p>Add a Project to Circle</p><p>A couple of clicks will get Circle CI building your project.</p><p>There are a few credentials that should also be set up:</p><p> GitHub OAuth Token Terminus Machine Token SSH Key Pair</p></li><li><p>Pantheon.io 21</p><p>Generate OAuth Tokens for all Services</p><p>GitHub Personal Access Tokens Pantheon Machine Tokens </p><p>OAuth tokens are getting to be very common; many services provide them.They work like passwords, but can have limited permissions, and may be revoked. </p></li><li><p>Pantheon.io 22</p><p>Copy OAuth Tokens into CI Envrionment Variables</p><p>OAuth Tokens:</p></li><li><p>Pantheon.io</p><p>Prevents com</p><p>poser install </p><p>from failing du</p><p>e to hitting the</p><p>GitHub rate lim</p><p>it.</p><p>23</p><p>Use OAuth Environment Variables in CI</p><p>dependencies: pre: - composer config -g github-oauth.github.com $GITHUB_OAUTH - terminus auth login --machine-token=$PANTHEON_MACHINE_TOKEN</p><p>circle.yml</p><p>Log in to Terminus via machine token, e.g. to later deploy from </p><p>dev to test.</p></li><li><p>Pantheon.io 24</p><p>Create SSH Key Pair Specifically for CI</p><p>$ ssh-keygen -C me+mysite-ci@example.com -f my-ci-keyfile</p><p>Add Private Key to Circle SSH Permissions Add Public Key to Provider SSH Keys</p><p>For linux servers: ssh-copy-id user@isp.com</p><p>mailto:me+mysite-ci@example.com</p></li><li><p>Pantheon.io 25</p><p>Dependency Hell</p><p>Bootstrapping Drupal 8 via a global Drush will ONLY WORK if the dependencies of the two projects are in perfect alignment. Upgrading one without upgrading the other is dangerous.</p><p>Drush Drupal 8</p><p>require autoload.php</p><p>Bootstrap require autoload.php</p></li><li><p>Pantheon.io 26</p><p>Simple Example of Dependency Hell</p><p>class Sub extends Base{public function foo(){return $this-&gt;bar();</p><p>}}</p><p>class Base{private function bar(){</p><p>}}</p><p>class Sub extends Base{public function foo(){return $this-&gt;boz();</p><p>}}</p><p>class Base{private function boz(){</p><p>}}</p><p>FancyLib v1.0.1 FancyLib v1.0.2</p><p>Sub.php Sub.php</p><p>Base.php Base.php</p><p>Semanti</p><p>c Versio</p><p>ning will</p><p> not sav</p><p>e </p><p>you!</p><p>Two aut</p><p>oloaders</p><p> that co</p><p>ntain m</p><p>ultiple </p><p>copies </p><p>of the sa</p><p>me libra</p><p>ry can s</p><p>till fail, </p><p>even if </p><p>both con</p><p>form to t</p><p>he same</p><p>public A</p><p>PI.</p></li><li><p>Pantheon.io 27</p><p>Dependency Hell Affects Drupal Console Too</p><p>Drupal Console, like Drush, keeps dependencies in sync with the Drupal 8 release with the same version number. Problems can still occur if versions become mismatched.</p><p>Drupal Console Drupal 8</p><p>require autoload.php</p><p>Bootstrap require autoload.php</p></li><li><p>Pantheon.io</p><p>Place Drush and Drupal Console in Drupals composer.json file:</p><p>cd /path/to/drupal-8-root</p><p>composer require drush/drush:8.*</p><p>composer require drupal/console</p><p>28</p><p>Solution is to Use a Single Autoloader</p><p>CRITI</p><p>CAL</p></li><li><p>Pantheon.io 29</p><p>Gentoos Composer Problem</p><p> As long as the necessary require statements are left in the code (where they belong), we can ignore Composer entirely and install the package with the system package manager. We set PHP's include directory for our users, so require('Class.php'); already looks in the right place. https://wiki.gentoo.org/wiki/Project:PHP/The_Composer_problem</p></li><li><p>Pantheon.io 30</p><p>How Could We Fix Things for Gentoo?</p><p>One Class.php shared by every application is not going to work!</p><p>Make an autoload.php that loaded versioned Class.php files from a global location?</p><p>How would all of the different versions of Class.php be managed?</p><p>How would you apply a security update for Class.php?</p><p>By the time we hav</p><p>e solved all of </p><p>these issues, we ha</p><p>ve pretty much re-</p><p>invented Compose</p><p>r.</p></li><li><p>Pantheon.io</p><p>Drush startup now happens in four phases:</p><p>31</p><p>Drush Startup</p><p>Drush Finder Drush Wrapper Drush Launcher Drush Application</p></li><li><p>Pantheon.io 32</p><p>Drush Finder</p><p>Responsible for finding the correct Drush to run</p><p> - Checks to see if it can find a Drupal site</p><p> - If the Drupal site contains a Drush script, use it</p><p> - If no Site-Local Drush is found, use global Drush</p><p>Does not consider alias files.</p><p>PHP Scriptnamed</p><p>drush</p></li><li><p>Pantheon.io 33</p><p>Drush Wrapper</p><p>Optional. Located at the Drupal Root if it exists.</p><p>User may customize this script to:</p><p>- Add site-specific options (e.g. commandfile locations)- Turn off global search locations with --local option- Select the location of the Drush launcher (if non-standard)</p><p>If there is no Drush Wrapper, then the Drush Finder will find and execute the Drush Launcher.</p><p>Shell Scriptnamed</p><p>drush.wrapper</p></li><li><p>Pantheon.io 34</p><p>Drush Launcher</p><p>Sets up the PHP operating environment.</p><p>- Select php.ini file to use</p><p>- Select php executable to use</p><p>- Passes info about environment to Drush</p><p>The launcher will always use the Drush application located in the same directory.</p><p>Shell Scriptnamed</p><p>drush.launcher</p></li><li><p>Pantheon.io 35</p><p>Drush Application</p><p>Contains all the code that is Drush.</p><p>- Load configuration and command file</p><p>- Parse site alias files</p><p>Might dispatch again, e.g. if site alias is remote.PHP </p><p>Applicationnamed</p><p>drush.php</p></li><li><p>Pantheon.io 36</p><p>Drush Phar</p><p>Bundles all of the code from the Drush application into a single file. </p><p>- Does not use Drush Launcher (no PHP executable selection).</p><p>- Will run a Drush Wrapper script if one is available.</p><p>If a Drush Wrapper or site-local Drush is found, they will be executed via redispatch. The site-local Drush may be a Phar or a regular Drush application.</p><p>Phar file named</p><p>drush.phar</p><p>Phar</p></li><li><p>Pantheon.io 37</p><p>Sharing Drush Aliases</p><p>cd "`dirname $0`"</p><p>private/vendor/bin/drush.launcher \</p><p> --local \</p><p> --include=../drush/commands \</p><p> --alias-path=../drush/aliases \</p><p> --config=../drush/config "$@"</p><p>drush.wrapper</p><p>drush</p><p>web</p><p>drush.wrapper</p><p>aliases</p><p>commands</p><p>index.php</p><p>aliases.drushrc.php</p><p>$ cd web</p><p>$ drush sa @live</p><p>$aliases["live"] = array (</p><p> 'remote-host' =&gt; 'server.isp.com',</p><p> 'remote-user' =&gt; 'www-admin',</p><p> 'root' =&gt; '/srv/www/live.example.com',</p><p> 'uri' =&gt; 'http://example.com',</p><p>);</p></li><li><p>Pantheon.io 38</p><p>Behat Driver Enhancement</p><p>cd /path/to/drupal-projectcomposer require drush-ops/behat-drush-endpoint</p><p>YES!</p><p>Adds behat Drush command to your site; used to remotely create content.</p></li><li><p>Pantheon.io 39</p><p>Behat Fixtures</p><p>Background: Given "places" terms: | name | | Kingdom of Imaginarium | | Empire of Fabrication | And "offices" terms: | name | | Grand Poohbah | | Undersecretary of Things | | Minister of Ministering |</p><p>$ drush...</p></li></ul>