GIT and deployment strategy Magento2 projects

Question

With Magento 1 I used a deploy tool that pulled in the GIT repo, ran commands like modman deploy-all and made sure the var directory was writable. For the .gitignore I used this one which worked pretty well.

But what about Magento 2?

What gitignore works best, how do you deploy your project and what command should one run pre- and post deploy. Looking forward to hearing some insights from the community.

Question will stay open for quite some time

By definition there can be no canonical answer to this, so it is likely too broad and also a poor fit for the Q&A nature of the site. Should likely be meta. But you already know this. That said, I'll allow it until the bounty expires. — philwinkle, Dec 16 '15 at 16:59
@philwinkle it might be broad but apparently not too broad since already 3 answers were given. As discussed here: http://meta.magento.stackexchange.com/questions/745/meta-website-is-starting-to-lose-its-purpose Meta is to be used for questions about MageSE, not random posts / questions If you want to delete it I can't stop you but it seems a lot of people are interested in the question and in my opinion it's a valid one, all be it not too specific. — Sander Mangel, Dec 20 '15 at 10:51
Two things: First, Sander is correct about Meta - it should only be used for questions about the SE platform as it relates to Magento SE (NB: we've maybe not policed Meta well enough to reinforce this rule). Second, "a lot of people [being] interested in" a question has nothing to do with whether a question can be answered canonically or not (and therefore with a question's suitability to the StackExchange format). It's frustrating for sure (I've come up against this myself). I'm inclined to see where this Q/A thread goes. Perhaps an A can be stated well enough to be exclusively "right"... — benmarks, Dec 21 '15 at 22:12
@benmarks in that case I've picked the wrong reason or subject for the bounty, my motivation behind it was to reward whoever took the time to write down a full answer for this. If this thread does not belong on here I'll copy it and post it somewhere online crediting the authors since I feel it still has value. Please notify me if before deleting — Sander Mangel, Dec 22 '15 at 07:27
It's okay, I think this thing is going to stay here; Marius, Phil and I have discussed. It's the most contention I have with the SE platform, that a valuable Q/A thread may not be allowed (or may be closed) because there's no "one true answer" (ref "Why Layout XML?" on SO) . Actually, I just re-read "What types of questions should I avoid asking?", and based on that I think there is ZERO problem with this thread staying here. — benmarks, Dec 22 '15 at 17:46
@benmarks thanks for the reply Ben, glad this falls within the rules of MageSE :) — Sander Mangel, Dec 22 '15 at 20:22
Seems quite fast deployment is described here https://www.yireo.com/tutorials/magento/magento-2/1839-proper-magento-2-deployment — Gediminas, Oct 31 '17 at 10:21
@benmarks thank you. I really appreciate when mods take a moment to keep the excellent information in SE sites. I've experienced too many diamond power mongers who give me the impression of closing great Q/As because they enjoy it -- and maybe resent that people like me can Google a Q/A and get answers to my problems from these "illegal" posts. — Krista K, Dec 28 '17 at 16:21

score 62 · Accepted Answer · edited Aug 17 '20 at 16:11

62

Steps below describe how to set up environment for custom module development, not for production.

Project initialization

Add repo.magento.com credentials and github access token to auth.json in composer home directory
Create project using the following command:

composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition .
Take this .gitignore and put into your project root. Almost all core files/directories are already added to the root .gitignore, but it is better to add the following 2 as well /update and /phpserver (just add these 2 lines to .gitignore)
Initialize new git repository in the project root
Add all untracked files to git and commit them
Start development of your modules as usual (put them under app/code/VendorName/ModuleName), now you will have only your custom code in your git repository

Magento installation

Make sure all filesystem permissions are set as outlined in the official guide
Install Magento using command line, e.g.:

${project_root}/bin/magento setup:install \ --db-host=localhost \ --db-name=magento \ --db-user=root \ --backend-frontname=admin \ --base-url=http://base.url.goes.here/ \ --language=en_US \ --timezone=America/Chicago \ --currency=USD \ --admin-lastname=Admin \ --admin-firstname=Admin \ --admin-email=admin@example.com \ --admin-user=admin \ --admin-password=123123q \ --cleanup-database \ --use-rewrites=1
Enable indexers cron job, e.g. on Ubuntu:

echo "* * * * * php ${project_root}/bin/magento cron:run &" | crontab -u www-data -
Magento will run in default mode and all missing content will be auto-generated upon first request. So no need to run compiler or static content deploy
[optional] If using PHP Storm, run the following command in to enable XSD support:

bin/magento dev:urn-catalog:generate .idea/misc.xml

edited Aug 17 '20 at 16:11

MClaudio

36
4

answered Dec 13 '15 at 19:47

Alex Paliarush

13,751
5
51
55

Hi Alex. Project initialization step 3 - could you expand it a bit? Did you find you had to manually copy that subdirectory to the root? (I am wondering if there is something not working correctly - I was not expecting that step.) – Alan Kent Dec 14 '15 at 04:00
@AlanKent currently you get all Magento-related files downloaded to vendor, including magento2-base, which is just a skeleton for a new project. Not sure why this step is not configured to be done automatically by composer, will try to find out. Regarding .gitignore copying from another repo, it is already being discussed, how to eliminate/simplify this step. – Alex Paliarush Dec 14 '15 at 06:51
Step 3 is not required. The marshaling of the files/folders is done during step 2. – Maddy Dec 14 '15 at 21:11
Thanks @Maddy. @AlanKent, copying magento2-base to the root is not necessary anymore (just verified), seems to be fixed recently. Removed this step from the answer. – Alex Paliarush Dec 15 '15 at 10:24
i put all of my code on the repo, already unstalled and everything, when i pull from that repo and just change the settings fort the admin pangel and the db credentials, will everything work correctly ? 2) since i forgot to exclude var/ and pub/ folder during the push, can i delete them completely, so they can delete on the remote repo, will they regenerate back ? Thanks.

Lachezar Raychev

Feb 19 '16 at 11:03

i ment already "*installed and everything" – Lachezar Raychev Feb 19 '16 at 11:25

@AlexPaliarush how to develop multiple magento modules(different git reps) in single magento instance – Aswanth Apr 01 '16 at 12:28

Why in that .gitignore do they do */* and /pub. What if you want files tracked in there? You have to git add -f – tread Nov 30 '16 at 15:12

Vladimir Kerkhoff · Answer 2 · 2015-12-16T11:45:39.390

28

For Initialisation and Installation follow the steps from Alex his answer for most of the steps, only differences I would recommend:

Git configuration

Only store the following files in your Git repository:

composer.json
composer.lock
app/etc/config.php

For your project custom code, also use separate modules that you include thru composer. Managing this thru composer is easier as you can lock a specific version/release that you want to deploy. This also forces you to use the same approach for internal and external modules.

Deployment

During development you update the modules on your environment (dev/test) with the command:

composer update

This will update the composer.lock file with the versions installed on that installation.

On staging/pre-production/production you can create/install the same setup with the command:

git pull
composer install

This will install all the same modules as used in dev/test to ensure that the testing before publishing to production is done with the same module versions as it is developed with.

After the installation to run the following commands:

bin/magento setup:upgrade
bin/magento setup:di:compile (or setup:di:compile-multi-tenant)
bin/magento setup:static-content:deploy

This will update the database (schema and data upgrade), generate the DI configuration and deploy all static view files.

edited Dec 16 '15 at 11:45

answered Dec 14 '15 at 09:11

Vladimir Kerkhoff

8,223
1
35
41

Maybe it make sense to use sample data instead of generating fixtures? Fixtures populate only the most critical modules and seem to be useful for performance testing only. – Alex Paliarush Dec 14 '15 at 09:28
Thanks, removed that part as it's not needed when using a site in production. – Vladimir Kerkhoff Dec 14 '15 at 09:29
This follows very closely the approach that I am using as well. This is also working well with Magento 1 as well(with a less complex build process) Let composer do its job, it works for deployments really well in my experience, and we have not seen many downsides other than complexities with .gitignore strategy when you do not follow the smaller footprint in git. – Aepod Dec 14 '15 at 17:24
This install looks like the 'integrator' way. It adds the repos in vendor/magento/*. No code will be in app/code/.. and other dirs. How would I have Magento 2 core dirs as in the .zip archive. Is it possible to add via composer a module (other git repo) and than to end up in app/code/... automatically? – Daniel Ifrim Mar 01 '16 at 23:36
First note that extra map was removed recently from core modules so there is only one place with module dirs - in vendor dir. If someone is interrested in having symlinked own modules in app/code, you can still do this: http://www.martin-kramer.com/2015/07/magento2-partial-symlink-composer-strategy You may need to remove the autoload JSON key in your module composer.json so you won't have 2 modules registration (not sure 100% about this but worked like this). – Daniel Ifrim Mar 02 '16 at 08:16
4

risky, composer is not a deployment tool. if something fails on composer install when running in production... – Claudiu Creanga Mar 14 '16 at 15:46
I like this approach. Isn't an extra step needed to set all the new config like database etc. Can this be done manually in app/etc/env.php ? – 11mb Mar 29 '16 at 09:58
You'd suggest to change small theme changes to Composer as well? – Kay Int Veen May 19 '16 at 09:15

score 4 · Answer 3 · edited Dec 19 '17 at 07:34

we run a different approach which does not involve a separate build-server/process, we locally develop as if in production

we then commit all files necessary for production. we then simply deploy the changesets to the server and run the upgrade command.

getting to a version which is suitable for development but also runs in production mode was the tricky part and is still not perfect but now we got a recipe which works.

the reason is that we want to have 100% control over what code goes into production. since magento2 generates a ton of code we must run it locally to be able to understand all the effects and being able to debug as if in production.

I'm aware that this not what many people recommend to do but for us it works best.

frontend-setup steps

In order for these scripts to work set your shop to production mode in your env.php and setup your theme in dev/tools/grunt/configs/themes.js. (the following steps were put into an ansible playbook)

delete var/cache
delete var/view_preprocessed
delete pub/static/* (don't delete the .htaccess)
delete var/composer_home
run php bin/magento cache:flush
run php bin/magento setup:static-content:deploy %your_languages%
delete all themes/languages you don't actually use from pub/static/frontend
remove hard-copies of less files from pub/static/frontend
run php bin/magento dev:source-theme:deploy --locale="%your_language%" --theme="%your_theme%" css/styles-m css/styles-l css/email css/email-inline
optional: we use a bash-script to change the absolute symlinks, created in step 9, into relative ones, making it possible to run grunt from outside the vm
run grunt less:your_theme

backend/di-setup steps

delete var/cache
delete var/generation
delete var/composer_home
delete var/di
run php bin/magento cache:flush
run php bin/magento setup:di:compile

Thanks for this @greenone83, this is the approach I've basically adopted although I generate my backend as part of the front end. I NEVER use setup:di:compile as I find it screws things up! One thing I don't understand is why setup:static-content:deploy creates files in generated/code (the location has changed from your post)? I've found that if I delete all of generated/code, with my site in production mode these files are automatically created when I browse around the site. — PedroKTFC, May 18 '19 at 13:57

score 3 · Answer 4 · answered Aug 23 '17 at 17:23

Re .gitignore, 2.2 onwards official Magento answer will be "config.php goes into git, env.php does not".

We are looking at composer plugins like the Mediawiki one to get internal dev closer to extension development and customer sites. Still exploring, not final yet.

I quite liked using the Composer "Path" repository type with a path of ../othergitrepo/app/code/*/* to pick up modules, but it uses symlinks which does not work so well with dev environments using Unison or similar.

score 2 · Answer 5 · edited Dec 19 '17 at 07:38

2

You should ignore these files too
/app/etc/config.php
/app/etc/env.php
/.idea/workspace.xml //phpstorm

edited Dec 19 '17 at 07:38

Jarnail S

4,697
30
45
80

answered Dec 14 '15 at 19:33

mrtuvn

3,273
5
28
39

2

If you ignore the config.php you will need to enable new extension again after pushing to an other environment, therefore its better to include it in your repository. – Vladimir Kerkhoff Dec 14 '15 at 20:06

GIT and deployment strategy Magento2 projects

5 Answers5

Linked