Here’s the situation.

MageFlow is developing Magento Extension called MageFlow Connector. It’s an integration extension that is used to connect a Magento Instance to MageFlow. We are using Agile development processes and state-of-the-art tools by Atlassian like JIRA, Confluence, BitBucket, Bamboo and others. Bamboo is used as build and deployment server that automates a lot of development operations incl building and packaging our software.

Packaging Magento extensions – standard method

Usually Magento extensions are packaged in Magento backend. There’s a form that needs to be filled in with information about specific extension like version number, release notes, files and folders that should be packaged etc. After that you need to upload your extension to Magento Connect and publish it.

It’s pretty good tool but it requires manual work in order to create a package. However in our case we just despise any manual work:) Everything should be automated to max. That’s also why we even started creating MageFlow. Anyway …

Packaging Magento extensions with Bamboo CI

Packaging an extension with Bamboo requires some manual labor too. That cannot be avoided in my opinion. However …

Here’s my wild guess … With a really tight integration between JIRA and Bamboo and some tricks/scripts full automation could be achieved on the way from clicking “Create release” in JIRA to a fully packaged extension as an artifact of a Bamboo build. But right now it’s bit too much work for too little to gain.

But – keeping my head cold and not aiming 100% automation yet I created a following way to package an extension with Bamboo CI.

Prerequisites

GIT. For extension development we also use modman – a very useful tool for every Magento developer provided by Colin Mollenhour. It is also assumed that you have development environment ready in your local machine or elsewhere where you can prepare the package metainfo.

You also need to have a (preferred – vanilla) Magento (DevMagento) codebase in a GIT repository and you need to have YourExtension in a GIT repository. Make sure your DevMagento works fine and you can log in to the backend (admin).

Let’s name these GIT repositories like that for example:

DevMagento – git@bitbucket.com:myaccount/magento.git

MyExtension: git@bitbucket.com:myaccount/my_extension.git

Prepare Magento extension and modman

First you need to add MyExtension to DevMagento by using modman. In terminal do:

cd /srv/vhosts/magento
modman init
modman clone git@bitbucket.com:myaccount/my_extension.git

Second you need to create MyExtension XML metadata for Magento Connect package. This is done with the Magento Connect tool in Magento backend. Go to System->Magento Connect->Package extensions. Fill in all required fields there. Add version, release notes, files and folders, authors, dependencies etc. Click on “Save” button.

No back to Terminal:

cd /srv/vhosts/magento
ls public/var/connect

You see 2 very important files there now:

MyExtension.xml and package.xml

A mindnote here that package.xml will be overwritten if you package another extension in the backend of the same Magento while being in the same GIT branch.

Now create var/connect folder in your modmanned extension folder and move the files there:

cd /srv/vhosts/magento/.modman/my_extension
mkdir -p var/connect
mv ../../public/var/connect/MyExtension.xml ../../public/var/connect/package.xml var/connect/

Add these files to GIT and to your modman file.

Run modman update to re-create these files under Magento var/connect folder as symlinks:

cd /srv/vhosts/magento
modman update my_extension

Run a packaging test from command line to see if it works:

cd /srv/vhosts/magento/public
./mage package /srv/vhosts/magento/public/var/connect/package.xml
Done building package

It’s needless to say that you need to “git add”, “git commit” and “git push” your stuff etc. Now we’re almost ready to make Bamboo CI to create MyExtension-1.0.0.tgz. Also – see paragraph below about Magento bug after you finish next paragraph about configuring Bamboo.

Configure Bamboo CI

We are using standalone Bamboo CI in our own server. I’m not sure if everything looks the same in the ondemand solution…

Running unit tests on your Magento extension is not covered by this post. However I’m planning another post about this topic, too.

Install modman on your build server and add it as Bamboo executable if you haven’t done so yet.

In Bamboo create a new build plan called My Extension with plan key MYX. (Bamboo upper case plan keys and thus build paths with uppercase characters is where the Magento bug will strike in with its forced lowercase paths).

Add 2 repositories to your Build plan:

1. DevMagento repository
2. MyExtension repository

Make your build plan be triggered by periodically scanning the MyExtension repository for changes.

Configure build steps

Step 1: checkout source code from DevMagento repository. You may want to check out clean repo each time. It’s quite quick and you don’t build that extension too often. The reason is that otherwise next step will complain.

Step 2: run modman init. I.e choose modman executable and add “init” as command argument.

Step 3: run modman clone. Choose modman executable and add “clone git@bitbucket.org:myaccount/my_extension.git” as command argument

Step 4: Add a script as build step with ${bamboo.build.working.directory} as an argument to that command. The script is provided in this gist:

This script does 2 things:

1. it runs mage script that creates Magento package
2. it moves created tgz package from modmanned extension var/connect folder to Magento’s var/connect folder. The point is that Magento’s Package.php that creates the package uses realpath() PHP function. realpath() resolves symlinks created by modman and that’s why the .tgz package is created under .modman/my_extension/var/connect, not under public/var/connect.

After configuring build steps configure your build artifacts. Specify public/var/connect/ as artifact location and *.tgz as Copy pattern. You can make the artifacts shared so that a Bamboo Deployment plan can use the artifacts and deploy them to – let’s say – your test server;) But that’s another whole story …

Thanks!

Bug in Magento packaging scripts

NB! There’s a bug/misdesign in Magento packaging scripts. The solution is provided as a gist:

You may need to fix it in 2 places: under lib and under download. That’s why I recommended to use a vanilla Magento repository because you need to fix core library files. Create a special Magento repository, fix the bug there and use this Magento for packaging your extensions.

Alternatives

Alan Storm has created a good piece of software that enables creation of a Magento extension package from command line. However I found it too late and I’m not sure about 100% match with my problem.

Summary

It’s a pretty long post that covers quite complex issues. Feel free to leave a comment here or write me directly if you have any questions or thoughts about it.

Update: please see Part 2 of the same topic.