Part 2: Continuous Integration server to package Magento Extension with a click from JIRA

Starting Magento Connect extension packaging process from JIRA

When I wrote the first part of Continuous Integration server to package Magento Extension I didn’t know how to make it work with a click from JIRA. As you may know in JIRA there’s a way to release versions and trigger Bamboo builds during this release process. That’s what I was aiming – to make it work from JIRA so that it would NOT require any technical knowledge from let’s say Project Manager or Release Manager. I didn’t know it yesterday but here at MageFlow we have a learning mindset. I know it today:)

First – there are now 2 build plans configured in our Bamboo for our MageFlowConnect extension:

  1. “continuous builds” or “CI builds” that are triggered by commits that are made to the git repository that contains MFX code.
  2. “release builds” or “manual builds” that are triggered by clicking “Release” in JIRA

There are 2 main problems with creating Magento extension packages automatically:

  1. when and how to update extension’s version number (in module’s etc/config.xml and in package.xml mentioned in previous post)
  2. how to update release notes inside package.xml. However this one is not mandatory and it’s not resolved yet by us either.

Prerequisites

Install xsltproc to your build server and configure an executable “XSLTProc” in Bamboo.

Updating version numbers

In Part 1 I’ve created a XSL stylesheet that does the job of replacing extension version number in package.xml. Exactly the same XSL can be used to replace version number in extension’s etc/config.xml, too.

In your “CI build” process you may want to have extension version number increased with every build automatically. So that’s what we do  – we set the extension version number to that already defined in package.xml with added Bamboo build number. For example if package.xml contains version number 1.1.3 and Bamboo build # is 43 then the final version number of that extension build would be 1.1.3.43.

In order to replace version numbers in the “CI build” process add following 2 XSLTProc tasks to the build job in Bamboo:

  1. –output public/var/connect/package.xml –stringparam package_version ${bamboo.buildNumber} utils/update_version.xsl public/var/connect/package.xml
  2. –output public/app/code/community/Mageflow/Connect/etc/config.xml –stringparam package_version ${bamboo.buildNumber} utils/update_version.xsl public/app/code/community/Mageflow/Connect/etc/config.xml

First one appends ${bamboo.buildNumber} to existing version number (e.g 1.1.3) in package.xml while second does the same in etc/config.xml

In order to replace version numbers in the “Release build” (build triggered from JIRA) process add following 2 XSLTProc tasks to the build job in Bamboo:

  1. –output public/var/connect/package.xml –stringparam package_version ${bamboo.jira.version} utils/update_version.xsl public/var/connect/package.xml
  2. –output public/app/code/community/Mageflow/Connect/etc/config.xml –stringparam package_version ${bamboo.jira.version} utils/update_version.xsl public/app/code/community/Mageflow/Connect/etc/config.xml

First replaces version in package.xml with the version number specified in JIRA (let’s say 1.1.4) while second does the same in extension’s etc/config.xml

That’s it!:)

Please feel free to comment or send me questions directly.

 

 

Continuous Integration server to package Magento Extension

Here’s the situation.

MageFlow is developing Magento Extension called MageFlow Connector. It’s an integration extension that is used to connect a Magento Instance to MageFlow. We are using Agile development processes and state-of-the-art tools by Atlassian like JIRA, Confluence, BitBucket, Bamboo and others. Bamboo is used as build and deployment server that automates a lot of development operations incl building and packaging our software.

Packaging Magento extensions – standard method

Usually Magento extensions are packaged in Magento backend. There’s a form that needs to be filled in with information about specific extension like version number, release notes, files and folders that should be packaged etc. After that you need to upload your extension to Magento Connect and publish it.

It’s pretty good tool but it requires manual work in order to create a package. However in our case we just despise any manual work:) Everything should be automated to max. That’s also why we even started creating MageFlow. Anyway …

Packaging Magento extensions with Bamboo CI

Packaging an extension with Bamboo requires some manual labor too. That cannot be avoided in my opinion. However …

Here’s my wild guess … With a really tight integration between JIRA and Bamboo and some tricks/scripts full automation could be achieved on the way from clicking “Create release” in JIRA to a fully packaged extension as an artifact of a Bamboo build. But right now it’s bit too much work for too little to gain.

But – keeping my head cold and not aiming 100% automation yet I created a following way to package an extension with Bamboo CI.

Prerequisites

GIT. For extension development we also use modman – a very useful tool for every Magento developer provided by Colin Mollenhour. It is also assumed that you have development environment ready in your local machine or elsewhere where you can prepare the package metainfo.

You also need to have a (preferred – vanilla) Magento (DevMagento) codebase in a GIT repository and you need to have YourExtension in a GIT repository. Make sure your DevMagento works fine and you can log in to the backend (admin).

Let’s name these GIT repositories like that for example:

DevMagento – git@bitbucket.com:myaccount/magento.git

MyExtension: git@bitbucket.com:myaccount/my_extension.git

Prepare Magento extension and modman

First you need to add MyExtension to DevMagento by using modman. In terminal do:

cd /srv/vhosts/magento
modman init
modman clone git@bitbucket.com:myaccount/my_extension.git

Second you need to create MyExtension XML metadata for Magento Connect package. This is done with the Magento Connect tool in Magento backend. Go to System->Magento Connect->Package extensions. Fill in all required fields there. Add version, release notes, files and folders, authors, dependencies etc. Click on “Save” button.

No back to Terminal:

cd /srv/vhosts/magento
ls public/var/connect

You see 2 very important files there now:

MyExtension.xml and package.xml

A mindnote here that package.xml will be overwritten if you package another extension in the backend of the same Magento while being in the same GIT branch.

Now create var/connect folder in your modmanned extension folder and move the files there:

cd /srv/vhosts/magento/.modman/my_extension
mkdir -p var/connect
mv ../../public/var/connect/MyExtension.xml ../../public/var/connect/package.xml var/connect/

Add these files to GIT and to your modman file.

Run modman update to re-create these files under Magento var/connect folder as symlinks:

cd /srv/vhosts/magento
modman update my_extension

Run a packaging test from command line to see if it works:

cd /srv/vhosts/magento/public
./mage package /srv/vhosts/magento/public/var/connect/package.xml
Done building package

It’s needless to say that you need to “git add”, “git commit” and “git push” your stuff etc. Now we’re almost ready to make Bamboo CI to create MyExtension-1.0.0.tgz. Also – see paragraph below about Magento bug after you finish next paragraph about configuring Bamboo.

Configure Bamboo CI

We are using standalone Bamboo CI in our own server. I’m not sure if everything looks the same in the ondemand solution…

Running unit tests on your Magento extension is not covered by this post. However I’m planning another post about this topic, too.

Install modman on your build server and add it as Bamboo executable if you haven’t done so yet.

In Bamboo create a new build plan called My Extension with plan key MYX. (Bamboo upper case plan keys and thus build paths with uppercase characters is where the Magento bug will strike in with its forced lowercase paths).

Add 2 repositories to your Build plan:

  1. DevMagento repository
  2. MyExtension repository

Make your build plan be triggered by periodically scanning the MyExtension repository for changes.

Configure build steps

Step 1: checkout source code from DevMagento repository. You may want to check out clean repo each time. It’s quite quick and you don’t build that extension too often. The reason is that otherwise next step will complain.

Step 2: run modman init. I.e choose modman executable and add “init” as command argument.

Step 3: run modman clone. Choose modman executable and add “clone git@bitbucket.org:myaccount/my_extension.git” as command argument

Step 4: Add a script as build step with ${bamboo.build.working.directory} as an argument to that command. The script is provided in this gist:

This script does 2 things:

  1. it runs mage script that creates Magento package
  2. it moves created tgz package from modmanned extension var/connect folder to Magento’s var/connect folder. The point is that Magento’s Package.php that creates the package uses realpath() PHP function. realpath() resolves symlinks created by modman and that’s why the .tgz package is created under .modman/my_extension/var/connect, not under public/var/connect.

After configuring build steps configure your build artifacts. Specify public/var/connect/ as artifact location and *.tgz as Copy pattern. You can make the artifacts shared so that a Bamboo Deployment plan can use the artifacts and deploy them to – let’s say – your test server;) But that’s another whole story …

Thanks!

Bug in Magento packaging scripts

NB! There’s a bug/misdesign in Magento packaging scripts. The solution is provided as a gist:

You may need to fix it in 2 places: under lib and under download. That’s why I recommended to use a vanilla Magento repository because you need to fix core library files. Create a special Magento repository, fix the bug there and use this Magento for packaging your extensions.

Alternatives

Alan Storm has created a good piece of software that enables creation of a Magento extension package from command line. However I found it too late and I’m not sure about 100% match with my problem.

Summary

It’s a pretty long post that covers quite complex issues. Feel free to leave a comment here or write me directly if you have any questions or thoughts about it.

 

Update: please see Part 2 of the same topic.

2 useful scripts for managing your GIT repository

First – GIT is not good at storing large and changing binary files (like images, mp3-s etc). It’s not meant for this. Don’t use it for this. Anyway, sometimes things happen and you have got some large images into your repository and the repo is growing and growing and growing…

I have published 2 scripts that help to manage your GIT repository. First one helps to find large files from your repository and the second one helps to remove these from the whole GIT repository history. If you thought that git rm would remove it or make your repository smaller then think twice:) It’s much more complex than that. If you have a distributed repository that has large binaries in it then you’re in bigger trouble. This case is not covered where although there are some methods to shrink those distributed repos, too. But it’s beyond this article’s scope.

Script for finding files:

Script for removing files:

PHP system, exec, passthru and return code 127

This is a mindnote for myself. If someone else finds it useful, too == profit.

Because of MageFlow development I need to use quite a lot shell commands, for example ping.

Whenever exec, passthru or system returns code 127 check the path of the command. It means that the command cannot be found. It also means that PHP runs the command in such a shell that does not necessarily have PATH set properly and thus the command is not found.

Example

system('ping mygreathost.com', $retval);

$retval is 127.

Specify full path to ping command:

system('/bin/ping mygreathost.com', $retval);

$retval is 0 or 1 depending if mygreathost.com is available or not.

Oneliner for retrieving Magento database password from local.xml

No comments:

Setting up Magento development environment, step by step. Part 3: Naming and structure

Naming and structure

Welcome back to Measure9 blog. Here I continue with article series “Setting up Magento development environment, step by step“. Please read Part 1 and Part 2, too.

I write about naming of things in this post. It’s important, it’s very important because in the long run it defines how you and you’re colleagues are going to talk about things, how you point at things and so on. It’s important to use common terms to build common understanding. Also I’ll explain the importance of SSH keys. I’ve learned to name things thru standards, conventions used in frameworks and long time industry-experience.

I’d like to start with this infamous and very good quote by Phil Karlton:

There are only two hard things in Computer Science: cache invalidation and naming things.

General rules about naming

  1. Never ever use special characters, other than ASCII letters in your file and folder names. Regardless of what Operating Systems claim there will be problems as soon as you move files around between different platforms and systems. I guarantee that. I’ll buy you a beer if you can prove that you’ll never had any problems with file/folder names with special characters 🙂 It’s especially important in non-English countries like Estonia (we have õ, ä, ö, ü), Germany, Sweden, Russia… Basically anywhere outside US and UK. (Side note – regardless of file names your application, your code, your files etc must be in / support UTF-8. That’s a standard.)
  2. Be very careful with case of names, especially when you’re working between case sensitive (all Linuxes), partly case sensitive (OSX) and case insensitive systems (Windows). Always prefer lower case where possible but as you know – Magento’s and Zend’s standards state some rules about class names etc. So just be careful and consistent.
  3. Avoid spaces in file and folder names.
  4. www-prefix is so-20th-century. My suggestion is to phase it out and let it go …

Development folders

Magento development folders/repositories/virtual hosts are kept under /srv/vhosts. Under that folder there are project folders. Usually one project has one, main domain name and it makes sense to name your development folder according to that. It’s easy to remember and it’s unique. Let’s say your customer is example.com. So your folder structure would look like this (explanation is below):

/srv
    /vhosts
        /example.com
            /conf
            /database
            /doc
            /design
            /keys
            /public
            /util

conf – virtual host configuration and other configuration files are in this folder

database – database change scripts, temporary dumps and other database-related stuff this is here

doc – here is project and code documentation (for example generated by ApiGen). Don’t hesitate to use external tools like Google Drive and a WIKI for documentation!

design – here are working PSD files of the theme, HTML templates etc, bought themes, generated/exported graphics, icons etc.

keys – keep SSL certificate keys and other keys in this folder. Here you can also keep project-related passwords in an encrypted file if you’re old-fashioned;)

public – this is the document root of your Magento. In other words this is the public folder that is served by your web server to end users. There are lots of other names that are used for this folder like html, pub, htdocs and so on. I decided to stick to public because it’s used in Zend Framework 2 and it says what it is. Always do remember that this folder is public, publicly accessible!

util – utility and helper scripts are kept in this folder.

Development hostnames (virtual hosts)

Most developers are working in local development machine and this tutorial assumes this way of working. In some cases people use hostnames like example.dev or example.local in their local development machine and these names are pointing to 127.0.0.1 i.e localhost. There are reasons to do like this but this is not good when you need to test integrations like Oauth. Google’s API Console does not support .local domain names. That’s why I suggest usage of following naming structure:

dev.example.com – point it to 127.0.0.1 in your /etc/hosts file.

test.example.com – point it to your test server in DNS.

However usually the DNS is under customer’s control and making changes there is time consuming or impossible. To overcome that you need to use YOUR domain for development and your own / your ISP-s DNS. If that’s the case you could use common naming scheme for development and test:

example.com.dev.mydomain.com – point it to 127.0.0.1

example.com.test.mydomain.com – point to to your test-server’s IP-address

staging.example.com – staging is also called pre-live. It’s an exact copy of live environment but usually it’s using less hardware / virtual servers. It’s the server where content and changes are staged before deploying these to production system (which is example.com or www.example.com).

If customer is having problems with DNS configuration then you can use your own DNS for the staging site also:

example.com.staging.mydomain.com – point it to the staging server’s IP-address

Usernames, passwords

My recommendation is to use e-mail addresses as usernames everywhere where possible. It’s not possible in most Linux shells but it’s possible in Magento’s frontend and backend. People, I mean – nerds and hackers, use all kinds of nicknames but it’s kind of … childish. Isn’t it? So use your e-mail as username in all your Magento solutions (BE and FE) and you and your colleagues know that and they don’t have to ask you every time. Easy to remember and easy to use!

About passwords. Try to avoid them as much as possible. They are just so insecure. Implement Oauth for Magento’s FE and BE login. Ask me if you need help with that.

If you need to create a password for a user for any reason then:

  1. make it complex (at least 12 characters, uppercase, lowercase, numbers and special symbols in it).
  2. make it expired so user must change it at next login and you don’t need to know it.
passwd -e username

This command would expire password for username  and she will be prompted for password change at her next login.

Another useful command is

passwd -d username

That command would remove password from the user’s account so that user can login with SSH key’s only (see below).

In Magento backend create a random password for your colleagues, forget it right there and ask your colleagues to use “Forgot password?” link to reset their password. You’ll test e-mail functionality of your Magento server this way, too 😉

SSH keys, logins, aliases

SSH keys are an excellent way to login without passwords. There is one very important thing to be remember about SSH key pairs:

SSH private key MUST NEVER leave the machine where it was created.

Translation to English: use SSH private key only on that machine where it was created. If you need another one, generate it and distribute another public key.

SSH key pairs can be generated with a simple command:

ssh-keygen

The default values are good in most cases. After you generate the key pair you will have 2 keys in your ~/.ssh folder:

  1. private key
  2. public key

The public key is “inherited” from the private key. Public key as the name says is public. After generating keys you need to distribute the public key to the servers where you want to log in. Usually the public key must be appended to the file ~/.ssh/authorized_keys. In Linux systems there is also command ssh-copy-id that is meant for distributing your public key. Use your password for the last time to distribute your public key and forget/delete your password:)

How to create headless virtualbox machine with Debian Wheezy

I’ve created a small helper script for those who want to be able to create VirtualBox virtual machine from command line via SSH and RDP.

Prequisites

As a prerequisite you need to download Debian’s network install CD image.

The script

Here’s the script and I’ll explain it below line by line:

How it works?

In short it works so that you run this script in your existing Linux and you give virtual machine name as the only parameter. It creates a new virtual machine with pretty safe values that all can be changed later (e.g amount of memory).

First we do initialization, check the scripts arguments and assign the first argument as VMNAME variable:

if test $# != 1; then
echo "Usage: $0 vm_name"
exit 0
fi
VMNAME=$1

Then we create and register new virtual machine of type 64bit Debian:

vboxmanage createvm --name $VMNAME --ostype Debian_64 --register

Then we allocate RAM, set power management, set DVD as first booting device, set network in bridged mode and we also specify that the machine would have 2 CPU-s with possibility to hotplug CPU-s:

vboxmanage modifyvm $VMNAME --memory 512 --acpi on --boot1 dvd --nic1 bridged --bridgeadapter1 eth0 --nictype1 virtio --cpus 2 --cpuhotplug on

After that hard drive image is created with size of 4 gigabytes. Make it larger if you have plenty of disk space handy.

vboxmanage createhd --filename ~/VirtualBox\ VMs/testvm/$VMNAME-disk01.vdi --size 4096 --variant Standard

Then we create 2 controllers, one for harddisks (SATA) and the other for CD/DVD (IDE):

vboxmanage storagectl $VMNAME --name "SATA controller" --add sata
vboxmanage storagectl $VMNAME --name "IDE controller" --add ide

Then we attach the newly created hard disk to the controller:

vboxmanage storageattach $VMNAME --storagectl "SATA controller" --port 0 --device 0 --type hdd --medium ~/VirtualBox\ VMs/$VMNAME/$VMNAME-disk01.vdi

As a last thing we attach the downloaded Debian 7 network installation CD image:

vboxmanage storageattach $VMNAME --storagectl "IDE controller" --port 1 --device 0 --type dvddrive --medium /mnt/raid/soft/debian-7.2.0-amd64-netinst.iso

… and we start the virtual machine:

vboxheadless -s $VMNAME &

You’ll be displayed a message similar to this one:

VRDE server is listening on port 3389.

Open a Remote Desktop Client and connect to your host machine’s IP and port 3389. You will see Debian’s installation screen. Install it and off you go!:)

Later you may want to set the default boot device to harddisk instead of DVD:

vboxmanage modifyvm $VMNAME --boot1 disk

Voting with Mobile-ID in Estonia – how it looks

ID-infrastructure in Estonia – general info

As you may know Estonia has really great digital identity infrastructure. We can sign fully legal documents (I mean ALL documents) either with our ID-card or with our Mobile-ID. While the first is more common the latter may need some explanation and examples. In both cases there are certificates on the chip (on the SIM-card) and they’re protected with strong crypto. The private certs and keys never leave the chip. For proper authentication 2 things are needed:
– your certificate (on the chip)
– your PIN-code that unlocks the private key on the chip

So the PIN code and the chip alone are pretty useless and thus secure. So much about theory. Now some examples.

Voting with Mobile-ID

I’m on OSX so the examples are based on that fact.

First you download an app from the official elections website https://www.valimised.ee/mac

Open the app and there you see choices for authentication. They’re pretty self explainable.

v1When I click on mobiil-ID (Mobile-ID) the following dialog appears. It asks for my mobile phone number in order to send me Mobile-ID authentication message. The phone number entered is not mine. Don’t use it:)

v2

 

 

After clicking on big blue button “Saada” (Send) the following waiting dialog appears. During that dialog is visible I get a message to my phone that opens special SIM-card application and asks for my Mobile-ID PIN-1:v3

On my mobile phone I see dialog like this:

 

screenshot_2013-10-10_1214

 

Mind the numbers 5304 and 4567. These don’t match in my example because I made the screenshots from different sessions. These must match, otherwise something’s fishy and someone else might have sent you authentication request (not good).

I enter my PIN-1 on my mobile and that opens my authentication certificate. Background magic happens in ID-infra and I see next screen in my voting app:

v4

 

It welcomes me with my name and my personal ID-code (blurred). Then it explains that I can vote and blahblah… In bold it’s important message: it says that I’ve already voted. The trick is that I can vote unlimited times electronically but only the last vote is counted! Even better – if I change my mind then I can got and vote on the real (physical) elections day and then the vote made there will count. But who bothers going out if you can vote like this?:)

I click “Hääletama” (To voting) and I see next screen:

v5

 

This is the list of candidates. I make my choice and I see it like this:

v6

 

I don’t want to advertise anyone so it’s blurred. Then I have button “Valin” (Vote). After clicking on that I see confirmation screen:

v7

 

After I click on “Kinnitan” (Confirm) a second message is sent to my mobile phone. I see waiting screen like this:

v8

 

On my mobile I see similar screen as above but this time I’m asked my PIN-2 code which is meant for opening my signing (signature) certificate. After entering PIN-2 I see confirmation screen like this:

 

v9

It shows me a QR code that I could use to verify if my vote was accepted and stored properly. It can be verified 30 minutes after voting and max 3 times for security reasons. That’s why I blurred the QR also.

That’s it. It takes max 5 minutes if you know who’s your preferred candidate!:)

I hope whole Europe would adopt our digital ID infrastructure soon.

Go, Estonia!

Building GNUPG for PHP 5.5 on OSX

It’s quite tricky to build gnupg php extension on OSX. I’m using homebrew, so I installed necessary dependencies first:

brew install gpgme

This goes well as most things with brew.

Then I’m trying to install GNUPG PHP PECL extension as the instructions tell me to do:

sudo pecl install gnupg

Bang!!!

I get weird errors like these:

duplicate symbol _gnupg_keylistiterator_class_entry in:
.libs/gnupg.o
.libs/gnupg_keylistiterator.o
duplicate symbol _gnupg_class_entry in:
.libs/gnupg.o
.libs/gnupg_keylistiterator.o
ld: 2 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [gnupg.la] Error 1
ERROR: `make' failed

“Tough luck”, I think. But wait… After some googling around PHP bugs I got some hints.

So here’s what you got to do:

  1. download pecl extension source code
  2. fix 2 lines in .h and .c files
  3. phpize, configure and make
  4. configure PHP extension
  5. == Profit
pecl download gnupg

In my case it downloads files to /Users/sven/soft/php-5.5.4/ext/gnupg-1.3.3/gnupg-1.3.3.tgz

Unpack the archive:

tar xzf gnupg-1.3.3/gnupg-1.3.3.tgz
cd gnupg-1.3.3

Open file php_gnupg.h and add comments and explanation around line 50:

/* moved next line to gnupg.c
zend_class_entry *gnupg_class_entry;
 */

Copy the line zend_class_entry *gnupg_class_entry; to buffer. Save and exit.

Open file gnupg.c:

Find lines (around 177) and paste copied buffer so it looks like this:

zend_class_entry *gnupg_class_entry;
/* {{{ objects_new */
zend_object_value gnupg_obj_new(zend_class_entry *class_type TSRMLS_DC){

Do similar things with files gnupg_keylistiterator.c and php_gnupg_keylistiterator.h.

Open file  php_gnupg_keylistiterator.h around line 69:

/* moved next line to gnupg_keylistiterator.c
 zend_class_entry *gnupg_keylistiterator_class_entry;
 */

Copy line zend_class_entry *gnupg_keylistiterator_class_entry; to buffer. Save and exit.

Open file gnupg_keylistiterator.c around line 72 and make it look like this by pasting buffer:

zend_class_entry *gnupg_keylistiterator_class_entry;
/* {{{ keylistiterator_objects_new */
zend_object_value gnupg_keylistiterator_objects_new(zend_class_entry *class_type TSRMLS_DC){

Save and exit.

Configure, build and install: ./configure make clean && make make install

Now you should have php extension gnupg.so somewhere. I have it in my Cellar:

/usr/local/Cellar/php55/5.5.4/lib/php/extensions/no-debug-non-zts-20121212/gnupg.so

Wherever it is, copy the path to buffer.

Open a new file (your php-s conf.d may be somewhere else!)

vim /usr/local/etc/php/5.5/conf.d/ext-gnupg.ini

And add following lines there:

[gnupg]

extension="/usr/local/Cellar/php55/5.5.4/lib/php/extensions/no-debug-non-zts-20121212/gnupg.so"

Save and exit. Be sure to replace path if your gnupg.so is somewhere else.

Test your php:

php --ri gnupg

It should display something like this:

gnupg
gnupg support => enabled
GPGme Version => 1.4.3
Extension Version => 1.3.3-dev

Thanks!

 

 

 

Setting up Magento development environment, step by step. Part 2: Underlying systems and processes

Underlying systems and processes

Welcome back to Measure9 blog. Here I continue with article series “Setting up Magento development environment, step by step“. Please read Part 1, too. In this post I’m going to shed some light to surrounding processes and most low level tools. So we go from  general to more specific with each subsequent article in this series.

Development process

A quite generic Magento Development Process looks like that. It involves an Issue Tracking System (ITS) and Version Control System (VCS). Also unit testing is in place. If you’re really after quality and fun in development then I do recommend to set up Continuous Integration solution. There is quite a number of solution like Bamboo, Jenkins and others.

Magento Development Process

 

 

Version control process

Version control is something that should be in place even if you’re working alone. It helps you to keep your work organised, establish good working habits and integrate your work thru various environments and phases of a project’s lifecycle.

There is a number of Version Control Systems available. To name a few that you already may have heard of:

  1. GIT
  2. Subversion
  3. Mercurial
  4. CVS
  5. ClearCase

See more from Wikipedia. The first 3 are quite modern while CVS is an old dinosaur and ClearCase is a proprietary yet very powerful and integrated system by IBM. Anyway, my gut feeling says that GIT is most widely used nowadays and there are reasons for that.

GIT is

  1. distributed – you can work offline and still keep stuff under control
  2. fast – I mean it’s really fast:)
  3. easy to use, supports easy branching and merging
  4. last but not least – invented by Linus “The Linux Guy” Torvalds himself. Quality…

Now about the process itself (when git is used):

VersionControlProcessGit

Explanation from left to right:

When you start a project you do git clone (1). This gives you working folder with all project’s files. You do it once. Then, every day after your morning coffee you do git pull (2). Git pull gives you updates that have been pushed to upstream by other developers. After that you do git branch (3) if you haven’t done it already. This gives you isolated branch where you can mess around with the code any way you want. There is a number of branching strategies around, you need to choose what suits you best. There’s also a good example of a successful git branching model. Now you’re working with the code and you do git commit (4) (save code version to git). It’s recommended to keep commits small, I’d say atomic. Minimum one commit per fix or per issue from the ITS or per file. You’ll see that it’s good to keep commits small because when they day of rollback comes (and it will!) then you’d be grateful you did small commits that can be rolled back one by one. Then you feel that you’re ready to share your work with others. So you do git pull (5) again to retrieve latest changes from the remote and master branch. Then you merge (6) master branch into your named branch and if it’s cool (no merging errors) you’ll merge your changes back to master. Then you do git push (7) and share your work with others.

Some day (when I have time) I’ll add command line examples here, too.

Choosing hardware and installing operating system

Magento is a resource hog and thus good development hardware is a must. I’m personally using Macbook Air that has SSD permanent storage. It’s fast but standard MB Airs have only 4 gigs memory that is too little:( So you better get yourself a machine with following specs:

  • >= 8 GB RAM
  • >=128 GB SSD disk
  • A good (dual head) video card (for dual monitors)
  • A good keyboard
  • A mouse
  • A pair of good speakers (in case you’re working in your own room;))

Operating system

OSX and Linux are both good choices for Magento (or any web) development. For OSX there’s excellent package manager homebrew that gives you all necessary tools if you’re not satisfied with already built-in Apache web server and somewhat oldish PHP 5.3.

With Linux it’s simple – choose a distro that you like. My recommendation is Ubuntu or Debian. The both have APT package manager that satisfies all your needs.

Installation and configuration of tools will be covered in future posts.

Command line, Terminal application

One more thing is the terminal application. I think the the Linux’ defaults are good but there’s a better alternative for OSX’ Terminal.app and this is iTerm2. Install and use this one instead of native Terminal.app. It provides a few very good convenience functions like “copy when select” and similar.