Resolving Magento extension rewrite conflicts using traits

A bit of history

On 5th of September I sent out a tweet:

Guys, this is serious. I guess, I just resolved Magento’s single intheritance problem by using traits. Details later. #magento #worksforme.

Those who have been serious in hacking Magento and 3rd party extensions know well what I’m talking about. But please let me explain further.

Magento extension rewrite conflicts

You have a Magento instance. You have created a custom module My_Custom that happens to rewrite a core model, let’s say Mage_Catalog_Model_Product. So you have following lines in your module’s config.xml:

Then you have have a 3rd party extension that rewrites the same core class. You’re stuck now? Not quite. You now have 3 good methods to fix the problem. So good so far. But then you need another 3rd party extension that does the same – rewrites catalog/product. Now you’re stuck, right!?
You cannot make your code extend the 1st extension because the the second wouldn’t work and you cannot make yours to extend the second either because the 1st wouldn’t work then. Chicken and egg problem or “nokk kinni, saba lahti” as we say in Estonian.

The solution – PHP 5 feature “traits”

There’s a solution to PHP’s single inheritance feature and Magento rewrites that together make a problem. The solution is achieved by utilizing PHP 5.4 (or newer) feature called traits. So get a coffee (it’s pretty long post) and let’s start.
We have a situation where 3 classes want to rewrite and extend Magento’s Mage_Catalog_Model_Product:
My_Custom_Model_Product (this is mine that I can freely change)
Extension_A_Model_Product
Extension_B_Model_Product
Let’s say Extension_A_Model_Product adds the following methods to catalog/product:
And let’s assume Extension_B_Model_Product wants to add the following methods to catalog/product:
And at last my custom module wants to add something extra to the core product model:

The idea is to make only My_Custom_Model_Product to rewrite catalog/product or at least to be the first one selected for rewrites. The other 2 3rd party module classes would just extend it and be used inside my product model via traits.

Add dependencies to your module

First it’s necessary to add dependencies to your module XML. This is to ensure the loading order of classes and rewrites.

Add traits

Folder Traits should be created under extension’s folder in similar manner to Block, Product and Helper:
My/Custom/Trait.
For naming we follow the same logic as other Magento building blocks but there’s no shorthand for traits as it exists for blocks, models and helpers (e.g catalog/product etc). No worries, the trait would still be found and loaded by the autoloader.
Under that folder create the trait folders and files for classes Extension_A_Model_Product and Extension_B_Model_Product
My/Custom/Trait/Extension/A/Model/Product.php

My/Custom/Trait/Extension/B/Model/Product.php

Trait for Extension_A_Model_Product

Trait for Extension_B_Model_Product

Add traits to My_Custom_Model_Product

Voilaa!:) Now you have all the 3rd party module functionality available in your extension. When Magento is calling getModel(‘catalog/product’) it will actually load My_Custom_Model_Product. My_Custom_Model_Product extends Mage_Catalog_Model_Product and it also exposes all methods from Extension_A_Model_Product and Extension_B_Model_Product.

A note about traits

With traits you can avoid duplicating original class method implementation’s part. You’ll just create a proxy for original class method’s thru the trait. However you need to maintain the traits whenever Extension_A or Extension_B are upgraded or changed. So you should document it that you created traits and solved multiple rewrite problem this way. Then you can look up the documentation when doing upgrade to extensions next time.

As always – feedback is welcome!

Setting up Magento development environment, step by step. Part 1: Introduction

Intro to article series

In this serie of articles I’m going to provide step by step instructions for setting up a fully working development environment for your Magento (or actually ANY PHP5) development project.
I start from the lowest level – from choosing the hardware and the article series will be finished with tips for tuning your Nginx, PHP5 and MySQL (MariaDB) to squeeze out the maximum performance that you need whole developing Magento.

About naming things

Here and there I call the database server as “MySQL” but what I mean is MariaDB 5.5.
Here and there I may call PHP5-FPM “PHP5” for the sake of shortness.
When I say PHP or PHP5 I mean the most recent version PHP 5.5.
OSX – I mean the current official version 10.8 Mountain Lion as of time of writing this post.
Linux – I mean Debian 7 – “Wheezy”.

Words of warning or The Disclaimer

First – nobody is (unfortunately:)) paying me anything to promote their software. All choices have been made based on my loooong experience in this industry and some research I’ve done before and during writing if this article series. I’ll try to provide some more justification of choices made under every section for every important component.
Everything described in this blog must be taken with caution. I mean – think twice (and measure 9 times!) before you put anything to production environments. I don’t take any responsibility or blame for data loss or loss of your job or whatever loss that may or may not be caused by following my instructions. In short – I write whatever I want and you’re the one that must think before executing anything written here. This is called “talupojatarkus” or “common sense” in Estonia🙂

Now when this is clear let’s get the hands dirty!

The structure of articles

I’ll give you instructions for 2 major development platforms – Linux and OSX. In general these 2 are perfect platforms for web development. One is free and open source while the other is commercial yet very good.  So you have choice.
The articles are organised like this where all major bullets are articles:

  1. Introduction (current article)
    1. A few words about suitable hardware
    2. No promises made but still … (planning following articles)
  2. Underlying systems and processes
    1. Development process
    2. Version control process
    3. Choosing hardware and installing operating system
    4. Remove unwanted stuff
    5. Command line, Terminal app
  3. Naming and structure
    1. Development folders
    2. Development hostnames (virtual hosts)
    3. Usernames, passwords
    4. SSH keys
  4. Components installation & configuration
    1. MariaDB
    2. PHP5-FPM
    3. Socket or TCP?
    4. Nginx
    5. a special file /etc/hosts
    6. Openssl for HTTPS
    7. keys, self-generated certificates
    8. optional – phpMyAdmin
  5. Source Code Version Control
    1. git …
    2. … or svn. No and why?
    3. Bitbucket
    4. Github
    5. Beanstalk
  6. Security
    1. Bind MySQL to localhost
    2. Keep your sites closed when in public WIFI-s
    3. Use HTTPS
  7. Installing Magento
    1. create database, db user and password
    2. file permissions
    3. special folders
    4. .gitignore
    5. useful utilities
  8. Choosing the IDE
    1. Netbeans
    2. PHPStorm
    3. Others (Komodo, Zend Studio)
  9. Setting up xDebug for debugging your solution
    1. “Local” remote debugger
    2. Breakpoints
    3. Call stack
    4. Variables
    5. Remote debugger
  10. Performance tuning, tips and tricks
    1. MariaDB
    2. Nginx
    3. PHP5-FPM
I leave myself freedom to make improvements to the topics while the serie is developing. 

A few words about suitable hardware

Currently I use Macbook Air with 4 GB RAM and I must say it’s too little. Magento is a big and hungry beast , all IDE-s are in hunger for more and more memory (Java!) and when you want to use xDebug then you’re pretty much stuck. So MB Air can almost do with SSD saving the day but my next machine will be something with 8 GB of RAM or more. I guess nobody’s talking about desktops nowadays any more, right? Still – an iMac is good choice in case you don’t need to move around to much. Since I do a lot of consultation and solutions architecture I have to be in the meetings and it would be a bit tedious to haul an Imac or – even worse – a PC with a monitor around, right…:) OK, I’m joking but in short – a good laptop is a good choice nowadays. Choose a PC laptop or a Mac laptop based on budget or whatever the reasons are – I’ll provide instructions for both.

No promises made but still …

As a busy guy I cannot make big promises but I’ll do my best to provide you a minimum 1 article a week. Hopefully I can do better than this. Feedback is very much appreciated as my ultimate goal is to develop a solution that would be kind of standard for all Magento development projects that are yet to be launched. So all help and feedback and comments are welcome.

Provide feedback

Here’s the first request to you, dear readers – please provide your wishes in the comments about what you’d like to read about in this serie or if there’s anything important that I forgot from the TOC. Thanks!

A mindnote regarding Zend 2.0 and interfaces

In Zend FW 2.0 the standard src folder (and namespace) structure is the following:

src\MyNamespace\Model - models
src\MyNamespace\Controller - controllers
src\MyNamespace\Helper - helpers, utilities

However I wanted to follow the same logic for interfaces I failed. The following does not work:
src\MyNamespace\Interface - interfaces

That’s because “Interface” (and interface) is a reserved keyword in PHP.

Mindnote here for next time –  I have to make exception for interfaces and name my namespace (folder) for interfaces in plural:
src\MyNamespace\Interfaces

Digital carbage and how to reduce it

Digital carbage

I’m afraid we will drown into digital carbage one day. That day is not necessarily in the far future but it can be this week or next.
One of the biggest soure of digital carbage is corporate e-mails with attachments. In every corporation, in every project there are working groups that still (in the year 2013) share a lot of information via e-mail. These people often add information to e-mails in form of attachments that makes it worse.

Attachments and versions

The problem with sending e-mail attachments is that when you send it to n persons you would instantly create at least n+2 different versions of the file. The situation is even worse if you added multiple attachments to your e-mail that you’re going to send – let’s say to 5 colleagues (3 from your office + 2 contractors or whoever).

By doing that you will create following versions of your file(s):
– one in each recipient’s mailbox (ouch – which one is original now?) – that is n
– one on your harddisk (so-called original) – that is n+1
– one in your sent-mail folder (is that original?) – that is n+2

When each recipient saves the attachment(s) from e-mail to harddisk more versions will be added.

My question is – which version is correct?

The solution

The solution is elegantly simple – use cloud services and shared documents. There are a number of such services while most popular are Google Drive and Microsoft Skydrive. Even Dropbox would work however it could impose some file locking issues. I know about corporate policies and rules (I worked at Ericsson for many years) but this is no excuse for polluting the digispace.
One document in one place shared to those who have to work with that. Easy. Simple. Done.
Any thoughts on that?

Logging PHP errors in the same folder with Magento error logs

Here’s another post regarding logging…

How to log PHP errors to the same folder than Magento error logs?

It’s quite simple with Nginx and PHP5-FPM. In Nginx virtual host configuration file there is quite likely block like that:

location ~ .php$ {
if (!-e $request_filename) { rewrite / /index.php last; }
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_param HTTPS $fastcgi_https;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param MAGE_RUN_CODE ee;
fastcgi_param MAGE_RUN_TYPE website;
fastcgi_read_timeout 600;
include /etc/nginx/fastcgi_params;
}

In order to make PHP error logs go to the same folder than Magento’s (/var/log/… add following lines to your virtual host configuration block:

    fastcgi_param   PHP_VALUE "log_errors=on";
fastcgi_param PHP_VALUE "display_errors=off";
fastcgi_param PHP_VALUE "error_log=$document_root/var/log/php_errors.log";

These are standard PHP directives.
log_errors – this one states that erros must be logged
display_errors – it’s always good to keep it off anywhere else than in your local development machine
error_log – this does the trick. It forces PHP logs for current virtual host to var/log folder under your Magento instance folder.

So the final block should look like this:

location ~ .php$ {
if (!-e $request_filename) { rewrite / /index.php last; }
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_param HTTPS $fastcgi_https;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param MAGE_RUN_CODE ee;
fastcgi_param MAGE_RUN_TYPE website;
fastcgi_read_timeout 600;
fastcgi_param PHP_VALUE "log_errors=on";
fastcgi_param PHP_VALUE "display_errors=off";
fastcgi_param PHP_VALUE "error_log=$document_root/var/log/php_errors.log";
include /etc/nginx/fastcgi_params;
}

Restart nginx and there you go. You can now tail or less the logs from /var/log/php_errors.log

Magento solution development in distributed environment

Background

Magento solutions get bigger and more complex, at least here at Eepohs. Magento development requires multiple displines like Frontend Development (JS, CSS, HTML, XML etc), Backend Development (PHP, JS, XML etc), System Integration (JSON, XML, REST, SOAP, (s)FTP etc…). So it’s quite obvious that Magento development is rather teamwork than a play of single freelancers. Freelancers can take your solution to some limit but sooner or later you need to hire a team that can support solution and take it further on.

Tools

There are a couple of tools that are crucial for teamwork and especially important when your team is not sitting together but is distributed in the room and possibly in time also. The tools for successful teamwork are:
– issue management tool – JIRA
– version control tool – GIT
– communication tool – Skype
– an IDE – Netbeans

Processes

When you have more than 1 developer you need a process. In fact you’d need a process when you’re working alone, too but it’s not that stringent in this case.
You have several processes while doing development:
– version control process and strategy
– issue management process
– general workflow
– development itself – writing code, chasing bugs, debugging etc.
– quality assurance (QA)
– deployment process

The latter seems to be quite tough because there’s a lot to be deployed between different stages and instances.
There are 4 stages (and n different Magento instances in each) in Magento development cycle
– development (local developer machine, local server)
– testing (internal testing by the QA team)
– staging or pre-live (pre-live testing by the QA team and the customer)
– live

Deployment of changes

 

A note about popular alternative – Subversion

Subversion is not good for Magento development for 2 reasons:
  1. it must be connected to central server. So you cannot version your files while working offline
  2. svn creates its own special control folder .svn with a lot of extra files inside every folder in Magento. Magento codebase is huge and svn creates a very big extra overhead for your filesystem. It makes it painfully slow and tedious.
  3. … I won’t start rant about merging in svn and git here…:)
  4. And yes – I have used svn, A LOT. So I know what I’m talking about:)

Magento specific issues a.k.a the Bad News

Attribute sets and attributes

How to manage attribute set and attribute migration between 4 stages described above?

The 2 above are actually quite simple to resolve. Magento offers excellent tools for that – install/update scripts. Just add your attributes and sets via install/update scripts and they will get to all environments automatically when code is deployed there.

Stores and Content

Store is the most crucial entity. Everything is depending on that, even configuration and content.

Configuration

That’s a tough one. How to version configuration? How to move conf changes from one Magento to the other?

Magento Deployment tools a.k.a the Good News

There’s a cure for all these issues above, though… I’ll shed some light to available tools in the upcoming posts.

A few thoughts about code quality

While working with Magento I see a lot of 3rd party software (Magento extensions). I must say that the code quality in general is very poor. It’s not that poor as in WordPress in the old days but it’s bad. I’m not even talking about following Magento Coding Standard. I’m talking about simple and basic things:

  1. PHP code that produces ERR in Magento log is to be considered broken. 
  2. PHP code that produces WARN in Magento log is to be considered nearly broken and dangerous.
  3. HTML code that does not validate is to be considered broken.
  4. JS is simple – if some JS is broken on they thru parsing then very likely the rest of it (all other libs, scripts) won’t be parsed correctly. It’s very visible in the frontend.
I cannot accept it. 
What to do? Educate yourself, check your code, use CodeSniffer, write Unit Tests and check the logs.

Write better log when developing Magento extensions

I’ve noticed that just a few developers add proper logging to their Magento extensions and libraries. Also – Magento core team is not part of those developers. Their logging habits are quite poor. I’m sorry, guys, but Magento is really bad when it comes to logging. Compare it to any J2EE project with proper Log4J set up and you’ll see the difference. Sometimes it’s good to learn from those old n’ bald Java guys, also:) OK, so much ranting this time… Now to the logging.

Is I said – Magento gives you tough times when it comes to debugging and searching for problems. Basically it just does not output any useful info to log files. It makes our – developers’ – life hard.

It is in our own hands to do it better because there are tools existing in Magento but these are regrettably underused.

What needs to be logged?

First – all problems and errors.
Whenever you use try/catch then please do write exception to log. Please do write all requests to API-s and responses. This would really help. There’s a special helper for logging exceptions: Mage::logException(Exception $e)
Use at least that one although it just spits randomly formatted Exception to var/log/exception.log. I recommend to use the solution below for exception logging also. Just change the Zend log priority to Zend_Log::CRIT or Zend_Log:ERR.

Second – debug log. When you develop anything then it’s really good to log output of some methods, input parameters that come from Ajax queries and so on. So it’s partly your little helper during development and it’s also part of good coding practices.

Here I have an advice for you about how to log. Do it like this and I’ll exlain why:

Mage::log(sprintf('%s(%s): %s', __METHOD__, __LINE__, print_r($var, true)), Zend_Log::DEBUG, 'mymodule.log');

Now I explain the format. After you have written a considerable amount of code you start noticing that you don’t need some debug lines any more. When you do logging without __METHOD__ and __LINE__ then you need to search thru all your source to find where the f*** did you add that line that outputs “OK” to log… And of course it just helps to discover problems quicker when you see __METHOD__ (which contains class (==file) information) and the line number in the code.
$var in print_r is your variable that you want to log. Be careful with large Magento objects! Because Magento object model is built so that it contains almost everything in every object and does that recursively then it would kill your PHP. Be warned!
Then there’s Zend_Log::DEBUG which is log priority level. Magento does not do smart logging that would consider log priority levels. But at least the log priority is displayed in the log file because Magento’s self log format is like that:
$format = '%timestamp% %priorityName% (%priority%): %message%' . PHP_EOL;

And the last parameter is file name. I support module-based logging. When I’m in a good mood I do logging to 2 files: my module’s log file and system.log. So the system.log would contain ALL logs and my module’s log file would contain logs from my module only.

Third – when you want to get really anal regarding all possible issues in your code then you want to log all notifications, too.

When in trouble then there’s never too much log. It’s really easy to set up log rotation in Linux or OSX systems so the large files aren’t a problem (now I gave you an idea for a good Magento extension;))

Upcoming topics in Measure9

In order to keep interest high I’d like to shed some light to upcoming topics at Measure9:

  • Magento Unit testing and Continuous Integration with Atlassian Bamboo CI
  • Automated acceptance testing in a Magento Development Project (w behat, mink, Gherkin)
  • Syncing content and configuration between Magento instances
  • Tools used in Magento development projects
  • Processes that lead to successful Magento projects
  • and much more…
Thx,
Sven \m/