Pacemaker for Magento — The most powerful Magento middleware solution for the leading ecommerce platform

The ecommerce platform Magento, no matter if the open source or the commerce edition, is already equipped with a standard solution for the export and import of data. So, why should you still use Pacemaker as a third-party tool? Besides the possibility to model, visualize and control processes, there are many other good reasons for using Pacemaker as powerful middleware connecting ERP, CRM or PIM solutions with Magento.

Pacemaker, THE middleware solution for Magento

Pacemaker offers significant advantages over the Magento standard functionalities and all other solutions on the market when it comes to import & export. However, these advantages are mainly used when larger data volumes are involved and therefore longer import runtimes are required. On closer examination, the following reasons for using Pacemaker become apparent.

• Extended functionalities compared to the standard
• High data throughput and low memory consumption during imports
• Security and scalability when operating large commerce systems
• Transparency & visualization of running processes
• Standardization and the resulting savings potential

Extended functionality

Pacemaker extends the Magento standard functionality regarding export and import by the possibility to import additional entities, especially attribute sets, attributes and categories. In many projects, especially when connecting to third party systems such as PIM or ERP, it is almost essential that these entities can also be provided and imported via an external system. In addition, in almost all projects it is necessary that stock levels and prices have to be transferred to the store in relatively close time intervals. Pacemaker explicitly offers the possibility to use CSV files that are reduced to a minimum. This increases performance and is easier to implement at existing systems.

On the other hand, there is almost always the requirement that orders must be transmitted back to the ERP. Pacemaker provides a generic plug-in for this purpose, which enables the transmission e.g. to SAP via BAPI with support for the OData protocol. Based on this, adjustments and extensions can be implemented quickly, easily and cost-saving.

In addition to this functionality, Pacemaker comes with so-called process pipelines, a design pattern taken from systems such as Gitlab, which enables the ecommerce operator controlling the processes and model dependencies between them. Especially in the case of Magento, which relies on index tables due to the EAV model and the resulting complex data structure, it must be ensured that import and indexing processes do not collide with each other, otherwise inconsistent data may occur.

Performance & memory consumption

Due to its streaming-based architecture, Pacemaker keeps memory consumption to a minimum while increasing speed to the maximum. The actual import takes place in several steps, whereby in the first run only the simple products are created, which form the basis for all further steps. In addition, the necessary information for the creation of additional product types such as Configurable Products is extracted from the original file and exported as CSV artifacts. These artifacts are then processed in the following steps, which can also be performed in parallel if required.

To make sure that performance and memory consumption are within our target range, we measure these indicators for each minor release. The following table gives you an overview of the performance of Pacemaker, based on a real case scenario.

The following legend gives you more informations about the mode used for the import:

• -C: Without Cache
• +C: With Cache
• +CCSD: With Cache + Change-Set Detection
• +CWCSD: With Cache + Cache Warming + Change-Set Detection
• The CSV file to be imported has ~450,000 lines and a size of ~550 MB, the amount of data contained in it is of the following order
• SKUs: ~14,300
• categories: ~100
• Store-Views: ~30
• attributes: ~150
• Option values: ~3.700

The measurement deliberately did not take into account the image import, because the customer has several million images and these have to be re-assigned with each import, which would make the benchmark unusable.

ATTENTION: In general it is very difficult to draw conclusions about the performance in another case from a benchmark, because many factors have an influence on the speed of an import. The actual performance can vary considerably depending on the case.

Security & Scalability

However, in addition to the highest demands on performance and memory consumption, security and scalability are the main focus when connecting third-party systems. Due to the ever faster growing complexity of the system landscape, it is becoming more and more important that the ecommerce operator or the responsible Magento partner retains control over the running processes and can thus ensure that the import and export of data takes place at the desired times, that prioritization is possible via flexible conditions and that processes can be repeated independently if necessary, e.g. if a third-party system fails.

Pacemaker decouples the decision if a pipeline is executed from its actual execution. This is done via the message queue and so-called runners (consumers) which run on any computer and execute the processes asynchronously. Thus, all processes controlled by Pacemaker can be quickly and easily distributed to several computers if required, thus significantly increasing the possibilities for scaling the system.

Transparency & Visualization

Especially with long-running processes, which are almost always triggered by CRON jobs, the ecommerce operator as well as the solution partner lacks an overview of what exactly is running and when. Especially when problems occur, many hours of detective work are often necessary to find the cause and provide a suitable solution. Erroneous data such as incorrect stock levels, which may mean that the products are no longer listed in the store, can quickly lead to high losses.

Pacemaker helps to solve these problems. On the one hand by supporting all parties involved in the development process breaking down the processes into small and independent steps, on the other hand by making the execution of these steps transparent. By visualizing the running and expired pipelines in the Magento Admin and the possibility of viewing the detailed information of each step and repeating it for debugging purposes (if provided data allows it) Pacemaker makes processes visible that were previously invisible.

Standardization & savings potential

The use of Pacemaker has led to a standardized approach to the integration of third-party systems. This standardization, supported by import formats for almost all relevant entities, predefined and out-of-the-box usable pipelines, best practices, a comprehensive and continuously updated documentation as well as a fast support reduces risks, minimizes costs and guarantees a fast ROI.

When we talk about standardization, the Pacemaker context also means that recurring processes are mapped in pipelines. This approach allows them to be reused in any project and to deliver them with Pacemaker. The standardization also leads to the fact that the know-how is recorded and made available to other developers and ecommerce operators. The definition of a pipeline is done in XML format, so it is understandable for non developers and can be managed like source code.

Especially in medium and large projects the use of Pacemaker can help to reduce the project risk and free up budget for other useful functionalities. The 1st-Class Support ensures that developers get fast and directly support solving occured problems.

Our experience with Pacemaker

As one of the leading Magento Solution Partners, we have been using Pacemaker successfully in almost all of our projects for several years. It has been shown that by standardizing our approach, we are able to significantly reduce costs and thus risks when integrating third-party systems, no matter whether ERP, CRM or PIM systems. The budget thus saved can be used for useful functionalities and the best possible user experience.

Of course, developers like to develop their own solutions, which is exciting and fun. This is the question that E-Commerce operators and solution partners have to ask themselves: Is it effective and goal-oriented to reinvent the wheel, in this case tools for the integration of third-party systems in Magento, again and again? Or does it rather make sense to rely on an established and rock solid solution and thereby significantly reduce costs and risk for the project?

With Pacemaker we provide our own customers, but indirectly through our Solution Partner Program all Magento ecommerce operators, with a solution that is already widely used, intensively tested and has proven its advantages in practice over the years. Pacemaker is available in three editions. In addition to the free Community Edition, which is often sufficient for smaller projects, the Professional and Enterprise Editions offer almost unlimited possibilities for larger projects including support.

From our point of view Pacemaker has all the prerequisites to establish itself as THE standard solution for the integration of third party systems in the Magento ecosystem.

You can find more information about Pacemaker, reference cases as well as the free Community Edition at https://pacemaker.techdivision.com.

Pacemaker is powered by TechDivision, one of the leading Magento Partners in Europe!