ScarabConnector for Magento

Getting Scarab Cloud services to your Magento webshop takes only a few clicks with ScarabConnector Community Edition.

Compatibility: 1.4, 1.5, 1.6, 1.7
Requirements: Registration for Scarab 30-day Free Trial

Install ScarabConnector for Magento

Log in to your Magento Admin and install the extension through MagentoConnect.


ScarabConnector for Magento requires cron to carry out the periodical exports of the product catalog. This means you need to setup Cron on your Magento system. Here's a link to Magento's Wiki pages on how to set up a Cron job:

ScarabConnector also uses timthumb.php, a small third-party script licensed under the GNU GPL v2 license and included in the ScarabConnector distribution.

1. Start data collection

Enable ScarabConnectorUnder System > Configuration you should now see ScarabConnector for Magento. Open the General tab and paste your MerchantID in the field provided there. Then change the state of the connector to Enable.

From this point on, the Scarab Recommender is collecting information about all user interactions happening in your webshop, including:

  • product views,
  • add to carts,
  • checkouts,
  • anonymized Customer IDs

The information collected is used to create analytical models for:

  • Related/similar products based on product view relationships
  • Also bought products based on checkout data
  • Personal recommendations based on users' individual browsing and purchase histories

Live Event View and Statistics

The Scarab Dashboard provides 2 tools to monitor the data collection process:

  1. Live Event View is a small listener lets you monitor in real time the user interactions collected by Scarab.
  2. Statistics gives you stats and graphs of webstore traffic and recommender performance data collected by Scarab.
It is recommended that you check and validate the integration using this Integration Checklist

Set up product catalog

ScarabConnector catalog export

Scarab Recommender needs to have up-to-date information about your products in order to carry out analysis and to display recommended items inside the recommender widgets. ScarabConnector creates a daily catalog export for this purpose, which includes only publicly available information about your products, such as itemID, item URL, image URL, category, price, author/brand.

To create your first catalog, click the Export Now! button in your ScarabConnector configuration panel. You can check the created scarab.csv file in your /export folder.

To have Scarab fetch a fresh copy of this catalog every day, you need to log in to your Scarab Dashboard, go to My Accounts, and enter the URL for the catalog. This URL is composed of your webshop domain, followed by the catalog file path:

To verify regular catalog updates, log back to Scarab Dashboard in a few days, and check Product Catalog. You should see a series of entries in the Catalog list, each with a regular daily time stamp.

Upload historical sales data

To speed up the Scarab Recommender's learning process, you have the option to upload historical sales data available in your store logs for analysis. This is an optional, one-time operation, as all future data is collected by the data collection scripts.

Simply click the Export Now! button which will create a scarabsales.csv in your /export folder. Take this file, and upload it to Scarab Dashboard > Historical Sales Data.

Once everything looks OK you can lay back for a few days/weeks, as now starts the warmup phase. This means our system will spend 4-6 weeks just learning about your customers' shopping habits, and building behavior models. During this period, you will receive an Analysis Report every two weeks, keeping you informed about progress.

2. Display Recommendations

Turn on widgets

The last Analysis Report you receive will instruct you to bring your recommender widgets live. We recommend you check the product recommendations generated by Scarab at this point by logging in to the Scarab Dashboard, and using Item Search to query our recommender models for specific products in your catalog.

It must be noted that Scarab services have different learning curves. Related product recommendations are typically the first to be ready for live service, as these analytical models rely on more abundant product view and add to cart data. On the other end, some services like our ALSO_BOUGHT service usually take much longer to achieve satisfactory coverae as they rely on much scarcer data (items bought together).

To bring the recommender widgets live, go to System > Configuration > ScarabConnector for Magento > Recommender widgets and simply enable the services that are ready for live service.

Bringing the recommender widgets live also means the start of your 30-day Free Trial of Scarab Cloud

Integrating the PERSONAL Widget manually

The PERSONAL recommender service is special in that it is not tied to specific pages, but a given user. Therefore, you have more freedom in defining where to display these recommendations.

We have provided easy switches for the standard Cart and My Account pages on the ScarabConnector configuration page, but the PERSONAL service can be inserted in any CMS page.

So we provide you with a simple block script that you can insert in your CMS page.

	{{block type="catalog/product_list" itemnumber="4" template="scarabresearch/personal2.phtml"}}

You can use any number of PERSONAL widgets, and define parameters and templates for each separately. Simply make copies of the personal2.phtml referenced in the script.

Once you have brought your recommender widgets live, you will be able to monitor recommender performance on the Statistics page of the Scarab Dashboard. Additionally to overall webshop add to carts and total sales, the stats showing the number of impressions, average CTR, cart contribution and Revenue contribution figures and rates will also come live.

3. Customizing widgets

ScarabConnector inserts a basic content block, leaving you free hand in designing its container box. Still, you may wish to customize the inside of the recommender widget too. You have 3 ways to customize the recommender widget:

  1. Set the number of items displayed in the widget using the dropdown provided by ScarabConnector interface.
  2. Customize scarabresearch.css to change widget styles
  3. Customize the rendering template of the widget. This includes changing the widget title, or adding/changing the information displayed along with each product (eg. adding the price, or the brand of the product to the thumbnails displayed).

    To achieve this, you'll need to refer to the Customizing the Recommender Widget page for details, and edit your template files accordingly.

    Your template files are located in your /var/www/app/design/frontend/default/default/template/scarabresearch folder:

    • box_view.phtml contains the template code for Product Page widgets
    • box_cart.phtml contains the template code for Cart Recommender widget
    • personal.phtml contains the template code for the Cart and Dashboard PERSONAL widgets
    • personal2.phtml contains the template code for the PERSONAL widget inserted manually. If you want to use multiple PERSONAL widgets, simply make a copy of the personal2.phtml file and reference it in the script inserted manually.
If you use a customized template, do not forget to make a backup of the template files before next upgrading your ScarabConnector, because the update overwrites these files.

Setting thumbnail size in widgets

Custom thumbnail size

ScarabConnector offers the use of timthumb.php for the purpose of resizing the product images displayed in the recommender widgets. By default, the widget displays the actual size product images, which are scaled to size by the browser.

To economize on bandwidth, and better control the widget behavior, you may enable the thumbnail resizing by selecting "Yes" in the dropdown menu, and entering the size you want for thumbnail width.