Difference between revisions of "Ruby Bindings for E-Commerce Tutorial"

Revision as of 17:32, 25 May 2009

One of the main things that people often want to do is to integrate Directed Edge's recommendation engine into their online store. Many of those stores are using Ruby already, so we decided to give a very concrete walk-through of doing recommendations based on previous customer purchases starting from exporting data all the way to keeping it in sync.

If you want to see all of the code in one go, you can get it in full syntax-highlighted glory here: ExampleStore Class, or download it here. You'll also want our Ruby bindings, which you can get from GitHub here (download).

Creating the testing database

You've probably got your own site's database that you're more interested in, but if you want to follow along in the code, you can use the store data that we randomly generated. We created 2000 users with 500 products and had each of those users "buy" between 0 and 30 products.

For that we created three very simple tables. Since we don't care about the other properties of the customers or products, all that those tables contain is an ID. The purchase table just contains customer and product columns.

It really can't get much simpler than that. You can get a dump of the database here. If you have a local MySQL running, you can create and import the database with these commands:

$ mysql --user=root

You should now be at the MySQL prompt:

mysql> create database examplestore;
mysql> create user 'examplestore'@'localhost' identified by 'password';
mysql> grant all on examplestore.* to 'examplestore'@'localhost';

Now back at the command line do:

$ mysql --user=examplestore -p examplestore < examplestore.mysql

Unless you changed the password above, the password is just password. You've now got the same data that the examples use imported to a database called examplestore.

Setting up the Ruby plumbing

Here we just do the standard ruby hashbang and import three modules. Ruby gems is required since we've pulled in activerecord via gem. ActiveRecord is the object relational mapping that is standard with Ruby on Rails and we'll be using it to access our MySQL database. And finally, we use the [Ruby bindings for the Directed Edge API.

#!/usr/bin/ruby

require 'rubygems'
require 'activerecord'
require 'directed_edge'

Setting up ActiveRecord

ActiveRecord handles connecting to the database for us and is the mechanism that most Ruby web apps use to do the same. To connect with ActiveRecord we have to supply the usual information — database type, host, user name, password and database name.

Based on the database that we imported above, these connection values should work.

ActiveRecord also handles most of the magic of mapping Ruby classes to database tables. It can figure out that the customers table corresponds to the Customer (and the some for products and purchases) just by inheriting from the ActiveRecord base class.

ActiveRecord::Base.establish_connection(:adapter => 'mysql',
                                        :host => 'localhost',
                                        :username => 'examplestore',
                                        :password => 'password',
                                        :database => 'examplestore')

class Customer < ActiveRecord::Base
end

class Product < ActiveRecord::Base
end

class Purchase < ActiveRecord::Base
end

In place of Customer, Product and Purchase you'll want to substitute in the values that correspond to your database.

ExampleStore class and constructor

We call the class that we're working with ExampleStore. Again, you'll want to change that to fit your needs. Just as a reminder, the full source of the class is here.

The interesting bit here is the connection to the Directed Edge database. Since several methods in the class use the Directed Edge database, we just set up the connection once. This, fairly obviously, assumes that your user / database name is examplestore and your password is password.

class ExampleStore
  def initialize
    @database = DirectedEdge::Database.new('examplestore', 'password')
  end

Exporting your data to a Directed Edge formatted XML file

Communication with the Directed Edge database is all done with our XML Format. The Ruby bindings make it easy to create files in that format that you can later import to your Directed Edge database.

Basic algorithm

• Create an instance of DirectedEdge::Exporter to export the items you're creating
• Loop through all of the customers in your store, creating a DirectedEdge::Item for each customer, and marking it as a customer by giving it a customer tag

• While looping through the customers, for each customer, also loop through all of their purchases, creating a link from the user to the purchased products
• Export each of those items, with the freshly created links and tags in place

• Loop through all of the products in your store, creating a DirectedEdge::Item for each product, and marking it as a product by giving it a product tag

• Export each of those items

• Tell the exporter that we're done and it can finish up the XML

ActiveRecord-fu

For those not coming from a Rails background, things like the code to find all purchases matching a given customer ID might seem a bit off the wall:

Purchase.find(:all, :conditions => { :customer => customer.id })

That little thinger there does just what we described above — produces a list of all rows in the purchases table that have the customer ID specified. ActiveRecord also automatically creates class accessors for each column of a table, so the .id there just corresponds to that column in the customers table.

Encoding product and user IDs from the database

In the MySQL tables the product table has a simple integer id field as does the customer table. We want to encode those as unique identifiers for the Directed Edge database, which doesn't have separate tables for different item types.

So what we do is take the integer ID and prepend the words product and customer. So the row from the MySQL customer table with the ID 1 becomes customer1. The product from the MySQL product table with ID 2 becomes product2. It's not exactly rocket surgery.

The code in the comments should help you with more fine-grained details.

Output

At the end you'll have a file called examplestore.xml in the directory that you run the script. You'll naturally want to customize that to put it somewhere more appropriate (like, say, /tmp) when incorporating this with your own site.

Implementation

  def export_from_mysql

    # Use the handy Directed Edge XML exporter to collect store data up to this
    # point
    exporter = DirectedEdge::Exporter.new('examplestore.xml')

    # Loop through every customer in the database
    Customer.find(:all).each do |customer|

      # Create a new item in the Directed Edge export file with the ID "customer12345"
      item = DirectedEdge::Item.new(exporter.database, "customer#{customer.id}")

      # Mark this item as a customer with a tag
      item.add_tag('customer')

      # Find all of the purchases for the current customer
      purchases = Purchase.find(:all, :conditions => { :customer => customer.id })

      # For each purchase create a link from the customer to that item of the form
      # "product12345"
      purchases.each { |purchase| item.link_to("product#{purchase.product}") }

      # And now write the item to the export file
      exporter.export(item)
    end

    # Now go through all of the products creating items for them
    Product.find(:all).each do |product|

      # Here we'll also use the form "product12345" for our products
      item = DirectedEdge::Item.new(exporter.database, "product#{product.id}")

      # And mark it as a product with a tag
      item.add_tag('product')

      # And export it to the file
      exporter.export(item)
    end

    # We have to tell the exporter to clean up and finish up the file
    exporter.finish
  end

Customizing

It's here that the bulk of the customization work will be needed to map things to your store. For starters you'll need to do a search and replace for Customer, Product and Purchases for them to be the same ActiveRecord subclasses that you defined up at the top of the code and the names of those should correspond to your database tables. You'll also need to switch the attribute names from id, product and customer to whatever the corresponding columns are called in your database. And as mentioned above, you'll want to give the resulting XML file an appropriate home.

You may also want to specify additional tags that correspond to product categories you might later want to filter on — book or album, for instance. Since products can have as many tags as you like, you can create those in addition to the product and customer tags or use your own scheme entirely.

Importing the XML to the Directed Edge servers

After all that muck above, this one's nice and easy. Once we've created the XML file using DirectedEdge::Exporter, it's easy to import to our Directed Edge database. Here we're using the DirectedEdge::Database instance that we created in the constructor.

  # Imports the file exported from the export method to the Directed Edge database

  def import_to_directededge
    @database.import('examplestore.xml')
  end

The only part you should have to modify here is making the file path correspond to where ever you decide to drop you XML export.

Adding new users / products

With these two methods we repeat some of the logic used in the importer:

• Create a new item with the identifier encoded as the type followed by an integer ID
• Add an appropriate tag to identify the item type
• Save the item (which uploads it to the Directed Edge server)

You can choose to update your Directed Edge database incrementally using methods like these or import occasional snapshots using the import / export mechanism. In most cases, doubly so for fast-changing data sets, it makes more sense to keep the Directed Edge database in sync with your local database this way.

  # Creates a new customer in the Directed Edge database that corresponds to the
  # given customer ID

  def create_customer(id)
    item = DirectedEdge::Item.new(@database, "customer#{id}")
    item.add_tag('customer')
    item.save
  end

  # Creates a new product in the Directed Edge database that corresponds to the
  # given product ID

  def create_product(id)
    item = DirectedEdge::Item.new(@database, "product#{id}")
    item.add_tag('product')
    item.save
  end

If you defined your own scheme for encoding items, you'll need to update that appropriately; the same goes for tags.

Adding a new purchase

As noted in the API Concepts, a purchase for us is just a link between two items, here a customer and a product. So, again, we encode the product and user names, create a new item corresponding to the customer and create a link between that item and the product's identifier.

Like we saw with adding new customers / products to the database, it's usually best to issue these calls when your database is updated so that the data on our servers reflects the data in your database.

  # Notes in the Directed Edge database that customer_id purchased product_id

  def add_purchase(customer_id, product_id)
    item = DirectedEdge::Item.new(@database, "customer#{customer_id}")
    item.link_to("product#{product_id}")
    item.save
  end

As usual, if you defined another means of mapping product integer IDs to Directed Edge item item identifiers, that will need to be reflected here.

Finding related products

Now we get to the part where we deliver our first kind of recommendations — related products. Some of the distinction between related and recommended items is discussed in the REST API documentation.

So, for products we want to find related products. So we issue a related query on the product ID we're interested in and specify that we're only interested in things with the product tag. The argument to related is a set of tags, so we give it an array (and Ruby sorts out turning that to a set). In this case we only have one item in that array.

We get back a list of identifiers and want to map those back to database IDs, so we substitute back out the product part of the strings and get just the integer part and map that back to an array.

  # Returns a list of product IDs related to the given product ID

  def related_products(product_id)
    item = DirectedEdge::Item.new(@database, "product#{product_id}")
    item.related(['product']).map { |product| product.sub('product', '').to_i }
  end

Once again, any changes you've made to the way that that items are encoded and tags you're using will need to be reflected here.

Delivering personalized recommendations

Much of the set up here is the same as related products above, except that now we're working with a customer ID rather than a product ID. The stripping out of the product part of the ID and mapping it back to a list of integers is the same.

  # Returns a list of personalized recommendations for the given customer ID

  def personalized_recommendations(customer_id)
    item = DirectedEdge::Item.new(@database, "customer#{customer_id}")
    item.recommended(['product']).map { |product| product.sub('product', '').to_i }
  end

Let's give it a spin

So, now we've finished defining our class, let's do something with it!

The example below shows:

• Last line of the class definition (end)
• Exports the MySQL database to a file
• Sends that file up to the Directed Edge servers
• Creates a new customer (with ID 500), updating the MySQL database is left up to the reader
• Creates a new product (with ID 2000)
• Creates a new purchase (with the customer and product we just created)
• Retrieves and prints products related to the product with the (MySQL) ID of 1
• Retrieves and prints personalized recommendations for the user with the (MySQL) ID of 1

end

store = ExampleStore.new

# Export the contents of our MySQL database to XML

store.export_from_mysql

# Import that XML to the Directed Edge database

store.import_to_directededge

# Add a new customer

store.create_customer(500)

# Add a new product

store.create_product(2000)

# Set that user as having purchased that product

store.add_purchase(500, 2000)

# Find related products for the product with the ID 1 (in MySQL)

store.related_products(1).each do |product|
  puts "Related products for product 1: #{product}"
end

# Find personalized recommendations for the user with the ID 1 (in MySQL)

store.personalized_recommendations(1).each do |product|
  puts "Personalized recommendations for user 1: #{product}"
end

See also

• ExampleStore Class - full source in one place of all of the examples covered here
• API Concepts - a refresher on the concepts at work here
• REST API and XML Format - A peek into what's going on behind the scenes