the Directed Edge Developer Base - New pages [en]

Recommendation parameters

2014-01-03T21:59:54Z

Scott:

''List'' parameters are just a comma separated list of strings, defaulting to an empty list.

{| class="wikitable"
! Type
! Name
! Default
! Description
|-
| float
| ''type''Weight
|
| Specifies the relative weights of various link types. See [[REST API#typeWeight|the description in our REST docs]]
|-
| list
| tags
|
| Only include items which possess the specified tags
|-
| list
| excludedTags
|
| Exclude items which possess the specified tags
|-
| list
| exclude
|
| Exclude the items with the given IDs
|-
| list
| items
|
| For a database-related request (i.e. basket request), specifies the list of items to find related items to; for a normal database GET request, specifies which items should be returned (useful for getting information about a number of items at once)
|-
| boolean
| showReferences
| false
| Show the "references" (back-links) for an item GET
|-
| list
| ignore
|
| Do not include or factor into ranking the listed IDs
|-
| boolean
| excludeLinked
| false (recommended) / true (related)
| Don't include the items directly linked from the query item (i.e. products already purchased by a customer)
|-
| int
| maxResults
| 20
| The maximum number of recommended entries to return
|-
| boolean
| includeProperties
| false
| Include the properties for the item as XML attributes in a related or recommended query
|-
| boolean
| includeTags
| false
| Include the tags for the item as an XML attribute (comma separated) in a related or recommended query
|-
| list
| include
|
| Hand-pick the listed IDs for inclusion in a recommended or related query
|-
| string
| tagOperation
| OR
| Specifies whether the list of tags specified in "tags" should be evaluated using a logical "or" or logical "and"
|-
| enum [ add, subtract, replace ]
| updateMethod
| replace
| Specifies the behavior POSTed updates to items
|-
| list
| itemsToRank
|
| Ranks a specific set of items using the "related" method
|-
| float (0 .. 1.0)
| popularity
| -1.0 (automatic)
| Skews results towards more niche or popular items
|}

Shopify Liquid Examples

2013-10-10T17:43:17Z

Scott:

Here are a few examples of custom styles that can be used in the Directed Edge custom style engine for Shopify.

Note that in general you'll probably want to incorporate the CSS that's here into your site's main CSS file(s), but it's included here to make the examples more complete and usable out of the box.

Also see our [[Shopify Liquid|variable reference]].

== Basic Example (Plain-text output) ==
This basic example demonstrates iterating through different recommendation types (called <tt>groups</tt>) and listing a maximum of five recommended products per group in a nested iteration:

<div class="shopify-custom-style">
<source lang="xml">
{% for group in groups %}
<h3>{{ group.label }}</h3>
<ul>
{% if group.bundle %}
<li><a href="{{ bundle.buy_link }}">{{ bundle.text }}</a></li>
{% else %}
{% for product in group.products limit:5 %}
<li>
<a href="{{ product.url }}">{{ product.title }}</a>
for {{ product.price | money }}
</li>
{% endfor %}
{% endif %}
</ul>
{% endfor %}
</source>
</div>

In the <tt>{% if group.bundle %}…{% else %}…{% endif %}</tt> block we handle the special case of a group being a bundle. In this case we want to have a different link <tt><nowiki>{{ bundle.buy_link }}</nowiki></tt> that adds the bundled products to the basket. Also we want to show a message <tt><nowiki>{{ bundle.buy_text }}</nowiki></tt> that contains the price of the whole bundle. For more information on bundles see the [[Shopify_Liquid|Variable Reference]].

== Default ==

This is a simple, but usable style that's shown by default when you switch to the custom layout mode. It features a simple list of up to 5 products per recommendations type, plus bundles. Products are shown with a simple 5 pixel border. This is a responsive style.

<div class="shopify-custom-style">
<source lang="xml">
<div id="recommendations">
{% for group in groups %}
<div class="recommendations-group">
<h3>{{ group.label }}</h3>
{% if group.bundle %}
<div id="bundle">
<img class="bundle-first"
src="{{ group.products.first.featured_image | product_img_url: 'compact' }}" />
<span class="bundle-plus">+</span>
<img class="bundle-last"
src="{{ group.products.last.featured_image | product_img_url: 'compact' }}" />
<span class="bundle-label"><a href="{{ bundle.buy_link }}">{{ bundle.text }}</a></span>
</div>
{% else %}
{% for product in group.products limit:5 %}
<div class="recommendations-product">
<div class="recommendations-product-image">
<a href="{{ product.url }}">
<img src="{{ product.featured_image | product_img_url: 'compact' }}">
</a>
</div>
<div class="title"><a href="{{ product.url }}">{{ product.title }}</a></div>
<div class="price">{{ product.price | money }}</div>
</div>
{% endfor %}
{% endif %}
</div>
{% endfor %}
</div>

<style type="text/css" media="screen">
#recommendations h3, .recommendations-group {
padding: 0.5em 0;
clear: both;
}
.recommendations-group .recommendations-product {
display: inline-block;
vertical-align: top;
width: 200px;
padding-bottom: 0.5em;
}
.recommendations-group .recommendations-product div {
margin: 0.5em 0;
}
.recommendations-group .recommendations-product img, #bundle img {
border: 10px solid #f6f6f6;
}
.recommendations-product-image {
position: relative;
bottom: 0px;
height: 180px;
}
.recommendations-product-image img {
position: absolute;
bottom: 0px;
}
#bundle > *{
vertical-align: middle;
margin-bottom: 1em;
}
#bundle .bundle-plus {
font-size: 3em;
}
#bundle .bundle-label {
font-size: 1.5em;
display: inline-block;
}
#bundle .bundle-last {
margin-right: 0.5em;
}
</style>
</source>
</div>

== Stylized recommendations with CSS 3 animations ==

The next example shows the real powers of customized templates in Directed Edge for Shopify. This example shows only the product images of the recommended products. If you hover over one of them a neat animation will enlarge that particular item and reveal a product's title, price and also the original price if the product is on sale.
As you will see CSS-code can be directly embedded into the liquid template.

Note hover that: '''JavaScript code will be stripped out before rendering'''.<br>
If you need to use JavaScript it has to be embedded in the Shopify templates. You can set a callback using <tt>{% assign directed-edge-callback = 'myJsFunction' %}</tt> before including the <tt>{% include 'directed-edge' %}</tt> snippet. Since you can set element ids and classes as you need you will be able to easily manipulate the elements later.

<div class="shopify-custom-style">
<source lang="xml">
<style type="text/css">
.recommendations-label {
font-family: Arial, sans-serif;
font-size: 2em;
}

.recommendation-group {
float: left;
display: block;
clear: both;
padding: 5px;
}

.recommended-item {
display: inline-block;
float: left;
text-align: center;
cursor: pointer;
height: 102px;
padding: 3px;
margin: 3px;
border: 1px solid #ddd;
border-radius: 3px;
}

.bundle-plus {
display: inline-block;
float: left;
text-align: center;
cursor: pointer;
height: 102px;
padding: 3px;
margin: 3px;
font-family: Arial, sans-serif;
font-size: 2em;
line-height:4em;
vertical-align: middle;
}

.recommended-image-container {
float: left;
width: 102px;
line-height: 100px;
}

.recommended-image-container img {
vertical-align: top;
}

.recommended-details {
width: 1px;
height: 102px;
float: left;
overflow: clip;
text-align: left;
transition: width 0.2s;
-webkit-transition: width 0.2s; /* Safari, Chrome */
}

.recommended-item:hover .recommended-details {
width: 120px;
}

.rtitle, .rprice {
margin-left: 1px;
text-align: left;
overflow: hidden;
white-space: nowrap;
text-overflow: ellipsis;
}

.sale-price {
text-decoration: line-through;
}
</style>

<div class="recommendations">
{% for group in groups %}
<div class="recommendation-group">
<span class="recommendations-label">{{ group.label }}</span>
<ul>
{% for product in group.products %}
<a href="{{ product.url }}">
<div class="recommended-item">
<div class="recommended-image-container">
<img src="{{ product.featured_image | product_img_url: 'small' }}" />
</div>
<div class="recommended-details">
<div class="rtitle">
{{ product.title }}<br/>
{% if product.compare_at_price_min > product.price_min %}
<span class="sale-price">{{ product.compare_at_price_min | money }}</span>
{% endif %}
{{ product.price | money }}
</div>
</div>
</div>
</a>
{% if group.bundle and product == group.products.first %}
<div class="bundle-plus">+</div>
{% endif %}
{% if group.bundle and product == group.products.last %}
<div class="bundle-plus">
<a href="{{ bundle.buy_link }}">{{ bundle.text }}</a>
</div>
{% endif %}
{% endfor %}
</ul>
</div>
{% endfor %}
</div>
</source>
</div>

Shopify Liquid

2013-10-07T16:27:18Z

Scott:

See also our page with [[Shopify Liquid Examples]].

== Liquid variables supported by Directed Edge ==
=== <tt>groups</tt> ===
The liquid variable groups contain all recommendation types that are available for the requested page. Each group has following properties:
{| class="variable-ref-table"
|-
| label || A formatted label of the group, for example, “Recommended Items”, “Recently Viewed”, etc. Those labels can be customized in our [http://shopify.directededge.com/settings#styling Appearance Settings].
|-
| handle || A machine friendly identifier for the group (e.g. <tt>bundle</tt>, <tt>recommended_product</tt>, etc.)
|-
| products || An array of all of the products in returned for this recommendations group.
The following example would print title and price of all products in all available groups:
<source lang="xml">
{% for group in groups %}
<ul>
{% for product in group.products %}
<li>
<b>{{ product.title }}</b><br />
{{ product.price | money }}
{% endfor %}
</ul>
{% endfor %}
</source>
A detailed description of all product’s properties can be found [http://docs.shopify.com/themes/liquid-variables/product here].
|-
| bundle || A boolean field that indicates if this group is a bundle. This is useful to render custom content for that particular group. The following snippet would display both product images separated by a plus sign and followed by a equals sign and the bundle price:
<source lang="xml">
{% for group in groups %}
{% if group.bundle %}
{{ group.products.first.featured_image | product_img_url: 'small' }}
+
{{ group.products.last.featured_image | product_img_url: 'small' }}
=
{{ bundle.price }}
{% else %}

{% endif %}
{% endfor %}
</source>
|}

=== <tt>bundle</tt> ===
The bundle variable is used to access extra information about bundles, not available in other groups:
{| class="variable-ref-table"
|-
| text || Bundle’s buy message (e.g. “Buy both for $20”) which can be adjusted in the [http://shopify.directededge.com/settings#bundle Directed Edge for Shopify Bundle Settings] page.
|-
| price || The total price of a bundle calculated using the method specified in the [http://shopify.directededge.com/settings#bundle Directed Edge for Shopify Bundle Settings] page. It can be formatted with the price filter just like elsewhere in Shopify templates: <source lang="xml">{{ bundle.price | money }}</source>
|-
| buy_link || A link that will add both products to the shopping cart and will ask the user to pick a variant if multiple variants are available for the bundled products.
|}

=== <tt>product</tt> ===

We support the same variables as Shopify, with the following additions. See their documentation for <tt>[http://docs.shopify.com/themes/liquid-variables/product product]</tt>.

{| class="variable-ref-table"
| buy_link || A link that will add the product to the shopping cart and will ask the user to pick a variant if multiple variants are available for the product.
|}

Shopify Javascript Callbacks

2012-10-23T06:32:46Z

Scott:

__TOC__
== Add a hover effect to the recommendations (jQuery) ==

This changes the text color and the border color of the image simultaneously.

<source lang="javascript">
{% assign directed-edge-callback = 'addHoverEffectToRecommendations' %}
{% include 'directed-edge' %}

<script type="text/javascript">
function addHoverEffectToRecommendations() {
function findIndexInRow(e) {
var children = e.parentElement.children;
for(var i = 0; i < children.length; i++) { if(children[i] == e) { return i; } }
}

function findTitleForImage(e) {
return $(e.parentElement).next().children()[findIndexInRow(e)];
}

function findImageForTitle(e) {
return $(e.parentElement).prev().children()[findIndexInRow(e)];
}

$(".directededge-image-cell").hover(function() {
$("img", this).css("border-color", "#DDD");
$("a", findTitleForImage(this)).css("color", "#FD885D");
}, function() {
$("img", this).css("border-color", "#FFF");
$("a", findTitleForImage(this)).css("color", "#242424");
});

$(".directededge-title-cell").hover(function() {
$("img", findImageForTitle(this)).css("border-color", "#DDD");
$("a", this).css("color", "#FD885D");
}, function() {
$("img", findImageForTitle(this)).css("border-color", "#FFF");
$("a", this).css("color", "#242424");
});
}
</script>
</source>

== Add the items to the DOM tree without any styling ==
<source lang="javascript">
{% assign directed-edge-jsonp = 'showRecommendations' %}
{% include 'directed-edge' %}

<script type="text/javascript">
function showRecommendations(data)
{
var container = document.getElementById("directededge-blocks");
var products = data.related_products;

for(var i = 0; i < products.length; i++)
{
var item = document.createElement("div");
container.appendChild(item);
item.className = "directededge-item";

var imageLink = document.createElement("a");
item.appendChild(imageLink);
imageLink.className = "directededge-image-link";
imageLink.href = products[i].link;

var image = document.createElement("img");
imageLink.appendChild(image);
image.className = "directededge-image";
image.src = products[i].image;

var title = document.createElement("a");
item.appendChild(title);
title.className = "directededge-title";
title.href = products[i].link;
title.innerHTML = products[i].title;

var price = document.createElement("a");
item.appendChild(price);
price.className = "directededge-price";
price.href = products[i].link;
price.innerHTML = products[i].price;
}
}
</script>
</source>

== Truncate recommendation titles (jQuery) ==
<source lang="javascript">
{% assign directed-edge-callback = 'truncateRecommendationTitles' %}
<script type="text/javascript">
function truncateRecommendationTitles() {
var max = 10;
$(".directededge-title-cell a").each(function() {
if(this.innerHTML.length > max) {
this.innerHTML = this.innerHTML.substr(0, max - 3) + "...";
}
});
}
</script>
</source>

Getting started with Spree/Products/Index

2010-11-28T09:18:23Z

Scott: Created page with "<source lang="html4strict"> <% unless edgy_recommended.empty? %> <h3 class="list-type"><%= edgy_message %></h3> <ul class="product-listing"> <% edgy_recommended.each do |..."

<source lang="html4strict">
<% unless edgy_recommended.empty? %>
<h3 class="list-type"><%= edgy_message %></h3>
<ul class="product-listing">
<% edgy_recommended.each do |product| %>
<li id="product_<%= product.id %>">
<%= link_to small_image(product), product %>
<%= link_to raw(product.name + " <span class='price selling'>#{product_price(product)}</span>"), product, :class => 'info' %>
</li>
<% end %>
</ul>
<h3 class="list-type">Our catalog:</h3>
<% end %>
<% content_for :sidebar do %>
<% hook :homepage_sidebar_navigation do %>
<% if "products" == @current_controller && @taxon %>
<%= render "shared/filters" %>
<% else %>
<%= render "shared/taxonomies" %>
<% end %>
<% end %>
<% end %>
<% if params[:keywords] %>
<% hook :search_results do %>
<% if @products.empty? %>
<%= t(:no_products_found) %>
<% else %>
<%= render "shared/products", :products => @products, :taxon => @taxon %>
<% end %>
<% end %>
<% else %>
<% hook :homepage_products do %>
<%= render "shared/products", :products => @products, :taxon => @taxon %>
<% end %>
<% end %>
</source>

Getting started with Spree/Orders/Form

2010-11-28T09:13:02Z

Scott: Created page with "<source lang="html4strict"> <%= error_messages_for :order %> <table id="cart-detail"> <thead> <tr> <% hook :cart_items_headers do %> <th colspan="2"><%= t("i..."

<source lang="html4strict">
<%= error_messages_for :order %>
<table id="cart-detail">
<thead>
<tr>
<% hook :cart_items_headers do %>
<th colspan="2"><%= t("item") %></th>
<th><%= t("price") %></th>
<th><%= t("qty") %></th>
<th><%= t("total") %></th>
<th></th>
<% end %>
</tr>
</thead>
<tbody id="line_items">
<% order_form.fields_for :line_items do |item_form| %>
<%= render 'line_item', :variant => item_form.object.variant, :line_item => item_form.object, :item_form => item_form %>
<% end %>

<% if @order.line_items.empty? -%>
<tr id="none">
<td colspan="5"><%= t("your_cart_is_empty.") -%></td>
</tr>
<% end -%>
</tbody>
</table>

<%= edgy_related_table :horizontal, :max_results => 5 %>
</source>

Getting started with Spree/Products/Show

2010-11-28T08:43:00Z

Scott: Created page with "<source lang="html4strict"> <% @body_id = 'product-details' %> <h1><%= @product.name %></h1> <div id="product-images"> <div id="main-image"> <%= render 'image' -%> </div..."

<source lang="html4strict">
<% @body_id = 'product-details' %>
<h1><%= @product.name %></h1>

<div id="product-images">
<div id="main-image">
<%= render 'image' -%>
</div>
<div id="thumbnails">
<%= render 'thumbnails', :product => @product -%>
</div>

<% hook :cart_form do %>
<div id="cart-form">
<%= render 'cart_form' %>
</div>
<% end %>

</div>

<div id="product-description">
<% hook :product_description do %>
<%= product_description(@product) rescue t("product_has_no_description") %>
<% end %>
<% hook :product_properties do %>
<%= render 'properties' %>
<% end %>
</div>

<%= edgy_related_table :vertical, :max_results => 4 %>

<%= render 'taxons' %>
</source>

Getting started for Java developers

2009-11-23T18:14:26Z

Scott:

{{GettingStarted|Java|Java/dist/com.directededge.jar|You'll also need the included com.noelios.restlet.jar and org.restlet.jar from Java/dist/lib.}}

== Examples ==

Currently there's no ExampleStore example like there are for [[ExampleStore PHP|PHP]], [[ExampleStore Python|Python]] and [[ExampleStore Ruby|Ruby]], but since the bindings follow very closely to the bindings in those languages, it's probably still worth your time to glance at one of those.

You'll also find several files in <tt>Java/test</tt> in the bindings that should give you a bit of an idea of how the Java API fits together.
{{FurtherReading}}

Getting started for Ruby developers

2009-11-23T18:07:52Z

Scott: Add gem.

{{GettingStarted|Ruby|Ruby/directed_edge.rb|You'll need to make sure you have rubygems and the rest_client Gem installed. Alternatively, you can install our [http://gemcutter.org/gems/directed-edge Gem] directly with <tt>sudo gem install directed-edge</tt>.}}
== Tutorial ==

* [[Ruby Bindings for E-Commerce Tutorial]] provides a detailed step-by-step walkthrough of how to get up and going with a store — it's based on the same code as below, but with lots of details.

{{ExampleStore|ExampleStore Ruby}}
{{FurtherReading}}

ExampleStore PHP

2009-11-23T17:53:30Z

Scott:

<source lang="PHP">

/*
* Ok, so the basics of how this thing works:
*
* It assumes that you have a local database for your store, and that store
* has a "customer" table and "products" table. In each of those tables
* you have a unique ID that corresponds to each customer and product,
* respectively.
*
* It also assumes there's a third table, named "purchases" that has a list of
* things purchased by customers. Basically a mapping from the customer ID to
* the product ID. All of this is pretty standard store stuff.
*
* So, then what this class does is it handles:
*
* - Getting that data from those tables over to Directed Edge
* - Doing incremental updates (adding customers, products, purchases)
* - Finding products related to a given product
* - Finding personalized product recommendations for a customer
*
* This is really just the starting point; there are some other exciting things
* that can be done with the API once you've gotten your feet wet.
*/

define('DB_HOST', 'localhost');
define('DB_USER', 'examplestore');
define('DB_PASS', 'password');
define('DB_NAME', 'examplestore');

define('DIRECTEDEDGE_USER', 'examplestore');
define('DIRECTEDEDGE_PASS', 'password');

define('EXPORT_FILE', 'examplestore.xml');

class ExampleStore
{
private $database;

public function __construct()
{

mysql_connect(DB_HOST, DB_USER, DB_PASS);

if(!mysql_select_db(DB_NAME))
{
throw new Exception("Could not connect to DB.");
}

$this->database = new DirectedEdgeDatabase(DIRECTEDEDGE_USER, DIRECTEDEDGE_PASS);
}

/**
* Export the list of products, purchases and customers to an XML file that
* we can later push to the Directed Edge webservices.
*/

public function exportFromMySQL()
{
$exporter = new DirectedEdgeExporter(EXPORT_FILE);

foreach($this->getProducts() as $product)
{
$item = new DirectedEdgeItem($exporter->getDatabase(), 'product' . $product);
$item->addTag('product');
$exporter->export($item);
}

foreach($this->getCustomers() as $customer)
{
$item = new DirectedEdgeItem($exporter->getDatabase(), 'customer' . $customer);

foreach($this->getPurchasesForCustomer($customer) as $product)
{
$item->linkTo('product' . $product);
}

$exporter->export($item);
}

$exporter->finish();
}

/**
* Import the file that we created using exportFromMySQL to the Directed Edge
* webservices.
*/

public function importToDirectedEdge()
{
$this->database->import(EXPORT_FILE);
}

/**
* Create a customer in the Directed Edge database that corresponds to the
* customer in the local database with $id.
*/

public function createCustomer($id)
{
$item = new DirectedEdgeItem($this->database, 'customer' . $id);
$item->addTag('customer');
$item->save();
}

/**
* Create a product in the Directed Edge database that corresponds to the
* product in the local database with $id.
*/

public function createProduct($id)
{
$item = new DirectedEdgeItem($this->database, 'product' . $id);
$item->addTag('product');
$item->save();
}

/**
* Create a purchase in the Directed Edge database from the product with
* the local database IDs $customerId and $productId.
*/

public function addPurchase($customerId, $productId)
{
$item = new DirectedEdgeItem($this->database, 'customer' . $customerId);
$item->linkTo('product' . $productId);
$item->save();
}

/**
* Returns a list of related product IDs for the product ID that's passed in.
*/

public function getRelatedProducts($productId)
{
$item = new DirectedEdgeItem($this->database, 'product' . $productId);
$related = array();

foreach($item->getRelated(array('product')) as $item)
{
$related[] = str_replace('product', '', $item);
}

return $related;
}

/**
* Returns a list of recommended products for the customer ID that's passed in.
*/

public function getPersonalizedRecommendations($customerId)
{
$item = new DirectedEdgeItem($this->database, 'customer' . $customerId);
$recommended = array();

foreach($item->getRecommended(array('product')) as $item)
{
$recommended[] = str_replace('product', '', $item);
}

return $recommended;
}

private function getCustomers()
{
$result = mysql_query("select id from customers");
$customers = array();

while($row = mysql_fetch_row($result))
{
$customers[] = $row[0];
}

return $customers;
}

private function getProducts()
{
$result = mysql_query("select id from products");
$products = array();

while($row = mysql_fetch_row($result))
{
$products[] = $row[0];
}

return $products;
}

private function getPurchasesForCustomer($customer)
{
$result = mysql_query(sprintf("select product from purchases where customer = '%s'",
mysql_real_escape_string($customer)));
$purchases = array();

while($row = mysql_fetch_row($result))
{
$purchases[] = $row[0];
}

return $purchases;
}
}

$store = new ExampleStore();
$store->exportFromMySQL();
$store->importToDirectedEdge();

$store->createCustomer(1000);
$store->createProduct(1000);
$store->addPurchase(1000, 1000);

print_r($store->getRelatedProducts(2));
print_r($store->getPersonalizedRecommendations(2));

</source>

Getting started for Python developers

2009-11-23T17:46:37Z

Scott:

{{GettingStarted|Python|Python/directed_edge.py|You'll need to make sure you have urllib, urllib2 and httplib2 installed. We used [http://peak.telecommunity.com/DevCenter/EasyInstall easy_install] to grab them.}}
{{ExampleStore|ExampleStore Python}}
{{FurtherReading}}

Getting started for PHP developers

2009-11-23T17:31:43Z

Scott:

{{GettingStarted|PHP|PHP/DirectedEdge.php|You'll also want to make sure that you have [http://pear.php.net/ Pear] and [http://pear.php.net/package/HTTP_Request2/ HTTP_Request2] installed to use the bindings.}}
{{ExampleStore|ExampleStore PHP}}
{{FurtherReading}}

Python ExampleStore

2009-08-10T10:46:16Z

Scott:

:'''Note:''' You will need to change the usernames / passwords in the <tt>ExampleStore</tt> constructor.

:''Comments will follow, but the flow is identical to the [[Ruby Bindings for E-Commerce Tutorial]]''.

<source lang="python">
#!/usr/bin/python

from sqlalchemy import Column, Integer, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

from directed_edge import Exporter, Item, Database

Base = declarative_base()

class Customer(Base):
__tablename__ = "customers"
id = Column(Integer, primary_key=True)

class Product(Base):
__tablename__ = "products"
id = Column(Integer, primary_key=True)

class Purchase(Base):
__tablename__ = "purchases"
customer = Column(Integer, primary_key=True)
product = Column(Integer, primary_key=True)

class ExampleStore(object):
def __init__(self):
self.database = Database("examplestore", "password")
engine = create_engine("mysql://examplestore:password@localhost/examplestore")
Session = sessionmaker(bind=engine)
self.session = Session()

def export_from_mysql(self):
exporter = Exporter("examplestore.xml")

for product in self.session.query(Product):
item = Item(exporter.database, "product%s" % product.id)
item.add_tag("product")
exporter.export(item)

for customer in self.session.query(Customer):
item = Item(exporter.database, "customer%s" % customer.id)
item.add_tag("customer")
for purchase in self.session.query(Purchase).filter_by(customer=customer.id):
item.link_to("product%s" % purchase.product)
exporter.export(item)

exporter.finish()

def import_to_directededge(self):
self.database.import_from_file("examplestore.xml")

def create_customer(self, id):
item = Item(self.database, "customer%s" % id)
item.add_tag("customer")
item.save()

def create_product(self, id):
item = Item(self.database, "product%s" % id)
item.add_tag("product")
item.save()

def add_purchase(self, customer_id, product_id):
item = Item(self.database, "customer%s" % customer_id)
item.link_to("product%s" % product_id)
item.save()

def related_products(self, product_id):
item = Item(self.database, "product%s" % product_id)
return [related.replace("product", "") for related in item.related(["product"])]

def personalized_recommendations(self, customer_id):
item = Item(self.database, "customer%s" % customer_id)
return [related.replace("product", "") for related in item.recommended(["product"])]

store = ExampleStore()
store.export_from_mysql()
store.import_to_directededge()

store.create_customer(1000)
store.create_product(1000)
store.add_purchase(1000, 1000)

print store.related_products(2)
print store.personalized_recommendations(2)
</source>

Ruby Bindings for E-Commerce Tutorial

2009-05-25T13:57:58Z

Scott:

One of the main things that people often want to do is to integrate Directed Edge's recommendation engine into their online store. Many of those stores are using Ruby already, so we decided to give a very concrete walk-through of doing recommendations based on previous customer purchases starting from exporting data all the way to keeping it in sync.

'''If you want to see all of the code in one go, you can get it in full syntax-highlighted glory here: [[ExampleStore Class]], or download it [[Media:example_store.rb|here]]. You'll also want our Ruby bindings, which you can get from GitHub [http://github.com/directededge/directed-edge-bindings/blob/6f5f421d25f4b43e6b39cb35702987b5598cc85f/Ruby/directed_edge.rb here] ([http://github.com/directededge/directed-edge-bindings/raw/6f5f421d25f4b43e6b39cb35702987b5598cc85f/Ruby/directed_edge.rb download]).'''

Not a Rubyist? No problem. You can use our API from any programming language, the language bindings just make things a little easier. You still may get something out of reading through the example here to get a feel for the integration process.

== Creating the testing database ==

You've probably got your own site's database that you're more interested in, but if you want to follow along in the code, you can use the store data that we randomly generated. We created 2000 users with 500 products and had each of those users "buy" between 0 and 30 products.

For that we created three very simple tables. Since we don't care about the other properties of the customers or products, all that those tables contain is an ID. The purchase table just contains ''customer'' and ''product'' columns.

<table style="padding: 0.5em; border: 1px solid #ccc; text-align: center;">
<tr>
<td>
<tt>customers</tt> table:

{| style="background: #eee; padding: 0.2em;" border="1" cellpadding="10"
! id
|}

</td><td>

<tt>products</tt> table:

{| style="background: #eee; padding: 0.2em;" border="1" cellpadding="10"
! id
|}

</td><td>

<tt>purchases</tt> table:

{| style="background: #eee; padding: 0.2em;" border="1" cellpadding="10"
! customer !! product
|}

</td>
</tr>
</table>

It really can't get much simpler than that. '''You can get a dump of the database [[Media:examplestore.mysql|here]].''' If you have a local MySQL running, you can create and import the database with these commands:

<pre>
$ mysql --user=root
</pre>

You should now be at the MySQL prompt:

<pre>
mysql> create database examplestore;
mysql> create user 'examplestore'@'localhost' identified by 'password';
mysql> grant all on examplestore.* to 'examplestore'@'localhost';
</pre>

Now back at the command line do:

<pre>
$ mysql --user=examplestore -p examplestore < examplestore.mysql
</pre>

Unless you changed the password above, the password is just ''password''. You've now got the same data that the examples use imported to a database called ''examplestore''.

== Setting up the Ruby plumbing ==

Here we just do the standard ruby ''hashbang'' and import three modules. Ruby gems is required since we've pulled in ''activerecord'' via [http://rubygems.org/ gem]. [http://ar.rubyonrails.org/ ActiveRecord] is the object relational mapping that is standard with [http://rubyonrails.org/ Ruby on Rails] and we'll be using it to access our MySQL database. And finally, we use the [http://github.com/directededge/directed-edge-bindings/tree/master Ruby bindings] for the Directed Edge API.

<source lang="ruby">
#!/usr/bin/ruby

require 'rubygems'
require 'activerecord'
require 'directed_edge'
</source>

== Setting up ActiveRecord ==

ActiveRecord handles connecting to the database for us and is the mechanism that most Ruby web apps use to do the same. To connect with ActiveRecord we have to supply the usual information — database type, host, user name, password and database name.

Based on the database that we imported above, these connection values should work.

ActiveRecord also handles most of the magic of mapping Ruby classes to database tables. It can figure out that the ''customers'' table corresponds to the ''Customer'' (and the some for ''products'' and ''purchases'') just by inheriting from the ActiveRecord base class.

<source lang="ruby">
ActiveRecord::Base.establish_connection(:adapter => 'mysql',
:host => 'localhost',
:username => 'examplestore',
:password => 'password',
:database => 'examplestore')

class Customer < ActiveRecord::Base
end

class Product < ActiveRecord::Base
end

class Purchase < ActiveRecord::Base
end
</source>

In place of ''Customer'', ''Product'' and ''Purchase'' you'll want to substitute in the values that correspond to your database.

== ExampleStore class and constructor ==

We call the class that we're working with ExampleStore. Again, you'll want to change that to fit your needs. Just as a reminder, the full source of the class is [[ExampleStore Class|here]].

The interesting bit here is the connection to the Directed Edge database. Since several methods in the class use the Directed Edge database, we just set up the connection once. This, fairly obviously, assumes that your user / database name is ''examplestore'' and your password is ''password''.

<source lang="ruby">
class ExampleStore
def initialize
@database = DirectedEdge::Database.new('examplestore', 'password')
end
</source>

== Exporting your data to a Directed Edge formatted XML file ==

Communication with the Directed Edge database is all done with our [[XML Format]]. The Ruby bindings make it easy to create files in that format that you can later import to your Directed Edge database.

=== Basic algorithm ===

* Create an instance of DirectedEdge::Exporter to export the items you're creating
* Loop through all of the customers in your store, creating a DirectedEdge::Item for each customer, and marking it as a customer by giving it a ''customer'' tag
:* While looping through the customers, for each customer, also loop through all of their purchases, creating a link from the user to the purchased products
:* Export each of those items, with the freshly created links and tags in place
* Loop through all of the products in your store, creating a DirectedEdge::Item for each product, and marking it as a product by giving it a ''product'' tag
:* Export each of those items
* Tell the exporter that we're done and it can finish up the XML

=== ActiveRecord-fu ===

For those not coming from a Rails background, things like the code to find all purchases matching a given customer ID might seem a bit off the wall:

<source lang="ruby">Purchase.find(:all, :conditions => { :customer => customer.id })</source>

That little thinger there does just what we described above — produces a list of all rows in the purchases table that have the customer ID specified. ActiveRecord also automatically creates class accessors for each column of a table, so the ''.id'' there just corresponds to that column in the customers table.

=== Encoding product and user IDs from the database ===

In the MySQL tables the product table has a simple integer id field as does the customer table. We want to encode those as unique identifiers for the Directed Edge database, which doesn't have separate tables for different item types.

So what we do is take the integer ID and prepend the words ''product'' and ''customer''. So the row from the MySQL customer table with the ID 1 becomes ''customer1''. The product from the MySQL product table with ID 2 becomes ''product2''. It's not exactly rocket surgery.

The code in the comments should help you with more fine-grained details.

=== Output ===

At the end you'll have a file called ''examplestore.xml'' in the directory that you run the script. You'll naturally want to customize that to put it somewhere more appropriate (like, say, ''/tmp'') when incorporating this with your own site.

=== Implementation ===

<source lang="ruby">
def export_from_mysql

# Use the handy Directed Edge XML exporter to collect store data up to this
# point
exporter = DirectedEdge::Exporter.new('examplestore.xml')

# Loop through every customer in the database
Customer.find(:all).each do |customer|

# Create a new item in the Directed Edge export file with the ID "customer12345"
item = DirectedEdge::Item.new(exporter.database, "customer#{customer.id}")

# Mark this item as a customer with a tag
item.add_tag('customer')

# Find all of the purchases for the current customer
purchases = Purchase.find(:all, :conditions => { :customer => customer.id })

# For each purchase create a link from the customer to that item of the form
# "product12345"
purchases.each { |purchase| item.link_to("product#{purchase.product}") }

# And now write the item to the export file
exporter.export(item)
end

# Now go through all of the products creating items for them
Product.find(:all).each do |product|

# Here we'll also use the form "product12345" for our products
item = DirectedEdge::Item.new(exporter.database, "product#{product.id}")

# And mark it as a product with a tag
item.add_tag('product')

# And export it to the file
exporter.export(item)
end

# We have to tell the exporter to clean up and finish up the file
exporter.finish
end
</source>

=== Customizing ===

It's here that the bulk of the customization work will be needed to map things to your store. For starters you'll need to do a search and replace for ''Customer'', ''Product'' and ''Purchases'' for them to be the same ActiveRecord subclasses that you defined up at the top of the code and the names of those should correspond to your database tables. You'll also need to switch the attribute names from ''id'', ''product'' and ''customer'' to whatever the corresponding columns are called in your database. And as mentioned above, you'll want to give the resulting XML file an appropriate home.

You may also want to specify additional tags that correspond to product categories you might later want to filter on — ''book'' or ''album'', for instance. Since products can have as many tags as you like, you can create those in addition to the ''product'' and ''customer'' tags or use your own scheme entirely.

== Importing the XML to the Directed Edge servers ==

After all that muck above, this one's nice and easy. Once we've created the XML file using DirectedEdge::Exporter, it's easy to import to our Directed Edge database. Here we're using the DirectedEdge::Database instance that we created in the constructor.

<source lang="ruby">
# Imports the file exported from the export method to the Directed Edge database
def import_to_directededge
@database.import('examplestore.xml')
end
</source>

The only part you should have to modify here is making the file path correspond to where ever you decide to drop you XML export.

== Adding new users / products ==

With these two methods we repeat some of the logic used in the importer:

* Create a new item with the identifier encoded as the type followed by an integer ID
* Add an appropriate tag to identify the item type
* Save the item (which uploads it to the Directed Edge server)

You can choose to update your Directed Edge database incrementally using methods like these or import occasional snapshots using the import / export mechanism. In most cases, doubly so for fast-changing data sets, it makes more sense to keep the Directed Edge database in sync with your local database this way.

<source lang="ruby">
# Creates a new customer in the Directed Edge database that corresponds to the
# given customer ID
def create_customer(id)
item = DirectedEdge::Item.new(@database, "customer#{id}")
item.add_tag('customer')
item.save
end

# Creates a new product in the Directed Edge database that corresponds to the
# given product ID
def create_product(id)
item = DirectedEdge::Item.new(@database, "product#{id}")
item.add_tag('product')
item.save
end
</source>

If you defined your own scheme for encoding items, you'll need to update that appropriately; the same goes for tags.

== Adding a new purchase ==

As noted in the [[API Concepts]], a purchase for us is just a link between two items, here a customer and a product. So, again, we encode the product and user names, create a new item corresponding to the customer and create a link between that item and the product's identifier.

Like we saw with adding new customers / products to the database, it's usually best to issue these calls when your database is updated so that the data on our servers reflects the data in your database.

<source lang="ruby">
# Notes in the Directed Edge database that customer_id purchased product_id
def add_purchase(customer_id, product_id)
item = DirectedEdge::Item.new(@database, "customer#{customer_id}")
item.link_to("product#{product_id}")
item.save
end
</source>

As usual, if you defined another means of mapping product integer IDs to Directed Edge item item identifiers, that will need to be reflected here.

== Finding related products ==

Now we get to the part where we deliver our first kind of recommendations — related products. Some of the distinction between ''related'' and ''recommended'' items is discussed in the [[REST_API#Related_.2F_recommended_resources|REST API]] documentation.

So, for products we want to find ''related'' products. So we issue a ''related'' query on the product ID we're interested in and specify that we're only interested in things with the ''product'' tag. The argument to ''related'' is a set of tags, so we give it an array (and Ruby sorts out turning that to a set). In this case we only have one item in that array.

We get back a list of identifiers and want to map those back to database IDs, so we substitute back out the ''product'' part of the strings and get just the integer part and map that back to an array.

<source lang="ruby">
# Returns a list of product IDs related to the given product ID
def related_products(product_id)
item = DirectedEdge::Item.new(@database, "product#{product_id}")
item.related(['product']).map { |product| product.sub('product', '').to_i }
end
</source>

Once again, any changes you've made to the way that that items are encoded and tags you're using will need to be reflected here.

== Delivering personalized recommendations ==

Much of the set up here is the same as related products above, except that now we're working with a customer ID rather than a product ID. The stripping out of the ''product'' part of the ID and mapping it back to a list of integers is the same.

<source lang="ruby">
# Returns a list of personalized recommendations for the given customer ID
def personalized_recommendations(customer_id)
item = DirectedEdge::Item.new(@database, "customer#{customer_id}")
item.recommended(['product']).map { |product| product.sub('product', '').to_i }
end
</source>

== Let's give it a spin ==

So, now we've finished defining our class, let's do something with it!

The example below shows:

* Last line of the class definition (end)
* Exports the MySQL database to a file
* Sends that file up to the Directed Edge servers
* Creates a new customer (with ID 500), updating the MySQL database is left up to the reader
* Creates a new product (with ID 2000)
* Creates a new purchase (with the customer and product we just created)
* Retrieves and prints products related to the product with the (MySQL) ID of 1
* Retrieves and prints personalized recommendations for the user with the (MySQL) ID of 1

<source lang="ruby">
end

store = ExampleStore.new

# Export the contents of our MySQL database to XML
store.export_from_mysql

# Import that XML to the Directed Edge database
store.import_to_directededge

# Add a new customer
store.create_customer(500)

# Add a new product
store.create_product(2000)

# Set that user as having purchased that product
store.add_purchase(500, 2000)

# Find related products for the product with the ID 1 (in MySQL)
store.related_products(1).each do |product|
puts "Related products for product 1: #{product}"
end

# Find personalized recommendations for the user with the ID 1 (in MySQL)
store.personalized_recommendations(1).each do |product|
puts "Personalized recommendations for user 1: #{product}"
end
</source>

== See also ==

* [[ExampleStore Class]] - full source in one place of all of the examples covered here
* [[API Concepts]] - a refresher on the concepts at work here
* [[REST API]] and [[XML Format]] - A peek into what's going on behind the scenes

ExampleStore Class

2009-05-25T13:56:55Z

Scott:

<source lang="ruby">
#!/usr/bin/ruby

require 'rubygems'
require 'activerecord'
require 'directed_edge'

ActiveRecord::Base.establish_connection(:adapter => 'mysql',
:host => 'localhost',
:username => 'examplestore',
:password => 'password',
:database => 'examplestore')

class Customer < ActiveRecord::Base
end

class Product < ActiveRecord::Base
end

class Purchase < ActiveRecord::Base
end

class ExampleStore
def initialize
@database = DirectedEdge::Database.new('examplestore', 'password')
end

def export_from_mysql
exporter = DirectedEdge::Exporter.new('examplestore.xml')

Customer.find(:all).each do |customer|
item = DirectedEdge::Item.new(exporter.database, "customer#{customer.id}")
item.add_tag('customer')
purchases = Purchase.find(:all, :conditions => { :customer => customer.id })
purchases.each { |purchase| item.link_to("product#{purchase.product}") }
exporter.export(item)
end

Product.find(:all).each do |product|
item = DirectedEdge::Item.new(exporter.database, "product#{product.id}")
item.add_tag('product')
exporter.export(item)
end

exporter.finish
end

def import_to_directededge
@database.import('examplestore.xml')
end

def create_customer(id)
item = DirectedEdge::Item.new(@database, "customer#{id}")
item.add_tag('customer')
item.save
end

def create_product(id)
item = DirectedEdge::Item.new(@database, "product#{id}")
item.add_tag('product')
item.save
end

def add_purchase(customer_id, product_id)
item = DirectedEdge::Item.new(@database, "customer#{customer_id}")
item.link_to("product#{product_id}")
item.save
end

def related_products(product_id)
item = DirectedEdge::Item.new(@database, "product#{product_id}")
item.related(['product']).map { |product| product.sub('product', '').to_i }
end

def personalized_recommendations(customer_id)
item = DirectedEdge::Item.new(@database, "customer#{customer_id}")
item.recommended(['product']).map { |product| product.sub('product', '').to_i }
end
end

store = ExampleStore.new

store.export_from_mysql
store.import_to_directededge

store.create_customer(500)
store.create_product(2000)
store.add_purchase(500, 2000)

store.related_products(1).each do |product|
puts "Related products for product 1: #{product}"
end

store.personalized_recommendations(1).each do |product|
puts "Personalized recommendations for user 1: #{product}"
end
</source>

Web Services Examples

2009-05-24T05:27:05Z

Scott:

So, now that you've been through the documentation on the [[API Concepts]], [[XML Format]] and [[REST API]], let's take a look at a few practical examples of working with the Directed Edge webservices.

For these examples we'll be using the command line tool [http://curl.haxx.se/ curl] to upload XML to the webservices.

== Importing a database ==

Let's start with a very simple database with three users (with the IDs ''user1'', ''user2'' and ''user3'') and three products (with the IDs ''product1'', ''product2'' and ''product3'').

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item id="user1">
<tag>user</tag>
<link>product1</link>
<link>product2</link>
</item>
<item id="user2">
<tag>user</tag>
<link>product3</link>
</item>
<item id="user3">
<tag>user</tag>
<link>product2</link>
</item>
<item id="product1">
<tag>product</tag>
</item>
<item id="product2">
<tag>product</tag>
</item>
<item id="product3">
<tag>product</tag>
</item>
</directededge>
</source>

{{CurlPut|database}}

This imports the structure above into the ''exampledb'' database.

== Exporting a database ==

{{CurlGet}}

Produces:

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="user1">
<tag>user</tag>
<link>product1</link>
<link>product2</link>
</item>
<item id="user2">
<tag>user</tag>
<link>product3</link>
</item>
<item id="user3">
<tag>user</tag>
<link>product2</link>
</item>
<item id="product1">
<tag>product</tag>
</item>
<item id="product2">
<tag>product</tag>
</item>
<item id="product3">
<tag>product</tag>
</item>
</directededge>
</source>

Which is what we imported just above.

== Adding an item ==

Adds a user with the ID ''user4'' to the database.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item id="user4">
<tag>user</tag>
</item>
</directededge>
</source>

{{CurlPut|user4|/items/user4}}

== Updating an item ==

Here we update the item above, but this time add the property ''city''. This overwrites all current contents of the item.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item id="user4">
<tag>user</tag>
<property name="city">Berlin</property>
</item>
</directededge>
</source>

{{CurlPut|user4-update|/items/user4}}

== Adding a tag to an item ==

Here we add the tag ''Berliner'' to the user that we created / updated in the previous sections. As noted in the [[XML Format]] description, here we're allowed to omit the item's ID.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<tag>Berliner</tag>
</item>
</directededge>
</source>

{{CurlPut|tag|/items/user4/add}}

== Adding a link to an item ==

This creates a link from our recently created user to ''product3''. Again, since we're updating the item we can omit the item ID.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<link>product3</link>
</item>
</directededge>
</source>

{{CurlPut|link|/items/user4/add}}

== Adding a property to an item ==

Sets the value of the property ''country'' to ''Germany''.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<property name="country">Germany</property>
</item>
</directededge>
</source>

{{CurlPut|property|/items/user4/add}}

== Add several tags / properties / links to an item ==

Just like you can add single tags, properties or links to an item, you can also add (or remove) several at a time:

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<tag>German</tag>
<property name="first name">Matthias</property>
<property name="last name">Schmidt</property>
<link>product2</link>
</item>
</directededge>
</source>

{{CurlPut|several|/items/user4/add}}

== Removing a tag from an item ==

Just like we could add a tag to an item incrementally, we can remove a tag (or property, or link) by uploading the same content to the ''remove'' sub-resource.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<tag>Berliner</tag>
</item>
</directededge>
</source>

{{CurlPut|tag|/items/user4/remove}}

== Removing a link from an item ==

And now we can do the same with removing the link to ''product3'':

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<link>product3</link>
</item>
</directededge>
</source>

{{CurlPut|link|/items/user4/remove}}

== Retrieving an item ==

Now that we've made a number of changes to the item that we've been working with, let's see what the current state of it in the database is:

{{CurlGet|/items/user4}}

Produces:

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="user4">
<tag>German</tag>
<tag>user</tag>
<link>product2</link>
<property name="first name">Matthias</property>
<property name="last name">Schmidt</property>
<property name="city">Berlin</property>
<property name="country">Germany</property>
</item>
</directededge>
</source>

== Deleting an item ==

And now let's clear out the item we've been working with from the database.

{{CurlGet|/items/user4|argument=-X DELETE}}

== Finding related items ==

Let's get the related items for ''product1'':

{{CurlGet|/items/product1/related}}

Gives us all related items:

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="product1">
<related>product2</related>
<related>user1</related>
<related>user3</related>
</item>
</directededge>
</source>

Hmm, well, that's not quite what we wanted, but have no fear, this is where the ''tags'' query attribute comes in handy:

{{CurlGet|/items/product1/related\?tags=product}}

And now we get back just items with the ''product'' tag:

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="product1">
<related>product2</related>
</item>
</directededge>
</source>

== Doing personalized recommendations ==

Note that when we want to do personalized recommendations we use the ''recommended'' pseudo-resource rather than ''related''. Our servers do a little different of a sort of magic for producing personalized recommendations than are used to find related products or pages.

{{CurlGet|/items/user3/recommended}}

Here are the personalized recommendations for ''user3''.

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="user3">
<recommended>user1</recommended>
<recommended>product2</recommended>
<recommended>product1</recommended>
</item>
</directededge>
</source>

But here we also get a user in the result set — just as above we want to specify the tags that we're interested in:

{{CurlGet|/items/user3/recommended\?tags=product}}

Which now gives us only product results:

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="user3">
<recommended>product2</recommended>
<recommended>product1</recommended>
</item>
</directededge>
</source>

So, now we get just products back, but ''user3'' is already connected to ''product2'' — meaning we assume that they've already purchased or rated it, so we don't actually want that in the result set. So let's throw in the ''excludeLinked=true''.

{{CurlGet|/items/user3/recommended\?tags=product\&excludeLinked=true}}

Now we get only things tagged ''product'' and none of the ones that the user already has a connection to:

<source lang="xml">
<?xml version="1.0" standalone="yes"?>

<directededge version="0.1">
<item id="user3">
<recommended>product1</recommended>
</item>
</directededge>
</source>

And there we have the (admittedly short) list of personalized recommendations for ''user3''.

XML Format

2009-05-20T05:31:27Z

Scott:

So, keeping in mind all of the basic structures from [[API Concepts]], let's see what they look like in XML.

There are three major cases in which we'll get or send XML to the Directed Edge [[REST API]]:

* '''Import / Export''' - this is just a list of all items with all of their links, properties and tags included.
* '''Updates''' - you can update single items in the database or add or remove links, properties and tags.
* '''Queries''' - for ''related'' or ''recommended'' items (there's a difference).

The first two basically have the same form, with the notable difference that imports and exports contain multiple items, whereas updates only contain one item. Queries contain a list of item identifiers that are related to the query item.

== Import / export example ==

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item id='user_1'>
<link>product_1</link>
<tag>customer</tag>
<property name='last name'>Schmidt</property>
<property name='first name'>Bob</property>
</item>
<item id='product_1'>
<tag>product</tag>
<property name='artist'>Beatles</property>
<property name='name'>White Album</property>
</item>
</directededge>
</source>

Here we've got a customer, named ''Bob Schmidt'' that's connected to an album called ''The White Album'' by ''Beatles''. So we've got two [[item]]s and one [[link]]. Let's start breaking this down a little.

If you imported this to your Directed Edge account using the [[REST API]], you'd have a database with two items. Similarly, if you requested a database export right after that, this is what you'd get back.

'''Before you start sending XML to our server, we recommend passing it through [http://www.xmlsoft.org/xmllint.html xmllint] or similar to make sure that the XML is valid.''' The most common problem that we see with users that are new to the API is sending invalid XML and then our servers choking on it.

== Related / recommended query example ==

: ''Results for things related to "Web 2.0" in our Wikipedia database''

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item id="Web 2.0">
<related>Ajax (programming)</related>
<related>Delicious (website)</related>
<related>Flickr</related>
<related>Web service</related>
<related>RSS</related>
</item>
</directededge>
</source>

The result structure here is pretty self-explanatory — the item is the one which was queried for and the list of results contains the identifiers of the related items. Here, notably, the order is important. Items near the top of the list rank higher than items further down.

The only difference in the XML between an ''related'' and ''recommended'' query is the name of the element that the results are wrapped in.

== Update examples ==

: ''Do a complete update on the item user_1''

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item id='user_1'>
<link>product_1</link>
<tag>customer</tag>
<property name='last name'>Schmidt</property>
<property name='first name'>Bob</property>
</item>
</directededge>
</source>

This is the same format as we used for an import except that there's only one item. As noted in the [[REST API]] if this is PUT to an item it will overwrite all existing data.

: ''Add or remove the tag "user" from an item''

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<tag>user</tag>
</item>
</directededge>
</source>

This is basically just a snippet from a usual item. One notable difference is that the item id may (but doesn't have to be) omitted for updating an item (using the add or remove methods noted in the API docs).

: ''Remove the property "last name" from an item''

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<directededge version="0.1">
<item>
<property name="last name"></property>
</item>
</directededge>
</source>

Similar to adding a tag, when removing a property it's worth noting that the value of the property is completely ignored.

== XML header ==

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
</source>

Good old garden variety [[Wikipedia:XML|XML]] header. We prefer UTF-8, but we shouldn't blow up if you send us other stuff.

== directededge element ==

<source lang="xml">
<directededge version="0.1">
</source>

Nothing too exotic here for the moment. This wraps up all of the stuff that you'll send our receive for the moment. Version is always ''0.1'' for the moment because, well, we're too lazy to bump the version. (Though we'll do better versioning once we're out of beta.)

== item element ==

<source lang="xml">
<item id='user_1'>
</source>

An item is the container for all of the stuff in our database. Exactly what items are is explained in [[API Concepts]].

=== id attribute ===

Every item ''must'' have an id attribute. That's the handle that we use to refer to the item everywhere. If you want to update or delete the item, you refer to it by its ''id''. You can use any scheme that you want to for creating item identifiers, so long as they're unique. Often something like ''type''_''number'' is convenient.

== tag element ==

Tags are always a child element of an ''item''. Tags can be free form as well. Usually they're things like ''user'', ''product'', ''page'', but they can also be much more specific like, ''artist'', ''album'', ''sci-fi'', etc. You can associate as many tags as you would like with a particular item.

== property element ==

<source lang="xml">
<property name='last name'>Schmidt</property>
</source>

Properties are a set of key-value pairs associate with an item. Again, they are free form. You don't need to insert all of the values that you store locally in your database, in fact, most of the time you don't need to associate any at all. These are only useful if you later would like to grab data from the Directed Edge [[REST API]] about the items that you're storing there. You can associate as many properties with an item as you like, so long as their names are unique.

=== name attribute ===

Every property ''must'' have a name attribute. This can be anything that you like, but there may only be one property with a given name per item.

== link element ==

<source lang="xml">
<link>product_1</link>
<link weight="3">product_2</link>
<link type="purchase">product_3</link>
</source>

The link element, surprise, surprise — indicates a link between two items. The identifiers used in the text, here ''product_1'', ''product_2'', and ''product_3'' are just the IDs of items in the database.

'''Links to items that do not exist will be ignored unless those items are defined in the same import.''' In other words, if you have a link to an item that's defined lower in the same batch of XML, no problemo. But if you create a dead link, that just gets ignored.

=== weight attribute ===

The weight attribute is what we use for stuff like rankings. Weights can be in the range of 1 to 10 (or zero if you want to explicitly say there's no weight).

So, if we want to say ''user_1 gave product_1 a rating of 5'' that looks like this:

<source lang="xml">
<item id="user_1">
<link weight="5">product_1</link>
</item>
</source>

=== type attribute ===

This specifies a type for the link. ''Link types'' can be imagined as a set of graphs superimposed upon each other. For example, in an e-commerce settings, these could be ''purchased'', ''rated'', ''category'', etc.

You can then mix and match these ''types'' in recommendations — for example if you wanted to show related products based mostly on purchase, but also considering category information, you can specify that at query time. See our [[REST API]] for details on how to query link types.

So, if we want to say ''user_1 purchaseds product_1 and product_2 gave product_1 a rating of 5'' that looks like this:

<source lang="xml">
<item id="user_1">
<link type="purchase">product_1</link>
<link type="purchase">product_2</link>
<link type="rating" weight="5">product_1</link>
</item>
</source>

=== ''weights'' vs. ''types'' ===

There is both a semantic and practical different between weights and types.

* '''Weights''' are used to represent things that are intrinsic to the data, but where the links being related are categorically similar. Ratings would be one notable example, repeat purchases would be another.

* '''Types''' are used to specify different categories of relationships. For things that are categorically different, types have the advantage that they may be mixed and matched at query time rather than needing to be encoded, as with weights, at the time the data is uploaded to our service.

== preselected element ==

This is a simple way to specify that another item should always appear in the recommendations returned for this item.

<source lang="xml">
<preselected>product_1</preselected>
</source>

== blacklisted element ==

This is a simple way to specify that another item should never appear in the recommendations returned for this item.

<source lang="xml">
<blacklisted>product_1</blacklisted>
</source>

== related element ==

<source lang="xml">
<related>user_1</related>
</source>

Totally no frills. Results from a ''related'' items query. Order matters.

== recommended element ==

<source lang="xml">
<recommended>product_1</recommended>
</source>

Same deal as the related element, but for the results of ''recommended'' queries.

== See also ==

* [[Web Services Examples]]

Introduction to Recommendations

2009-05-20T01:26:43Z

Scott:

: ''This is a reprint from an article by Directed Edge co-founder Scott Wheeler that originally appeared in two parts in Gründerszene ([http://www.gruenderszene.de/it/we-think-youd-also-like-and-the-math-of-suggestion-–-teil-1/ Part 1], [http://www.gruenderszene.de/it/we-think-youd-also-like-and-the-math-of-suggestion-teil-2/ Part 2])

I still buy vinyl records. There’s something nostalgic about friction between a needle and wax producing my music in the age of iTunes. When I buy records I don’t go to one of the Kreuzberg outlets near to where I live, I go across town to Rotation Records. Why? Because of Niko.

Niko owns Rotation and his knowledge of the the genres that he stocks make it worth taking 90 minutes out of my day to get his input. Niko doesn’t know who I am, but if I ask and tell him a little about the records I’ve bought in the past, he can get a pretty good idea of what I might be interested in. It’s in his interest to know his stuff - it makes his customers buy more records and keeps them coming back to Rotation.

Recommendations have no doubt been a component of sales since the advent of commerce. But we do live in the age of iTunes, the age of Amazon, where millions of products are stored in far away warehouses and shipped through complicated, automated logistics. So, who is the “Niko” in the world of e-commerce?

[[Image:Amazon-recommendations.png]]

Amazon was one of the pioneers in the use of automatic recommendations to drive sales. $5 billion, or 25% of the annual sales come from suggesting products to users by showing related books or personalized music recommendations. iTunes’ “Genius” suggests music downloads and Last.fm’s recommender system builds personalized radio streams and shows related concerts. Berlin is home to at least three startups focusing on different elements of recommendations: Plista’s web personalization, Mufin’s music analysis and my own startup, Directed Edge’s, pluggable recommendations offered as a web-service.

But aside from large e-commerce sites and startups that themselves are focusing on recommendations, how are recommendations relevant to the larger startup world? In a survey by ChoiceStream in 2007, 45% of users surveyed were found to be more likely to shop at a store that provided product recommendations, and for users that spent more than $1,000 online in the previous six months, the number went up to a full 69%. The same survey showed that 41% of users were more likely to pay attention to advertising that was personalized to their tastes. In social media and news sites recommendations reduce bounce rates by showing related content within the site.

So that’s the “why”, now what about the “how?”

== Collaborative Filtering ==

Collaborative filtering is the best known branch of recommender systems. In non-computer scientist terms it just means looking at histories of ratings, purchases and clicks and figuring out which products or users are similar to each other based on that.

Let’s imagine that Bob and Sarah have both given ratings to “The Art of the Start” and “Information Rules”.

[[Image:ratings.png]]

If we want to figure out how similar Bob and Sarah’s tastes are, we can plot their ratings on a graph:

[[Image:user-based.png]]

If we recall a little school geometry, we can figure out how far apart their tastes are. (You do remember the Pythagorean theorem, don’t you?)

[[Image:l2-norm.png]]

In linear algebra, the branch of mathematics that dominates collaborative filtering, is called the “euclidean norm” and is one of the simplest ways to measure how far apart two users’ tastes are. If we were to add a third user, Josh, and computed his distance from Bob and Sarah, we could say, “Josh’s taste is closer to Sarah’s than to Bob’s”.

In practice, these systems aren’t computed for two or three users and two books, they’re computed for hundreds, thousands, or even millions of products and users. But let’s keep things simple: If we had measured the distance between Bob and every other user in an online store, we could then say, “Here are the 10 users closest to Bob.” And from there we have a basis for delivering recommendations. There’s a pretty good chance that Bob will like some of the books that those other 10 people like him have purchased and that he doesn’t own yet.

But there’s a catch.

You see, every time a new book is added to the set of ratings above, a new dimension is added to the computations. Two books, two dimensions; three books, three dimensions; a million books, a million dimensions. And if there were 100,000 books and 100,000 users, measuring the distance between each user would take at least 15 billion mathematical operations. In computer science, that computation is said to scale “quadratically”. In practical terms, it means just throwing more hardware at the problem won’t make it scale.

Because of the computational limitations of user-based collaborative filtering, around 2001 there was a shift to what’s known as “item-based collaborative filtering”. In item based collaborative filtering, instead of first finding related users and then using those users to find related products or content, item-based strategies first compute the similarity of every item and use that to recommend products.

Let’s take the ratings above, but add a third book, “Blue Ocean Strategy” and assume that Bob gave it a rating of 5 and Sarah a rating of 7.

[[Image:item-based.png]]

Now, instead of seeing how similar Bob and Sarah are, we see how similar each product is to the others. To deliver recommendations for Bob we can then see which product he’s liked and jump to products which are similar to those.

Item-based collaborative filtering works well when the set of items, or products, is fairly static and smaller than the number of users, as is typical in an online store. The advantage of item-based strategies is that we can quickly compute recommendations for Bob without having to first compare him to every other user of the store. If Bob has rated 100 items, we can look at his top 10 highest rated items, find the items related to those that he hasn’t purchased and rank them based on how similar they are to his top 10.

To do this, however, item-based collaborative filtering requires a pre-processing step where the similarity of every product is first computed. Even some of the big names in collaborative filtering - Amazon, Last.fm, for instance - until recently didn’t update their recommendations more often than every few days or once a week because computing the item similarity matrix takes so long.

The complexity of this computation is the same as creating user-based recommendations; it’s the same 15 billion mathematical operations. One of the most active areas of recommendations research, and one of the defining characteristics of commercial recommender systems, is mitigating this, “curse of dimensionality”, as some papers call it, a problem common to collaborative filtering, image processing and natural language processing.

== Graph-Based Recommendations ==

In computer science lingo, a graph is just a set of connections between things. Those connections are called “edges”. A “directed edge” is a link from one item to another, like a link from one web page to another, and the term my company’s name comes from. One of the most well known applications of graph theory is used in Google’s PageRank, since it ranks the importance of pages based on who links to them and what they link to.

Let’s imagine a small set of friends in a social network:

[[Image:graph.png]]

Bob is a friend of Sarah, and Sarah is a friend of Amy, who is a friend of Josh. Peter is friends with both Amy and Josh. Intuitively, we can guess that Peter and Sarah might get along well since they share some common friends, even though there’s no explicit connection between them.

Graph-based recommendation systems are a less widely known subfield of recommender systems, but they deliver recommendations based on the connections between people or products. For the mathematically inclined, it can be shown that you in fact can model any traditional collaborative filtering system within a graph. For instance, if we go back to our list of ratings, we can imagine a connection between Bob and “The Art of the Start”. Graph connections, or “edges” can also have weights - meaning, they tell us how strong the connection between those two items is. So Bob has a connection to “The Art of the Start” with a weight of 8, and a connection to “Information Rules” with a weight of 7.

By looking at how strongly Bob is connected to the products in his “neighborhood” of the connection structure, just like we could guess that Peter and Sarah might get along well, we can also guess which products Bob is likely to be interested in. In some cases, graphs provide a way to find more obscure links from people to products because there can be a path from Bob through Peter to, say, “Blue Ocean Strategy”.

When we want to see how similar products are in item-based collaborative filtering, we first have to compare every product to every other product, as noted above. In a graph, we can accomplish the same by stepping back one step in the connection structure, to every user that has bought “Information Rules” and then step forward from each of those users to get a full set of “co-purchased” items. Rather than having to iterate over every product, we can quickly build the set of potential matches and often deliver recommendations in real-time without a batch processing step.

So, those are the foundations of our electronic “Niko” in the digital age. For a more technical introduction to recommender systems, check out O’Reilly’s Programming Collective Intelligence.

API Concepts

2009-05-19T01:38:11Z

Scott:

Directed Edge's recommendation engine is built on top of our [http://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/ in-house database]. The database works somewhat differently than traditional databases and its interface is our [[REST API]], so it's useful before getting started to have a feel for how to go about translating things from the way that your site models data to the way that our database works.

== Everything is an item, or ''A brief introduction to schema-less databases'' ==

[[Image:user-item.png|user-item.png]][[Image:product-item.png]]

'''Users, products, places, web pages — they're all items for us.'''

Really? Yep. Above are a couple of examples of things that could be items in our database.

=== Unique Identifiers ===

Items have an identifier (''user_1'' and ''product_1'' in the example above). That can't be changed once it's imported to the database. It's the handle that we use to grab a hold of things later on. These ''must'' be unique. Otherwise the database doesn't know how to tell two items apart. You can use any scheme that you like to encode these. Most people just use something like the type of the object (product, for example) and then the numeric ID that's stored in their SQL database, but you could also use the URL to a page here or anything else that makes sense for your data.

=== Tags ===

Items can also have a collection of text tags. These are useful in filtering results later on. If you, for instance, wanted to find all of the books that are recommended for a certain user, it'd be useful to have some items tagged as ''book''. You can also give an item as many tags as you want, so it's no problem to have ''product'', ''book'' and ''sci-fi'' all associated with the same item.

Note however, that tags are just ways that we can label items; they're not actively used in finding similar items.

=== Properties ===

[[Image:database-table.png|right|250px|]]

Properties are almost as simple. They're just a bunch of key-value pairs that are associated with a given item. Unlike in a traditional database where you'd have a [[Wikipedia:Database schema|schema]] for your users table, another one for your products table and another one for ... well, whatever else you wanted to store, properties in the Directed Edge database are completely ad-hoc.

Just to get the gears turning in your head, examples of things that can be stored in properties are: price, address, first name, last name, author, etc. Pretty much all of the standard fare that would go into a database column.

If you've got some products that have a property named ''price'' and some that don't, that's just fine with us.

At the moment properties aren't used at all by the recommendations algorithms, but we may add support for doing that later. For now they're just a convenient way of keeping tabs on the information that is being recommended.

== Links, or ''the way you connect stuff'' ==

So, we're huge [[Wikipedia:Graph theory|graph theory]] wonks. No, no, not like Excel. Like webs of information.

[[Image:graph.png]]

In computer science lingo, a graph is just a sets of connections between items. Those connections are called ''edges'' and the items are called ''nodes''. All sorts of things can be modeled in graphs, like the way that web pages are connected to each other (there the links between pages are ''edges''), or as above, a set of friends in a social network.

'''Let's make this really easy just to be clear: ''a graph is just a collection of connected stuff.'' '''

Here's an example of the two ''items'' from above connected via an ''edge''. This would indicate, for instance, that the user had bought that product.

[[Image:user-product-link.png]]

'''An ''edge'' is just the math term for what we call a ''link''. They're the connections.'''

Every ''object'' in the Directed Edge database is an ''item''; the relationships between them are ''links''.

=== Examples of links ===

* A '''user (item) purchases a product (item)''', so we make a link from the user to the product.
* A '''user (item) gives a rating of 5 to a product (item)''', so we create a link between the user and the product with a ''weight'' of 5.
* A '''user (item) clicks on a product category (item)''', so we make a link from the user to the category page.
* A '''user (item) is friends with another user (item)''', so we make a link from the user to the other user.
* A '''user (item) is a fan of a band (item)''', so we make a link from the user to the band.
* A '''web page (item) is connected to another web page (item)''', so we make a link from the first page to the second page (just mimicking the HTML link structure).

So when getting up and going with the Directed Edge system, you have to decide which bits of information you want to base the recommendations on.

=== Directed edges ===

You notice up there that those arrows aren't bi-directional? They don't have a pointy thing on both sides? Well, guess where the name of our company comes from: those are ''directed edges''. In graph theory you can have undirected edges and directed edges. All that ''directed'' means here is that they have a direction. A connection from ''A'' to ''B'' is not the same as a connection from ''B'' to ''A''. One says, ''Bob is a friend of Sarah'' and another implies ''Sarah is a friend of Bob''.

Web page ''A'' can link to web page ''B'' without implying that web page ''B'' also links back to web page ''A''. That link is a ''directed edge''.

It's the same kind of deal when a user buys a product. We usually want to represent ''a user bought a product'' rather than implying that the product has some sort of fundamental connection with the user (though it might, for instance, if the user was also the author of it).

The only typical use case where this matters in using the API is that if you have friends in a social network, and the friendships are reciprocal (like Facebook) rather than possibly uni-directional (like Twitter), you have to put ''two'' links in there to convey the friendship.

=== Link types ===

We've also introduced the notion of ''typed links''. '''Typed links''' specify something like:

: ''The relationship between this customer and this product is '''purchase'''.''

'''Link types can be mixed and matched at query time.''' So if you're asking for ''related products'' you can ask for a mix of recommendations based 80% on purchases and 20% on common tags by saying that you'd like 0.8 of the ''purchase'' link type and 0.2 of the ''tag'' link type.

You can create whatever link types seem appropriate for your site's data.

== See Also ==

* [[Introduction to Recommendations]] for the basics on how recommendations work.
* [[XML Format]] for information on how to represent these lovely concepts in simple XML.
* [[REST API]] for documentation on communicating with our webservices.