How-to: Historic Order Import

During implementation of Relewise, it is strongly encouraged that you set up preliminary behavioral tracking to help "warm up" the Relewise engine with user behavior. By tracking orders before the launch of Relewise's services on your site, you ensure that our engine is ready to provide a personalized experience for your users, by way of product popularity, product relations, and more.

In short Relewise recommends setting up behavioral tracking on your site as soon as possible during implementation.

However, not all situations allow for tracking to be performed ahead of time. For those cases, it may be desirable to load in a Historic Order Import, which can help inform the Relewise engine of product relations and popularity going back a number of weeks or months. By default, Relewise recommends loading in data from the past 3 months, though it may be relevant to load in more if your sales tend to be seasonal in nature.

This guide aims to describe what exactly this order import is, how to perform it, and what to be on the lookout for when performing it.

What is a Historic Order Import

In short, a historic order import is a list of TrackOrderRequest calls made to the Relewise API, based on data extracted from your current order-handling system. By setting up a looping code to consume the individual orders, you load data from orders into the Relewise system, in order to populate it with product sales, product relations, and popularity.

By performing this import, you establish a network of related products, useful for Relewise's recommendation algorithms. You also provide product popularity, useful both for recommendation and search sorting purposes.

Typically, your order data will be extractable from your order handling system as a .csv file. You will then need to use or create a script that can process this data, and loop through each order in turn, to ingest the information into your Relewise dataset. We provide an example of such a script further down.

What does a Historic Order Import Require?

Similar to a regular order tracking request, the data in the historic order import doesn't have to contain much. The basic information needed per order is:

• Order ID
• LineItem, which contains:
  • Product ID
  • Quantity
  • Line total
• Subtotal

Optionally, you may want to include user data as well, to help bridge previous authenticated user behavior with future activity. Keep in mind that this is only relevant for Authenticated users, where you have a consistent identifier that can be applied to the order tracking, both historic and going forward. This is most commonly relevant for B2B setups, where your customer base is smaller and more easily identifiable.

Pitfalls of Performing a Historic Order Import

It is important to keep in mind during a historic order import that once an order is ingested, it is not easily undone. As such, it is important to ensure that your data is exactly the way you want it before you start the process of loading it into the dataset.

Product IDs must be the same!

The most important thing when performing a historic order import is to ensure that you are using the same product IDs in the ingested data, as you are using in your TrackProductViewRequest calls.

Using incorrect IDs will result in, at best, the order import having no effect. At worst, it will create a bunch of products in the Relewise database that are not a match for your actual products, and will require you to clean them up - a tedious and time-consuming effort.

When to do a Historic Order Import

As a rule of thumb, a historic order import should be done once, just before you go live with using Relewise's services. This is only necessary if you have not already set up your system to track user behavior; if you have, the historic order import is irrelevant.

Performing a historic order import weeks or months before going live with Relewise will mean that the relevance of the ingested data will have decayed, thus rendering it less effective, or potentially worthless.

Performing the historic order import much more than a few days after you have gone live with Relewise risks polluting actual live-tracked data with data that is now functionally outdated, which may skew results negatively.

The sweet spot is to plan the order import as close to the launch date as possible.

Historic Order Import Examples

The following is an example of how to perform a historic order import. Please bear in mind that this is a template that will require modification to fit your specific needs, but it serves as a baseline for understanding the process.

async Task Main()
{
  
 var path = @"CSV-Path";

 var datasetId = new Guid("00000000-0000-0000-0000-000000000001");
 string apiKey =  "your API Key";
 string serverUrl = "server-url"; 
    var client = new DataAccessor(datasetId, apiKey, serverUrl);

    var products = new List<ProductResultDetails>();
    Guid? nextPageToken = null;

    // Get all products (once)
    while (true)
    {
        var productQuery = new ProductQuery(
            new FilterCollection(),
            excludeProductsWithNoVariants: false,
            pageSize: 1000,
            nextPageToken: nextPageToken
        );

  var response = await client.QueryAsync(productQuery);  

  nextPageToken = response.NextPageToken;
        products.AddRange(response.Products);

        if (!nextPageToken.HasValue)
            break;  
    }

 var existingProductIds = new HashSet<string>(products.Select(p => p.ProductId));

 // Once we have all products, parse the CSV
    var formattingLanguage = CultureInfo.GetCultureInfo("da-DK");
    var orders = new Dictionary<string, Order>();
    int skippedProducts = 0;

    using (TextFieldParser csvParser = new TextFieldParser(path))
    {
        csvParser.SetDelimiters(new string[] { "," });
        csvParser.HasFieldsEnclosedInQuotes = true;
        csvParser.ReadLine(); // Skip headers

  while (!csvParser.EndOfData)
  {
   string[] fields = csvParser.ReadFields();
   if (fields.Length < 5) continue;

   var orderId = fields[0];
   var productId = fields[1];
   var currencyCode = fields[2];
   var priceStr = fields[3];
   var quantityStr = fields[4];

   //Checks that the product actually exists in the dataset
   if (!existingProductIds.Contains(productId)){ skippedProducts++; continue;}
   
   if (!int.TryParse(quantityStr, out int quantity) || quantity <= 0)
    continue;

   if (!decimal.TryParse(priceStr, NumberStyles.Currency, formattingLanguage, out decimal unitPrice) || unitPrice <= 0)
    continue;

   if (!orders.TryGetValue(orderId, out var order))
   {
    order = new Order(User.Anonymous(), new Money(currencyCode, 0), orderId, new List<LineItem>());
    orders[orderId] = order;
   }

   //The calculation here is quantity * price, so if you have the full order price and not the single unit price, change this line
   var lineTotal = quantity * unitPrice;
   order.LineItems.Add(new LineItem(productId, quantity, lineTotal));
   order.Subtotal.Amount += lineTotal;
  }

 }

 var ordersToImport = orders.Values.Where(x => x.LineItems.Count > 0).ToArray();
    $"Importing '{ordersToImport.Length}' orders into Relewise, skipped '{skippedProducts}' line items".Dump(); 

 await new Tracker(datasetId, apiKey, serverUrl).TrackAsync(ordersToImport, CancellationToken.None);
}