Chapter 1: How the catalog is structured

##Overview

Catalog Management is the old catalog-editing UI in Commerce Manager, which has been there since the first version of Episerver Commerce. Catalog UI is the rewrite interface which integrates into CMS UI since Episerver Commerce 7.5. Catalog UI is vastly superior to Catalog Management in almost all aspects, and it’s recommended to use by Episerver. However we’ll show screenshots in both interfaces to see how they’re connected.

The catalog system - as the heart of Episerver Commerce - has been there for long, long before the merge and long before CatalogContentProvider. In the most basic concept, a catalog can be viewed as a tree - a catalog is the root, the nodes are the branches, and the entries are the leaves. It helps to grasp some ideas about the catalog system, but the catalog is much more complicated than that, which all kinds of relations and associations we’ll discuss shortly.

The catalog itself, can be seen as the container in the system. You can import and export it at will. Whenever you export a catalog, all other information come along (the nodes, the entries, the warehouses, the inventories information, the prices, and the metadata classes). And you can import it to another system. You can use many ways to transfer catalog information between system, such as writing the code to update the entries directly from CSV files, use ServiceAPI to update from a PIM like InRiver, … but the most common way to date, is to export and import the catalog.¹²

A catalog can have multiple languages. Unlike CMS content, when a page might, or might not have a language, a catalog content - node or entry, always have all languages as defined in their true catalog. You can’t delete a language branch. If your catalog has sv but your entry does not have version on that language yet, a new sv version will be created on fly.

Under catalog, you usually have nodes. You can create an entry to be a direct children of a catalog. However, in that case, it actually means that entry does not belong to any nodes.

Under catalog, you can have nodes[^foo114]. And under the node, you can have another nodes, or entries. The naming can be a little confusing here: It’s sometimes called node, or CatalogNode, but in the new Catalog UI, it’s called Category to match with “Category concept” of CMS content.

There are two kinds of relations between nodes: - Each node would have one parent node. Nodes which have no parent node will be direct children of the catalog. This relation is defined by the ParentNodeId in CatalogNode table.

• A node can be linked to one or multiple nodes. When a node is linked, it will appear in as a normal child node in the parent node. This relation is defined by the CatalogNodeRelation table.

A node might contain, of course, many entries. And vice versa, an entry can belong to multiple nodes. This kind of relation is defined is NodeEntryRelation table.[^foo115]

And then we have relations and associations between entries.

Node-entry relation

Before Commerce 11, there is no way to know which is the “true” parent node of an entry. Conventionally, the first relation with lowest SortOrder is assumed to be the parent node. This is not entirely correct, and it comes with a limitation: you can really drag and drop an entry within a node. If you have a category of “smart phone”, you would want to make the most popular phone, like iPhone 14 or Galaxy S22 to the top of the category, because those are more likely to be bought buy a customers. Prior Commerce 11, you just … can’t. It is not supported in Catalog UI, and if you want a workaround, it can be completed (for example adding a metafield to the entry type and use that as sort order).

From Commerce 11, the concept of primary NodeEntryRelation is introduced. So you can explicitly set a NodeEntryRelation to be primary, meaning that node is the true parent of that entry.

The entries

If the catalog system is the heart of entire Episerver Commerce, the entries can be called the heart of the catalog system. In the end it’s the only thing your customers care about, right? And it might no be simple as you might think.

There are 4 types of entries in the system:

• Product: Well it’s a product. Normally, a Product is a place holder of information, it does not have inventories or prices for itself, so it’s not sell-able. (I’ve seen some customers sell products directly, it’s possible but it will be a lot more of works. I would suggest you to stick with the “standard” implementation instead). A product might have one or more variations.
• Variation/SKU: It’s easiest to explain Product-Variation in term of a TV series. Let’s pick the LG CX OLED TV for example (I have one of these, it’s a pretty awesome TV by the way). We can have it as a product with all specifications (resolution, OS, features and App). There are 4 variations by sizes (48, 55, 65 and 77 inches). Here’s a screenshot from LG website:

A variation has its own inventories and prices. In the end, it’s the thing your customers actually buy.

• Package: A package consists of two or more variations, which itself act as a variation. Think of a package as a collection of items. You might have Harry Potter books as seven variations, and then you have a collection of them as a package. Package has its own prices and inventories. Therefore, when you buy, you all of items as one. Customers should not have the ability to change the quantity or remove items from a package.
• Bundle: A bundle is just a collection of things you can add to the cart at once, but then they acts as individual items. They are something customers buy together, but are not forced to do so. You can change quantity of or remove items from the cart if you’d like to. Bundles have no prices or inventories of its own.

And then comes the tangible relations and associations between them.

There are three type of relations between entries:

• ProductVariation between a product and its variations.
• BundleEntry between a bundle and its entries.
• PackageEntry between a package and its entries.

.

If you look at the tab Variations/SKUs when editing a Product in Commerce Manager, you’ll see “Quantity” column. This is not entirely correct as the Quantity is never used in a ProductVariation relation. Catalog UI reflects this better. It only matters in BundleEntry and PackageEntry relations.

And then between entries, you can set the associations. It is to show relations between products, for example, they can be used to indicates the products that could be displayed as CrossSell or UpSell items. The framework does not use these information, it just allows you to store them and use them as you see fit.

Relations between entries are managed by IAssociationRepository, it is a fairly simple interface with just three methods:

 1 public interface IAssociationRepository
 2 {
 3     /// <summary>
 4     /// Gets the associations for the catalog content spe\
 5 cified by the content link.
 6     /// </summary>
 7     /// <param name="contentLink">The content link.</para\
 8 m>
 9     /// <returns></returns>
10     IEnumerable<Association> GetAssociations(ContentRefer\
11 ence contentLink);
12 
13     /// <summary>
14     /// Removes the associations.
15     /// </summary>
16     /// <param name="associations">The associations to re\
17 move.</param>
18     void RemoveAssociations(IEnumerable<Association> asso\
19 ciations);
20 
21     /// <summary>
22     /// Updates matching associations and adds new associ\
23 ations for an entry.
24     /// </summary>
25     /// <param name="associations">The associations.</par\
26 am>
27     void UpdateAssociations(IEnumerable<Association> asso\
28 ciations);
29 }

Variations

As the most important entry types in the system, variation and package receive a bit of special treatment. While they share some tables with other entry types, they have one specific table, Variation, which contains some of very important information about your SKU. You can view or edit those values in the Pricing tab in both Commerce Manager and Catalog UI:

• Display Price (or List Price) is an interesting field - it’s long obsolete since R3 with the birth of pricing system, however, if you have a catalog exported from old version, it might still be there. If you delete the value (aka set the value to be null), the field will be hidden. This field is not mapped to any property in strongly typed content, nor displayed in Catalog UI.
• Min Quantity and Max Quantity are the lower and upper limit a customer can add this SKU to cart. Those values will be used in several workflows during checkout, so they are particularly useful if you want to configure something like “Only buy by batch of at least 4”, “Only 10 per cart”, etc. Note that almost every value of quantity in Episerver Commerce is stored in decimal. While this might not really be useful in normal shop (it does not really make sense to have 1.5 TV or 2.3 Blu-ray discs, right?), it’s a must for B2B scenarios or liquor stores.
• Merchant is a field supposed to identify the merchant provides this SKU, but I hardly see any real world uses of it. This is not reflected in strongly typed content.
• Weight: It should describe itself, but only the value without unit. The unit is the base weight setting of the Catalog. (which can be either kilograms or pounds). Optimizely Commerce does not really use the unit in any calculations, the setting is more of a convention. You can assume that the unit of the weight, for example, as gram instead. But in most of the cases, sticking with the setting would help you avoid later confusions.
• Height, Width and Length: Those are values of the dimensions. Like weight, they are not tied to an unit. You can set the unit in the base length setting of the Catalog (which is by default, is null, but you can choose between inches and centimeters).
• Package: Choose the shipping package the SKU might come with. You can define the packages in Administration/Order System/Shipping/Shipping Packages section of Commerce Manager. Previously, which come with dimension (Height, Width and Length) settings. Prior to 8.7, this is the only way to define the shipping dimensions, but you now can use much more convenient way with the Height, Width and Length attributes. However, this is still useful when you have non-box SKUs, such as flower vases.
• Tax category: The tax category this SKU belong to. You usually don’t update this manually, but you can still go to Administration/Order System/Tax Configuration to add a tax category and try.
• Track Inventory: This is one important flag. If it’s set to false it does mean you don’t want to track the inventory of this SKU, so any check for inventory such as available quantity etc. will be skipped during checkout. That can be useful in cases such as your SKU is a software and is distributed via download.

The product-variant configuration

The most common configuration of catalog is product-variation. One product has multiple variants in size, color, material: you have a T-shirt, where you store information like brand, material, etc. You then have variants of that T-Shirt, with different size and color.

It is, of course, not the only configuration.

Another fairly common case is to have standalone SKUs: a variant is an entry of its own and does not belong to a product. This can be used when the item you are selling is unique, and have no variants.

Another, less common scenario is the product-product-variant. The first level product defines general information, like T-shirt. The second level product defines size, while the variants defines other information like color. This is used in a few websites - and to be honest I’m not sure why it’s needed. It requires customizations on framework level and the benefits aren’t that clear. In my opinion, yes, you can, but that doesn’t mean you should.

The two systems.

As mentioned in the brief history, Episerver Commerce was based on Mediachase eCF. Mediachase eCF built its catalog API:s around one interface - ICatalogSystem (If you worked with Commerce R3 and earlier, you might use the instance of it instead - CatalogContext.Current). ICatalogSystem methods use mostly DTO objects as inputs and outputs. The DTO:s are actually typed DataSets, and in most case, maps to several tables in database.

ICatalogSystem had its days. By today standards, it might not be the best API design, but it worked quite well until Commerce R3, which is the compatibility release for Episerver CMS 7. The biggest new feature in CMS 7 was the content concept (Before that, we had pages, and pages), so it became a big demand to have a content API:s to work with catalog/node/entry, so Episerver Commerce can be considered as a first-class product in Episerver framework. It was made real in Commerce 7.5, and has been improved and refined ever since.

The content API:s to work with catalog content use ICatalogSystem internally. To be fair, ICatalogSystem is a performant, proven, reliable API:s, so why not use it directly?

First, using the new API:s means you use the same API:s as the Episerver CMS. Working on a unified API:s allow you to increase the code- recognition. Once you’re familiar with the API:s, it’ll be much quicker for you to know what the intention the code are doing to do (code-read), and much unlikely for you to use wrong method (code-write). In the end, it’s your productivity which improves.

Second, the new API:s use a much more efficient caching mechanism, or it’s the old system which does not cache thing very effectively. For the content way, if you load an object, you can cache it by its ContentReference and that’s it. Every call to that ContentReference will hit the cache easily. Caching the DTO:s is a much harder problem. A CatalogEntryDto, for example, can have multiple rows, and depends on the ResponseGroup you specified, the data in some table might not be present. You get the cache, but hey, do you know if the next time can you re-use it when your parameters change?

Third, with the content way, you can read or update both of the “static” properties and the metafields in the same API:s. They are, after all, just properties. With the old way of ICatalogSystem, you can only update static properties and will need the help from MetaObject when you want to read or update the metafields.

Forth, the POCO (Plain Old C# Object) approach in the content way is arguably move intuitive than the DTO:s (DataSet:s) approach of ICatalogSystem. Let’s see how to change a name of an EntryContentBase vs a CatalogEntryDto:

1 entryContent.Name = "This is new name";

vs

1 entryDto.CatalogEntry[0].Name = "This is new name";

Tell me, which one will catch your eyes?

And finally, ICatalogSystem simply provides nothing to work with versions. New content API:s are simply superior because they are designed to work with versions.

In short, if you start your Optimizely Commerce Cloud project today, you should be using the latest and greatest version, and you should be using the IContentRepository. There are very little reasons to use ICatalogSystem, content is the way forward.

ReferenceConverter - the bridge.

One of the biggest features in Episerver CMS is its content provider system, which allow you to plug your own content provider to the system. So anything can be considered content, and can be manipulated with one unified API:s. However, to do that, you have to solve the first issue - the identity.

There are two problem Commerce had to resolve when they try to plug the catalog structure into the content provider system. Firstly, the data are stored in three separated tables (and you might already know, Catalog, CatalogNode and CatalogEntry), with the auto incremented identity. So a content with ID = 1 can be a Catalog, a Node or an Entry. Secondly, there can be multiple catalogs, while you need a “root” content to attach your content provider system.

The second one can be easily resolved by introducing a virtual root. It’s just another content, but virtual, you can never edit or delete it. It’s there, created on the fly whenever you need it. But the first one is a bit tricky. That was when ReferenceConverter came to life.

The content provider system requires every content to be identified by a specific ContentReference, which is supposed to be unique in the system. A ContentReference consists of three parts:

A sample catalog content reference: 123_1_CatalogContent

• The first one is the content id, which is mandatory. The ID is supposed to be unique in the system.
• The work id, which identifies the version of the content.
• The content provider name. This will allow the content system to know which provider should be handling the content. For catalog content, it’s always CatalogContentProvider who does the leg works.

ReferenceConverter is a small class resides in Medichase.Commerce namespace. Its API:s are actually quite simple:

 1 public class ReferenceConverter
 2     {
 3         /// <summary>
 4         /// Gets a <see cref="ContentReference"/> instanc\
 5 e with the specified commerce object ID and type
 6         /// encoded in the content ID.
 7         /// </summary>
 8 public virtual ContentReference GetContentLink(int object\
 9 Id, CatalogContentType contentType, int versionId)
10 
11         /// <summary>
12         /// Gets the actual id of commerce object.
13         /// </summary>
14 public virtual int GetObjectId(ContentReference contentLi\
15 nk)
16 
17         /// <summary>
18         /// Gets the type of the commerce object. Parse t\
19 he content ID to take two most significant bits to
20         /// determine CatalogContentType.
21         /// </summary>
22 public virtual CatalogContentType GetContentType(ContentR\
23 eference contentLink)
24         
25         ...
26     }

These are two most interesting parts of the class. As I said above, the content id is supposed to be unique, while the catalog id, catalog node id and catalog entry id are not. The solution? Bitwise comes to rescue.[^foo33]

Episerver Commerce use the first two bits of the id to store the content type. So basically:

• 00: CatalogEntry
• 01: CatalogNode
• 10: Catalog
• 11: Do you want to guess? Well, it’s the Catalog root. You can always get the root’s content link by GetRootLink() method of ReferenceConverter. It’s always the boring same content reference every time.

So these methods work in an extremely efficient way to convert the id to ContentReference back and forth: From an id, add the two first bit (MSB - most significant bit), based on the content type, then add the fixed content provider name (CatalogContent) to get the ContentReference. From a ContentReference, clear the two first bits to get back the id. Easy, huh?