Chapter 3: Associations, relations and assets

If you work with CMS before - prepare to be surprised. How catalog entities are related to each other is much, much more complicated than how CMS contents are.

Assocations are the connections between products, so you can cross sell, or up sell other products. For example, if you are selling a table, you would want your customers to buy a chair.

Relations are even more complicated. You have relations between nodes (how node are linked together), between nodes and entries (how entries belong to nodes), between entries (how variants belong to products, how variants belong to packages and bundles).

Assets - or what we will be talking about - is how to associate CMS assets (which are “contents”) to products and categories. Products can be boring without all images, videos, and product manuals - and that’s why we need them.

Working with associations and relations

Managing associations and relations in CatalogContentProvider is done by ILinksRepository (which itself inherits from IAssociationRepository and IRelationRepository). The default implementations of those interfaces are built on top of ICatalogSystem, while they are positioned in lower layer of CatalogContentProvider.

IAssociationRepository is a quite simple interface with only three methods which are all easy to understand (which comes from the nature of Associations, as we learned in previous section). Just to remind you, an association defines two entries to have a connection - they can be sold together, customer might be suggested the alternatives.

 1 public interface IAssociationRepository
 2 {
 3     /// <summary>
 4     /// Gets the associations for the catalog content specified by the content link.
 5     /// </summary>
 6     IEnumerable<Association> GetAssociations(ContentReference contentLink);
 7 
 8     /// <summary>
 9     /// Removes the associations.
10     /// </summary>
11     void RemoveAssociations(IEnumerable<Association> associations);
12 
13     /// <summary>
14     /// Updates matching associations and adds new associations for an entry.
15     /// </summary>
16     void UpdateAssociations(IEnumerable<Association> associations);
17 }

Associations and Relations groups.

When you define an association or relation, you can set “group” of it. It’s just a custom string so you can filter the associations later on. For example, you can have a association group named CrossSell to define the products which can be suggested to customers when they have a product in cart. By default, there is only one group for association (“default”) and one group for relation (well, “default”, too) in the system. Those can’t be set by the Catalog UI, only by code. You can define more association groups by adding this to your initialization module:

1     var associationDefinitionRepository =
2         context.Locate.Advanced.GetInstance<GroupDefinitionRepository<AssociationGro\
3 upDefinition>>();
4     associationDefinitionRepository.Add(new AssociationGroupDefinition { Name = "Cro\
5 ssSell" });
6     associationDefinitionRepository.Add(new AssociationGroupDefinition { Name = "Ups\
7 ell" });
8     associationDefinitionRepository.Add(new AssociationGroupDefinition { Name = "Pro\
9 EpiserverCommerce" });

These association groups will be added if they haven’t existed in your system, otherwise nothing is change. Now when you edit an association in Catalog UI, you can choose one of those:

You can do the same with relation by using RelationGroupDefinition, however, this is not used elsewhere in the Catalog UI. All the entry relations are “default” by the way.

IRelationsRepository, in the other hands, is much more complex and hard to follow. The problem with it, is, it tried to unified the relations between nodes, between entries, and between nodes and entries into one model. It’ll be hard to come up with really good method names, and you’ll have to stick with some abstractions. Here comes the Source and the Target.

 1 public interface ILinksRepository : IRelationRepository, IAssociationRepository
 2 {
 3     IEnumerable<Relation> GetRelationsBySource(ContentReference contentLink);
 4     
 5     IEnumerable<T> GetRelationsBySource<T>(ContentReference contentLink) where T : R\
 6 elation;
 7     
 8     IEnumerable<Relation> GetRelationsByTarget(ContentReference contentLink);
 9     
10     IEnumerable<T> GetRelationsByTarget<T>(ContentReference contentLink) where T : R\
11 elation;
12     
13     void RemoveRelations(IEnumerable<Relation> relations);
14     
15     void UpdateRelations(IEnumerable<Relation> relations);
16     
17     void SetNodeParent(ContentReference contentLink, ContentReference newParentLink);
18 }

You might read the documentation for each method to understand which one to use, but I doubt you will remember which is which. I’ve been there, staying confused to guess which one to use. However, it might be a little easier if you look at Relation types. It’s a simple class with three properties: SortOrder, Source and Target.

And this is from the documentation:

• For a NodeRelation, Source is the ContentReference of the categorized item (aka children), while for an EntryRelation (ProductVariation, PackageEntry, BundleEntry), it is the product/package/bundle itself.
• And for a NodeRelation, Target is the ContentReference of the category, while for an EntryRelation, it is the entries (in the product/package/bundle).

You have ProductVariation, PackageEntry, BundleEntry and NodeRelation extend from Relation with extra information.

A better ILinksRepository

Make no mistake, designing a good API:s is hard (probably as hard as writing a good book, which I am trying to do). We simply can’t replace ILinksRepository, but we can create a bunch of extension methods which use it in a simple, easy understandable way to build up the functions we need. The following methods will return a list of ContentReference of the variations belong to a product, the entries belong to a package and bundle, respectively.

 1 public static IEnumerable<ContentReference> GetVariations(this ProductContent produc\
 2 tContent, ILinksRepository linksRepository)
 3 {
 4     return linksRepository.GetRelationsBySource<ProductVariation>(productContent.Con\
 5 tentLink).Select(r => r.Source);
 6 }
 7 
 8 public static IEnumerable<ContentReference> GetPackageEntries(this PackageContent pa\
 9 ckageContent, ILinksRepository linksRepository)
10 {
11     return linksRepository.GetRelationsBySource<PackageEntry>(packageContent.Content\
12 Link).Select(r => r.Source);
13 }
14 
15 public static IEnumerable<ContentReference> GetBundleEntries(this BundleContent bund\
16 leContent, ILinksRepository linksRepository)
17 {
18     return linksRepository.GetRelationsBySource<BundleEntry>(bundleContent.ContentLi\
19 nk).Select(r => r.Source);
20 }

Those method will return the parent-children relations (The source is the “parent”). We can have reversed method to get the relations the way around. The following method will return the parent products of a variation, parent packages and bundles of an entry, respectively:

 1 public static IEnumerable<ContentReference> GetProducts(this VariationContent variat\
 2 ionContent, ILinksRepository linksRepository)
 3 {
 4     return linksRepository.GetRelationsByTarget<ProductVariation>(variationContent.C\
 5 ontentLink).Select(r => r.Source);
 6 }
 7 
 8 public static IEnumerable<ContentReference> GetPackages(this EntryContentBase entryC\
 9 ontent, ILinksRepository linksRepository)
10 {
11     return linksRepository.GetRelationsByTarget<PackageEntry>(entryContent.ContentLi\
12 nk).Select(r => r.Source);
13 }
14 
15 public static IEnumerable<ContentReference> GetBundles(this EntryContentBase entryCo\
16 ntent, ILinksRepository linksRepository)
17 {
18     return linksRepository.GetRelationsByTarget<BundleEntry>(entryContent.ContentLin\
19 k).Select(r => r.Source);
20 }

The ILinksRepository parameter is optional, it was added to allow the testability. You can add another overload without it even simpler to use.

The final thing is node-node relations, and node-entry relations. We only need to get parent nodes of a node and parent nodes of an entry, you can always get the children entries or nodes of a node by using IContentRepository.GetChildren<T>, which I believe to be easier way to remember.

 1 public static IEnumerable<ContentReference> GetParentCategories(this EntryContentBas\
 2 e entryContent, ILinksRepository linksRepository)
 3 {
 4     return linksRepository.GetRelationsBySource<NodeRelation>(entryContent.ContentLi\
 5 nk).Select(r => r.Target);
 6 }
 7 
 8 public static IEnumerable<ContentReference> GetParentCategories(this NodeContent nod\
 9 eContent, ILinksRepository linksRepository)
10 {
11     return linksRepository.GetRelationsBySource<NodeRelation>(nodeContent.ContentLin\
12 k).Select(r => r.Target);
13 }

Note that GetParentCategories will only return the parent categories which nodeContent is linked to, it does not include the true “parent” which is specified by nodeContent.ParentLink.

Relations in Commerce 11

As we talked earlier, the relations has been changed quite drastically in Commerce 11. If you want to be up to speed (and you should), then let’s go through what is changed.

Firstly, the Source and Target parts have been obsoleted, both in properties and methods (Yay!). There are new Parent and Children parts. So now you always know what is Parent, and what is Child - the bigger entity is Parent, while the smaller entity is Child. You don’t even have to remember the rules with Source and Target.

And as we mentioned earlier, ILinksRepository has been obsoleted. You should change to IRelationRepository and IAssociationRepository when it applies. Changing from GetRelationsBySource and GetRelationsByTarget to GetChildren and GetParents is trickier. I don’t really have any tip here, just go through them one by one, and figure out if you are trying to get the children, or the parents.

Another big change in how relations are handled in Commerce 11 is the introduction of IsPrimary property for NodeEntryRelation.

As we learned earlier, an entry can belong to multiple nodes, but only one of them is “true” - primary node, and others are linked. However, it is not easy to tell which is primary, and which is linked. In 10.x and earlier, the way to determine the primary node is based on the the lowest SortOrder value. That was based on a wrong assumption, and it comes with the cost of unable to sort the entries within a node. To do that, the SortOrder needs to change and it will screw up the relations.

Worry no more, in Commerce 11, the problem has been corrected. SortOrder is now just … sort order. Node-entry relation has now a new property named IsPrimary - and for all the node relations of an entry, only one can has that as true - and that will be the primary node - and all other ones are linked. When you upgrade to Commerce 11, the relation with lowest SortOrder will be selected to be primary (because it was supposed that way). After that, SortOrder will be truly what it is meant to be: You can drag and drop the entries in a node. Let’s say you have a category of iPhone - and there are plenty of model being sold. At this time of writing, iPhone X is the hot shot, and you would naturally want it to be on top of your category. How would you do it - drag and drop it to the top, baby

Assets

Products always need some assets. Images, videos, documentations such as specifications and manuals,…As a developer/an editor, you’ll need a way to work with assets.

There are two ways to work with assets [^foo134]. The old one sticks with ICatalogSystem - you’ll have to access CatalogItemAsset DataTable directly. The new one works via the IAssetContainer interface (which is implemented by EntryContentBase and NodeContent), and is accessible as CommerceMediaCollection. To understand about those APIs, we should take a look at the asset systems. In older versions of Commerce, there is a builtin asset system, which allows you to manage files/folder. This system can be accessed in Commerce Manager. It worked, but with some very important limitations: you can’t manage the assets in Catalog UI, and you can’t use CDN for those files. It’s almost impossible to put those files in some external storage, such as Amazon S3.

Then comes Episerver Commerce 7.5. As it’s fully integrated to Episerver CMS, it’s now possible to use the asset system in CMS - which has no limitations as above.

There is a hidden flag - UseLegacyAssetSystem to let the system know which way you want to work with assets. If it’s set to true, the Media tab in Catalog UI will be disabled. By default, the Assets tab in Commerce Manager is disabled.

If you start a new site, there is no reason to work with old asset system. If you upgraded from R3 or earlier to 7.5 and later, you should migrate your site to use new asset system. There is a tool to do that. There are two things the new one is definitely better:

• It’s unified API:s, you can work with catalog content all the ways.
• The new asset system integrates with CMS asset system, which allows some nice features such as CDN.

OK it’s enough of theory, let’s have some code.

As mentioned above, we’ll work with CommerceMediaCollection, which itself is a collection of CommerceMedia, as following:

 1 public class CommerceMedia : ICloneable
 2 {
 3     public ContentReference AssetLink { get; set; }
 4 
 5     public string AssetType { get; set; }
 6 
 7     public string GroupName { get; set; }
 8 
 9     public int SortOrder { get; set; }
10     
11     //Other methods and properties omitted for brevity.
12 }

Those properties should explain themselves. AssetLink is the ContentReference to the asset (which is supposed to be another CMS content), AssetType is the type of class you defined to map with the asset (we’ll talk about it later). GroupName is the group you want the asset to be, for easier management (such as “videos”, or “images”, or “manual”). SortOrder is the order they appear in the Asset tab in Catalog UI, with one convention: the first in the list will be the default one.

You can add assets from any type implements IContentMedia, but it is more convenient to define your class like this:

 1 using System;
 2 using EPiServer.Commerce.SpecializedProperties;
 3 using EPiServer.DataAnnotations;
 4 using EPiServer.Framework.DataAnnotations;
 5 
 6 namespace EPiServer.Commerce.Sample.Models.Files
 7 {
 8     [ContentType(GUID = "872AA39E-5B79-43BF-B7D5-F34D415553BD")]
 9     [MediaDescriptor(ExtensionString = "jpg,jpeg,jpe,ico,gif,bmp,png")]
10     public class ImageFile : CommerceImage
11     {
12         /// <summary>
13         /// Gets or sets the description.
14         /// </summary>
15         public virtual String Description { get; set; }
16     }
17 }

This is CMS-related stuff. The most important thing in this example class is the MediaDescriptor attribute, which allows you to specific which extensions you want to handle by this class. So in this example, any jpg, jpeg, jpe and so on files uploaded to Media gadget will be handled by ImageFile, and if you drag and drop that file into the Asset list of a catalog content, its AssetType will be EPiServer.Commerce.Sample.Models.Files.ImageFile.

CommerceImage extends ImageData by adding LargeThumbnail property. Together with Thumbnail inherited from ImageData, there are two Blob properties you can use:

1 [ImageDescriptor(Width = 256, Height = 256)]
2 public virtual Blob LargeThumbnail { get; set; }
3 
4 //Inherited
5 [ImageDescriptor(Width = 48, Height = 48)]
6 public override Blob Thumbnail { get; set; }

The first one belongs to the CommerceImage itself, it defines a Blob named LargeThumbnail with the size of 256x256. Well, as you might guess, it’s the big header image whenever you edit an entry or node in Catalog UI:

The second one is inherited from MediaData, it defines a Blob named Thumbnail with the size of 48x48. Can you guess? It’s the thumbnail in the entries list of Catalogs gadget:

The first asset of type CommerceImage in your asset list (which has smallest SortOrder value) will be taken to generate those thumbnails. They are background transparent, and if you don’t like the size, just override it in your class:

 1 [ContentType(GUID = "872AA39E-5B79-43BF-B7D5-F34D415553BD")]
 2 [MediaDescriptor(ExtensionString = "jpg,jpeg,jpe,ico,gif,bmp,png")]
 3 public class ImageFile : CommerceImage
 4 {
 5     /// <summary>
 6     /// Gets or sets the description.
 7     /// </summary>
 8     public virtual String Description { get; set; }
 9 
10     [Editable(false)]
11     [ImageDescriptor(Width = 128, Height = 128)]
12     public override Blob LargeThumbnail { get; set; }
13     
14     [Editable(false)]
15     [ImageDescriptor(Width = 64, Height = 64)]
16     public override Blob Thumbnail { get; set; }
17 }

Now the thumbnail header should be only 128x128. The thumbnail list of Catalogs gadget, however, is styled by an CSS class, so it’s not resized automatically on the UI. If you want to, you’ll have to override this class yourself:

1 .Sleek .epi-thumbnailContentList.dgrid .dgrid-row .epi-thumbnail {
2     width: 48px;
3     height: 48px;
4 }

There are two important limitations regarding the AssetType:

• The full name of the type must not exceed 190 characters in length. It’s the maximum length of the column for AssetType in CatalogItemAsset table. CatalogContentScannerExtension will validate this during the site start up, and throw an exception if it finds any media class which does violate that rule. This allows the data to be properly indexed.
• Asset type must be resolve-able when you import or export the catalog. If you import or export your catalogs in the context of Commerce Manager (which is supposed to have no custom content types), the asset mappings with entries and nodes will be skipped.

Catalog content versions

One of the most important features in CatalogContentProvider is it bring versioning to catalog content. It was somewhat limited with Commerce 7.5 (the languages handling was a bit sloppy), but it has been much more mature since Commerce 9. The versioning system in Commerce 9 is now more or less on par with versioning in CMS, and it’s a good thing.

If you’re new to Episerver CMS/Commerce, then it might be useful to know how version and save action work in content system. Of course, you can skip this section if you already know about it. The version status is defined in EPiServer.Core.VersionStatus. When you save a content, you have to pass a EPiServer.DataAccess.SaveAction to IContentRepository.Save method.

The documentation for those enum:s are pretty good, and the combinations of SaveActions can be quite complicated, but we can consider a basic case so you’ll get the idea:

 1 var parentLink = ContentReference.Parse("1073741845__CatalogContent");
 2 var contentRepo = ServiceLocation.ServiceLocator.Current.GetInstance<IContentReposit\
 3 ory>();
 4 
 5 //Unsaved content, should have status of NotCreated.
 6 var variationContent = contentRepo.GetDefault<VariationContent>(parentLink);
 7 variationContent.Name = "New variation";
 8 
 9 //Save the content, now it is CheckoutOut
10 var variationLink = contentRepo.Save(variationContent, SaveAction.Save, AccessLevel.\
11 NoAccess);
12 
13 //A saved content is readonly. To edit it, we must create a "writable" clone
14 variationContent = contentRepo.Get<VariationContent>(variationLink)
15     .CreateWritableClone<VariationContent>();
16 variationContent.Code = "New-varation";
17 
18 //Check in, in the UI, it's Ready to Publish, which mean the edit was complete. 
19 //The content status is now CheckedIn.
20 variationLink = contentRepo.Save(variationContent, SaveAction.CheckIn, AccessLevel.N\
21 oAccess);
22 variationContent = contentRepo.Get<VariationContent>(variationLink);
23 
24 //Oops, made a typo. reject it.
25 variationContent = contentRepo.Get<VariationContent>(variationLink)
26     .CreateWritableClone<VariationContent>();
27 //Now it's Rejected.
28 variationLink = contentRepo.Save(variationContent, SaveAction.Reject, AccessLevel.No\
29 Access);
30 
31 //Correct the mistake.
32 variationContent = contentRepo.Get<VariationContent>(variationLink).CreateWritableCl\
33 one<VariationContent>();
34 variationContent.Code = "New-variation";
35 
36 //Publish it directly. Use ForceCurrentVersion flag so no new version will be create\
37 d 
38 //it will overwrite the "rejected" version.
39 variationLink = contentRepo.Save(variationContent, 
40     SaveAction.Publish | SaveAction.ForceCurrentVersion, AccessLevel.NoAccess);

One thing to remember about the code above is that we used the versioned ContentReference:s (WorkId > 0). By default, if you pass a ContentReference without version to IContentRepository.Get<T>, you will get back the published version (of the master language), or the CommonDraft version if there is no published version available. With WorkId, a specific version is returned (given that version exists).

One fundamental change in Commerce 9 versioning is the uniqueness of WorkId, as we mentioned earlier. Prior to Commerce 9, WorkId is only unique for a specific content, but now it’s unique across system. It does mean from the WorkId, you can know anything, from the content itself to the version you’re pointing to. This also means WorkId triumphs everything else. So for some reasons, you get your ContentReference wrong, such as the ID points to a content, but the WorkId points to a version belongs to another content, then the WorkId wins, and the content returned is the content WorkId points to (In Commerce 8, the content ID wins). That’s why you should always make sure you get the correct version of ContentReference. If you want to load a version with a specific status without knowing its WorkId, make sure can use IContentVersionRepository. For example, to get the latest “Previously Published” version in English:

1 var contentVersionRepository = ServiceLocator.Current.GetInstance<IContentVersionRep\
2 ository>();
3 var contentLink = ContentReference.Parse("83__CatalogContent");
4 var versions = contentVersionRepository.List(contentLink);
5 var previouslyPublished = versions.OrderByDescending(c => c.Saved)
6         .FirstOrDefault(v => v.Status == VersionStatus.PreviouslyPublished && v.Lang\
7 uageBranch == "en");

The unique WorkId across system is, again, consistent with CMS. The other changes, comes at a much lower level - database.

Versioning was reason catalog system in Commerce 7.5 was significant slower than Commerce R3. The non-version parts (ICatalogSystem and MetaDataPlus) are still fast, but the implementation of versioning in Dynamic Data Store¹⁰ was the bottleneck. Firstly, DDS was was not designed to handle a “store” with multiple millions of rows. Secondly, the queries are generated automatically and they are less than optimal to access data.

The idea for storage was pretty simple and perhaps was chosen because it looked quite straightforward. Each version (which was call CatalogContentDraft) is stored in one row, and except the “static” data which is supposed to be on all version (such as content link, code, etc), all properties (aka IContent.Property) are serialized and stored in a big column of NVARCHAR(MAX). Compared to the way it’s stored in metadata classes, this was, of course, slower and contribute to the slowness of system as well. Having data in two places means you have to sync it every time you save, which adds even more overheads to the system.

To improve the performance, those issues must be addressed: DDS must be ditched, serialization should be minimized and synchronization should be reduced. All were done in Commerce 9, by rewriting the versioning storage entirely. The draft versions of catalog contents are now stored in more or less the same way with “published” versions (from the previous section, you might already know about CatalogContentProperty), and inline with what CMS does. Version information are stored in ecfVersion table, while version properties are stored in ecfVersionProperty table. ecfVersionProperty has same “schema” as CatalogContentProperty.

Language versions.

If you have been working with CMS content - then language is an area that Catalog has significant difference compared with CMS content.

It’s quite far to say that Episerver CMS is more “advance” when it comes to language settings. CMS allows you to set a content exists in a certain language or not. However, Catalog content does not have such flexibility. There are some characteristics to keep in mind:

• All catalog content language versions are defined by the language setting in the catalog. A content (node or entry) will exist in all languages enabled in its parent catalog. However, it does not mean that when you enable a language, versions in that language will be created automatically. The missing language versions will be created on-the-fly on demand the next time you request it.

Let’s take an example, your catalog was English only, then because you start selling in Sweden market, you want to add Swedish version - that can be done by enable Swedish language in CMS, and then add it to your catalog. At this point your nodes and entries still only have versions in English. When you request the Swedish language version (explicitly or implicitly), it is created on the fly, saved to database and returned to you.

• CMS content in general, has the concept of master language. For catalog content, the master language of products is defined by the default language of their catalog. When a property is NOT decorated with CultureSpecific attribute (aka its corresponding metafield has MultiLanguageValue = false), it is supposed to save in master language, and is shared between other languages. Those properties are not editable when you edit non-master language version:

By default, these properties are non-CultureSpecific: - Code - Name - Markets - Assets - Prices, Inventories are also un-editable in non-master languages

That goes the same when you update the content with API:s. This code will not throw any error, but it does not really save anything to database:

1 var contentRepository = ServiceLocator.Current.GetInstance<IContentRepository>();
2 var contentLink = ContentReference.Parse("83__CatalogContent");
3 var content = contentRepository.Get<FashionItemContent>(contentLink, CultureInfo.Get\
4 CultureInfo("sv"))
5     .CreateWritableClone<FashionItemContent>();
6 //Assuming Facet_Size is non CultureSpecific.
7 content.Facet_Size = content.Facet_Size + " edited";
8 contentRepository.Save(content, DataAccess.SaveAction.Publish, EPiServer.Security.Ac\
9 cessLevel.Publish);

When a content is loaded, how are its properties treated? If the being requested language version is master language, all property will be loaded from master language. What if the language is different from master language? CultureSpecific properties are loaded from that language, but non-CultureSpecific properties are loaded from master language.

The same rule of “WorkId triumphs everything else” also applies in this case. That means if you want to load a content with a WorkId points to a language which is different to the language you want to load, then the language pointed by WorkId will be loaded. IContentRepository and IContentVersionRepository both provide methods for you to get a specific version of a content.

Version settings

There are two important settings for version in Commerce:

The first one is DisableVersionSync. This can be set by a key in appSettingssection. If this setting is true, when you update catalog content from lower level than CatalogContentProvider, such as using ICatalogSystem directly, the update will also delete all other versions (only latest, published version is kept). This setting comes in handy when you don’t not want to keep the old versions in the system, only latest, published one is needed. This is pretty much the same behavior with R3 where no versions existed. When you don’t need versioning, this might be one way to improve performance.

The second one is UIMaxVersions. However, this setting affects both catalog content and CMS content, so think carefully before using it. This setting allows you to set the maximum number of versions per content you want to keep, if the number is reached, the oldest version would be trimmed when you save a content. You can either set this by code:

1 EPiServer.Configuration.Settings.Instance.UIMaxVersions = 1;

Or by adding uiMaxVersions (note it’s not UIMaxVersions, the attributes in siteSettings are case-sensitive) attribute to siteSettings in episerver.config.

If you don’t set the value, by default both CMS and Commerce will keep 20 most recent versions of each catalog content. It’s a best practice to set this to a specific value of your choice to keep your version history manageable (and in long term, maintain performance, too many versions might be bad for performance).

Another option to clear old versions is to use the extended flag for SaveAction - ClearVersions. Instead of just using Save.Publish when you publish content, make sure to add the extended flag:

1 var extendedAction = SaveAction.Publish.SetExtendedActionFlag(ExtendedSaveAction.Cle\
2 arVersions);

Note that compared to two previous options, this option is more flexible. You can apply it on certain catalog content which match specific criteria.

Import and export catalogs

It’s fairly common that sites use an external system to manage catalog information. One of the examples is to use Product Information Management - PIM - system, then export the changes and import to Episerver Commerce site. You might have heard about PIM, like inRiver, and their connectors to export the catalog to a format that Episerver Commerce understands.

CatalogImportExport

This is the most common way to import/export catalog.

The export is always full export - so everything in the catalog will be exported. However, import can be partly, you might have catalog which only contains part of the catalog. That make senses because you don’t always update all catalog information, and importing only the changes will help you save both time and resource.

Import and export, out of the box, can only be done in Commerce Manager.

The heart of import/export catalog is CatalogImportExport, but you will mostly work with its wrapper, ImportJob. The reason we need ImportJob was because importing can be a long task and ImportJob has an async implementation with feedback - i.e. the caller can know what is the current progress - how many entries were imported etc. Unlike many other classes where you create a new instance by the inversion of control framework, like structuremap, you create a new instance of ImportJob by the constructors, either

1 var importJob = new ImportJob(sourceZipFile, overwriteDuplicates);

or

1 var importJob = new ImportJob(sourceZipFile, sourceXmlInZip, overwriteDuplicates);

or

1 var importJob = new ImportJob(sourceZipFile, sourceXmlInZip, overwriteDuplicates, is\
2 ModelsAvailable);

Here you are passing the path to the catalog zip file, the name of the catalog file inside that zip (default is “catalog.xml”), and if you want to overwrite catalog items if existing ones found, and if the strongly typed content models are available (which will affect if the asset links will be imported or not). The next step is just to run it:

1 importJob.Execute(addMessageAction, cancellationToken);

addMessageAction is an Action<IBackgroundTaskMessage> so you can have a callback to your method to decide what to do with the messages the importer sends to you (either display them to the UI, log them, or just ignore), and cancellationToken is a CancellationToken so you can cancel the job (only has effect if the import job is extracting the zip file).

To export catalog, it’s even simpler:

1 var importExport = new CatalogImportExport();
2 using(FileStream fs = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite\
3 ))
4 {
5     importExport.Export(CatalogName, fs, folderPath); 
6 }

This will export your selected catalog (via CatalogName) to the filePath in folderPath. Normally, you would want filePath to have file name of Catalog.xml. You can then zip this file to make it easier to store, or to transfer.

CSV Import

CatalogImportExport is not the only way to import the changes from an external source - you can also use CSV for such task. However while a Catalog.xml is very self-contained and has everything you need for a catalog, CSV files are very simple. So you would need an extra part - a mapping file to let Commerce knows which column in CSV maps to which field in catalog.

For example here’s a mapping rule:

1 <RuleItem><SourColumnName>Code</SourColumnName><SourColumnType>System.String</SourCo\
2 lumnType>
3 <DestColumnName>Code</DestColumnName><DestColumnType>NVarChar</DestColumnType>
4 <FillType>CopyValue</FillType><CustomValue />
5 <DestColumnSystem>True</DestColumnSystem></RuleItem>

You can define and change these rules by Commerce Manager:

Creating mapping rules might not be the most interesting part of your job, but fortunately you only have to do it once - as long as your CSV format does not change.

 1 var enDataContext = CatalogContext.MetaDataContext.Clone();
 2 enDataContext.UseCurrentThreadCulture = false;
 3 enDataContext.Language = "en";
 4 var mappingMetaClass = new EntryMappingMetaClass(enDataContext, metaClassName, 1);
 5 Rule mappingRule = Rule.XmlDeserialize(dataContext, mappingFilePath);
 6 char chTextQualifier = '\0';
 7 if (mappingRule.Attribute["TextQualifier"].ToString() != "")
 8 {
 9     chTextQualifier = char.Parse(mappingRule.Attribute["TextQualifier"]);
10 }
11 IIncomingDataParser parser = null;
12 System.Data.DataSet rawData = null;
13 parser = new CsvIncomingDataParser(sourceFolder, true, char.Parse(mappingRule.Attrib\
14 ute["Delimiter"].ToString()), chTextQualifier, true, Encoding.Default);
15 rawData = parser.Parse(Path.GetFileName(csvFilePath), null);
16 System.Data.DataTable dtSource = rawData.Tables[0];
17 string userName = Thread.CurrentPrincipal.Identity != null ? Thread.CurrentPrincipal\
18 .Identity.Name : String.Empty;
19 return mappingMetaClass.FillData(FillDataMode.All, dtSource, mappingRule, userName, \
20 DateTime.UtcNow);

CSV can only be used for import, there is no built in way to export data to a CSV. But that would be easy to do so, as CSV is an universal, well known format.

Optimizing your catalog structure

As the most frequently hit entities your system, Catalogs have a big impact of performance for your website - so put some thoughts into organizing it properly will definitely help.

There are a couple of things to consider:

• Do not put too many entries in one node. This has been greatly improved in Commerce 9, but still, Catalog UI will still struggle to manage that many items.

By default, Catalog UI will display a node in a product - variations structure, variations will appear as children of products. However if the number of entries in a node reaches 2000, then it will only display flat structure. The threshold can be changed by SimplifiedCatalogListingThreshold value in AppSettings.

There is no definitive value for the optimal number of entries per node, it totally depends on your catalog, but speaking from experience, you might want to try to limit it to less than 2000. Anything bigger than that and you should be thinking about having sub-category.

And it’s not just Catalog UI. As we will learn later, the hierarchical routing system will need to find all of children of a node to find the next matching RouteSegment. Having too many children, of course, can’t be a good thing. Or there are scenario when you call IContentLoader.GetChildren on that big category, without paging (in case you are indexing your content via Find, for example). Pretty sure SQL Server will choke on such requests.

Depending on your catalog, the maximum number of entries per category might vary from less than 500 to less than 2000. Anything more than that and you should think about introducing sub-categories.

• Think about what metafields to put in a product, and what to put in its variations. It might be crystal clear to do so, but it’s not always the case. Having multiple variations to share same metafields will reduce the number of rows we have to load from CatalogContentProperty. Less rows mean better performance. That might not sound a lot, but imagine you have 100.000 variations, in 10 languages. Each metafield less means 1.000.000 less rows in the table. It can be some thing.
• Same goes to CultureSpecific and non CultureSpecific properties. Non CultureSpecific fields will be shared across all languages, so unless necessary, don’t make your property CultureSpecific. (Of course, if your site is going to have many languages. If it has only 2-3 languages then the difference is very small, though.)

Optimizing your catalog operations

In previous section we talked about how to structure your catalog for the best performance. But that’s only the half of the picture - what even more important is how you work with your catalog.

As we learned before, by default, the catalog content is cached - which is a good thing, because the next time you need it, Episerver Framework will happily load it from memory, save you both precious time and disk I/O. However, if your catalog content is only loaded for one use, cache is not helping here. And remember your server’s memory is limited - and one instance of catalog content is not that small, depends on your modeling, it can be easily 10MB or 20MB. No cache can live forever. It will, sooner or later, be removed for new catalog content. The more that happens, the worse it is for performance - even worse if Garbage Collector needs to kick in often. If you are continuously loading and discarding catalog contents from cache, then you are probably doing something wrong.

As a rule of thumb, only load the content when you absolutely need it. I’ve seen more than often when the developers feel generous to load more than they need - and in the end, the site struggles when it has to serve multiple concurrent users.

Here’s something to keep in mind. If you can - do a review of your projects and fix the following problems. The performance gain might surprise you:

• You don’t have to load an entire content just to get its code. That’s what ReferenceConverter is for - a new method GetCode(ContentReference) was added recently and was even improved in Commerce 10+. It’s much more faster and memory-efficient than a content. This is also true when you want to get, for example, ContentReference of all variations of a product. You don’t have to get all the VariationContent - ILinksRepository (or IAssociationRepository) can provide the lightweight solution for you. The general rule is - if you are not using the entire content, but just a small part of it, then look around. The framework might already provide something for you, with (much) better performance.
• Keep in mind that UrlResolver.GetUrl also load contents, and not just one - it loads content recursively until it finds the configured root (which we will discuss more in part 3 - Advanced Stuffs). Sometimes, call to UrlResolver.GetUrl is inevitable, but in many cases, you can cache it …somewhere. If you are using Episerver Find (or even better, Find.Commerce), then the content url can be included in the index and reused for later calls.
• If you are using IContentLoader.GetChildren for the product listing page, you are probably doing it wrong. Thing is, when a customer visits a product listing page, they unlikely click on every product - just one or two out of ten. Loading all of the children, even with paging, is not a good idea. A common best practice is to avoid loading the content until you needs to - and avoid loading multiple content in product listing page. That’s what the search provider (or Find/Find.Commerce) is for. We will talk more about this is part 5 - Searching.