Linq provider for CommerceServer : thanks to CSMetal !

Mar 28

Linq provider for CommerceServer : thanks to CSMetal !

Commerce Server, Development No Comments »

A couple of weeks ago, Kerry Havas, Microsoft Commerce Server Product Architect, has released the second version of CSMetal.

What is CSMetal? It’s a free tool for Commerce Server which generates strongly typed classes called by a dedicated Linq provider based on the Commerce Foundation API => ie, that do not replace the Commerce Foundation API!

Kerry did a very well job with CSMetal as the development becomes very easy for all developers who don’t know Commerce Server!

Before using the Linq provider for Commerce Server we have to generate the entities with the CSMetal command line.

You need to have:

Le ChannelConfiguration.config file correctly configured
- Please, check that the useSharedCommerceContexts attribute is set to false.
- Add to the message handler node with an attribute name called "CommerceQueryOperation_CommerceEntityDefinition" the below nodes:

<!-- Note that RepositoryMetadataLoader is replaced by a slightly customized one -->
<Component name="RepositoryMetadataLoader" type="Microsoft.Commerce.Metal.Generator.OperationSequenceComponents.RepositoryMetadataLoader, Microsoft.Commerce.Metal.Generator, Version=1.0.0.0, Culture=neutral" />
<!-- Note that this is added to add overlay capability to the Metadata repository -->
<Component name="MetadataOverlayProcessor" type="Microsoft.Commerce.Metal.Generator.OperationSequenceComponents.MetadataOverlayProcessor, Microsoft.Commerce.Metal.Generator, Version=1.0.0.0, Culture=neutral" />

Note that the last child node allows you roughly to generate new entity or add new attribute to existing entity. More details below.

MetadataDefinitions.xml file correctly configured
- Add to the RelationshipTypes node, a child one:
  - <RelationshipType name="Inherits"/>
- Review the “CSMetal_Beta2_ReadMe.rtf” file, and see that we have to add 2 new entities:
  - Basket
    - Order status enumeration
  - Ads
    - Size enumeration for the ads
The famous OrderObjectMappings.xml and OrderPipelineMappings.xml files, properly configured.

Put all the beyond dedicated Commerce Server files in a directory, and add some files to the proper execution of CSMetal:

CSMetal.exe
CSMetal.exe.config which is roughly a copy of the web.config of your web.application
GeneratorSettings.xml which allows you to define the output directory for the generated assemblies. And cherry on the cake with your own namespace!
Microsoft.Commerce.Metal.Generator.dll for the code generation.

So, at the end you will have:

As explained beyond, Kerry introduces a new concept called “MetaDataOverlays”:

You can extand your metadata with an external file and do not apply any modification to the MetaDataDefinition.xml file. That allows you to keep the MetaDataDefinition as a repository and the overlays files. This provides a way to manage the changes you have made to the metadata by keeping it in a separate file.

Just add a new xml file called Metadata_CSMetal.xml in a new directory called MetadataOverlays

<?xml version="1.0" encoding="utf-8" ?>
<!-- 
This file contains the Metadata Repository
-->
<MetadataDefinitions xmlns="urn:schemas-microsoft-multi-channel-commerce-foundation-metadata">
    <DefaultChannel>
        <CommerceEntities>
            <!-- Added for Linq based Search -->
            <CommerceEntity name="CatalogEntity">
                <DisplayName value="Catalog Entity" />
                <Properties>
                    <Property name="Phrase"    dataType="String">
                        <DisplayName value="Phrase"/>
                    </Property>
                    <Property name="ReturnTypes"    dataType="Integer">
                        <DisplayName value="Return Types"/>
                    </Property>
                    <Property name="FirstItem"    dataType="Integer">
                        <DisplayName value="First Item"/>
                    </Property>
                    <Property name="WhereClause"    dataType="String">
                        <DisplayName value="Where Clause"/>
                    </Property>
                    <Property name="TotalItemCount"    dataType="Integer">
                        <DisplayName value="Total Item Count"/>
                    </Property>
                    <Property name="ResultsPerPage"    dataType="Integer">
                        <DisplayName value="Results Per Page"/>
                    </Property>
                </Properties>
            </CommerceEntity>
            <!-- Added for Linq based Search -->
  
            <!-- Catalog Start -->
            <CommerceEntity name="Catalog">
                <Relationships>
                    <Relationship name="DependantCatalogs" type="Relationship" modelName="Catalog" isMultipleItems="true" >
                        <DisplayName value="Dependant Catalogs" />
                    </Relationship>
                    <Relationship name="SourceCatalogs" type="Relationship" modelName="Catalog" isMultipleItems="true" >
                        <DisplayName value="Source Catalogs" />
                    </Relationship>
                </Relationships>
            </CommerceEntity>
  
            <!-- CreditCard Start -->
            <CommerceEntity name="CreditCard">
                <Relationships>
                    <Relationship name="PaymentAccount" type="Inherits" modelName="PaymentAccount" isMultipleItems="false">
                    </Relationship>
                </Relationships>
            </CommerceEntity>
  
            <!-- Basket Start -->
            <CommerceEntity name="Basket">
                <Relationships>
                    <Relationship name="LineItems" modelName="LineItem" type="Relationship" isMultipleItems="true" />
                    <Relationship name="Addresses" modelName="Address" type="Relationship" isMultipleItems="true" />
                    <Relationship name="Payments" modelName="Payment" type="Relationship" isMultipleItems="true" />
                    <Relationship name="Shipments" modelName="Shipment" type="Relationship" isMultipleItems="true" />
                    <Relationship name="RequestedPromoCodes" modelName="RequestedPromoCode" type="Relationship" isMultipleItems="true" />
                </Relationships>
            </CommerceEntity>
  
            <!-- ShopperList Start -->
            <CommerceEntity name="ShopperList">
                <Relationships>
                    <Relationship name="LineItems" modelName="LineItem" type="Relationship" isMultipleItems="true" />
                </Relationships>
            </CommerceEntity>
  
            <!-- LineItem Start -->
            <CommerceEntity name="LineItem">
                <Relationships>
                    <Relationship name="ItemLevelDiscounts" modelName="Discount" type="Relationship" isMultipleItems="true" />
                </Relationships>
            </CommerceEntity>
  
            <!-- CashCard Start -->
            <CommerceEntity name="CashCard">
                <Relationships>
                    <Relationship name="PaymentAccount" type="Inherits" modelName="PaymentAccount" isMultipleItems="false">
                    </Relationship>
                </Relationships>
            </CommerceEntity>
  
            <!-- GiftCertificate Start -->
            <CommerceEntity name="GiftCertificate">
                <Relationships>
                    <Relationship name="PaymentAccount" type="Inherits" modelName="PaymentAccount" isMultipleItems="false">
                    </Relationship>
                </Relationships>
            </CommerceEntity>
  
            <!-- Shipment Start -->
            <CommerceEntity name="Shipment">
                <Relationships>
                    <Relationship name="ShippingMethod" modelName="ShippingMethod" type="Relationship" isMultipleItems="false" />
                </Relationships>
            </CommerceEntity>
  
            <!-- Payment Start -->
  
            <!-- PaymentAccount Start -->
            <CommerceEntity name="PaymentAccount">
                <Properties>
                    <Property name="Id" dataType="String">
                        <DisplayName value="Id"/>
                        <Description value="Unique ID of the payment." />
                    </Property>
                    <Property name="CustomerName" dataType="String">
                        <DisplayName value="CustomerName"/>
                        <Description value="Name of Customer." />
                    </Property>
                    <Property name="BillingAddressId" dataType="String">
                        <DisplayName value="BillingAddressId"/>
                        <Description value="Id of billing address" />
                    </Property>
                    <Property name="DisplayName" dataType="String">
                        <DisplayName value="DisplayName"/>
                        <Description value="DisplayName." />
                    </Property>
                </Properties>
                <Relationships>
                    <Relationship name="PaymentMethod" modelName="PaymentMethod" type="Relationship" isMultipleItems="false" />
                </Relationships>
            </CommerceEntity>
            
            <CommerceEntity name="Payment">
                <Relationships>
                    <Relationship name="PaymentAccount" modelName="PaymentAccount" type="Relationship" isMultipleItems="false" />
                </Relationships>
            </CommerceEntity>
        </CommerceEntities>
    </DefaultChannel>
</MetadataDefinitions>

Now it’s time to run the CSMetal.exe:

In the directories, you will find 3 cs files (if you configure the generation for odata), one for the context, one for the entities and one for odata:

Copy the files GeneratedContexts.cs and GeneratedTypes.cs in the project CSMetalLinqProvider and re run a new compilation for this project.

Your Commerce Entities are now available through your Linq provider for Commerce Server Smile

To get a category, just write down these lines of code:

var category = new CategoryContext(commerceServiceProxy)
                  .IncludeChildCategories()
                  .Single(o => o.Id=="Departments" && o.CatalogId==DefaultCatalog);
var childCategories = category.ChildCategories;

or for a product :

var product = new ProductContext(commerceServiceProxy)
                 .Single(o=>o.CatalogId==DefaultCatalog && o.Id=="AW200-12");
var productCode = product.ProductCode;

That’s just a simple sample of what the Commerce Server Linq Provider is able to do and I’ll try to show you more in my next posts. But, I’m sure that will simplify the life for your developer Smile

Jul 12

How to preview your Commerce Server marketing data

Commerce Server, Development 3 Comments »

As you should know, Microsoft Commerce Server allows you to manage your marketing campaigns of your e-Commerce website through 2 modules of the Marketing System :

These two modules are all schedulable. Which means that you can set a start and end date for the display of an ad or on a discount application.

By default, the Marketing Business Tools allows you to preview the rendering of an ad:

But nothing on a discount… and my two cents, that like my customers, you will tell :”Hey, Gaël, it’s not enough! I need to view my ads or my discounts in my website. I have to be sure that everything is set up correctly…”

Ok, so it’s pretty easy to see the result of your marketing campaign items if they are all applicable right now. Just open your web application and that’s it Sourire However, don’t forget that you might have constrains on your ads or discounts like for example:

the ad can be displayable on a particular type of pages (home, products, basket, …),
the ad or the discount is eligible if the connected user meets the requirement like to be a man or a woman
etc, etc, … (if you have questions on all the different condition, do not hesitate to contact me )

But what can you do if your marketing campaign items are expected in 2 or 3 months? Do you have to wait the d day like you was expecting your first party? So, for sure the answer is…

… really depending of the quality of your Commerce Server Consultant/Expert Sourire Let’s pick up one of the below answers:

“That’s not possible at all! Commerce Server don’t do those kind of feature!”
- Meaning that’s time for you to change your consultant/expert => Are you looking for a Commerce Server expert?
“Change the server’s date and refresh your cache to see the result” (truthful!)

Handy if you have staging and live on the same box, isn’t it?
Most of the time it’s an IT task, and I guess they have something else to do… (I do hope )

Don’t forget to rollback the date, so again an IT task…

“Deactivate your eligible marketing campaign items and activate only the ones you want to test by changing the start date to the current day time.”
- Need to have a staging environment
- Painful, I have customer who has more than 100 advertisements and at least the same amount of discount…
- Confusion when you have to reactivate the true eligible marketing campaign items
  - Have to be sure which have to be reactivate…
- Possible errors when you modify the begin and end date
- Some guys can tell you that a custom development for activating /deactivating a list of campaign items can be the solution. Right, but that’s still not an acceptable answer as you still have to tick/untick the campaign items…
“No worries, we have the solution!”…

The solution is pretty simple. Fist, you have to know that the Commerce Server runtime loads the different caches and context information during the first load of the application or each time the cache is refreshing. That’s of course for better performance considerations.

In the case of discouts and advertising the caching system is loaded by two stored procedures (and an another one called by these 2 stored procedures) which is located in the marketing system database:

mktg_spRuntimeLoadAdvertisements
mktg_spRuntimeLoadDiscounts
mktg_spRuntimeLoadReferenceTables

=> These stored procedures are called at the first load or each time the caches are refreshing.

The Commerce Server team did a really great job with the caching system on discounts and advertisements. Indeed, it’s possible to tell to the runtime which stored procedures to load and so, we are now able to include our own logic within them.

The idea, here, is to have two applications. One for the preview and another one for the default behavior. In the preview web application, we will load our two new stored procedures by the runtime.

In order to do that, we will tell to the runtime to load our 2 stored procedures by modifying the web.config file, in the caches section web.config > CommerceServer > Caches (see MSDN: caches Element).

In the preview application web.config’s file we will changing the default caching section by:

<cache
    name="Advertising"
    type="Advertising"
    refreshInterval="900"
    retryInterval="30"
    loaderProgId="Commerce.CSFLoadAdvertisements"
    writerProgId="Commerce.CSFWriteEvents">
    <config
        key="LoadAdvertisementsProcedure"
        value="spGael_staging_mktg_spRuntimeLoadAdvertisements" />
</cache>
<cache
    name="Discounts"
    type="Discounts"
    refreshInterval="0"
    retryInterval="30"
    loaderProgId="Commerce.CSFLoadDiscounts"
    writerProgId="Commerce.CSFWriteEvents">
    <config
        key="LoadDiscountsProcedure"
        value="spGael_staging_mktg_spRuntimeLoadDiscounts" />
</cache>

The default behavior for the pipeline is to compare the start and end date with the current local time, but as you may know, changing that means re develop all the pipeline components and that’s of course not my solution Sourire . To workaround that, I will move back the start and end date by calculating the difference between the current date time and the preview date, and so retrieve it on all the scheduled discounts and advertisements.

Let’s execute the default stored procedure to load the advertisements caching system “mktg_spRuntimeLoadAdvertisements”:

Only the advertisements with the start and end date in the time frame of the 8th of July came up. For my post, I’ve created a new ad only visible between the 1st and the 31st of August:

If I execute again the stored procedure “mktg_spRuntimeLoadAdvertisements”, I got the exactly same result. Surprised? No! As my ad is not eligible. The start date is on the 1st of August and we are the 8th of July 2010:

Now, copy and paste the “mktg_spRuntimeLoadAdvertisements” stored procedure code and create a new one call “spGael_staging_mktg_spRuntimeLoadAdvertisements”. Just change the start and end date by retrieving the days number between the preview date and now.

You will note that the preview date is stored in a table called “altima_doPreview”. Like that, I can have a simple web part to change the preview date on the fly (don’t forget to refresh your cache).

If we now execute the stored procedure, we got just one result, because the preview date is on the 15th of August and only my ad is eligible:

Note that in my result, the start and end date have switched to 37 days back:

Start date: 1st August 2010 – 37 days => 25 June 2010
End date:, 31 August 2010 – 37 days => 25 July 2010

So, now time to run our preview web application (don’t forget to update the web.config) and miracle… my new ad is now displayed Sourire

Start the default behavior web application and note that my ad is not displayed

To finish, I give you here the code for the discounts stored procedure:

Hope this help!

May 06

How to consume the Commerce Server data for Power Pivot with a RESTful service

.Net, Commerce Server, Development, Power Pivot, SQL Server, WCF 1 Comment »

To follow up my last two posts on Commerce Server and Power Pivot (Create simple and effective reports for Commerce Server with PowerPivot (aka Gemini) et Understand and analyze your Commerce Server data with Power Pivot), I received a couple of questions on how to retrieve the Commerce Server data on a non local environment.

Indeed, most of you have no direct access to the SQL Server databases (I hope so ^^): the e-Commerce farm is hosting in another datacenter with its own vLan to secure the databases. In another word, there are no way to connect to your databases via a simple connection string.

So, what are the options to get the data?

SQL Server replication SQL Server between the production environment and your local environment:
- Pros:
  - Many way to synchronize your databases (transactional , snapshot, …);
  - Strong and tested feature;
- Cons:
  - You must have a SGBD (yes guys! Some of the consultants hasn’t got a SGBD);
  - Expose the databases is not a best practices for a security point of view (even if you filter by IP), to secure it, you must have a dedicated internet line between your datacenter and the different end points. Which is quite expensive;
Export the data in a text format and then import them to the local SGBD:
- Pros:
  - Excel and Power Pivot have a strong text import feature;
  - Excel is well known by the most of the consultants (the same who hasn’t go a SGBD). So, not difficult for them to get the data!
- Cons:
  - Store the files on a local file system is not easy to manage in a long term perspective;
  - Difficult to share the Power Pivot file between different people;
Use the Power Pivot capacity to consume a data feed stream (Have a look on this MSDN article)
- Pros:
  - The databases are not expose to public lan
  - You can still secure the databases access (integrated security for exemple)
  - Xml format is based on the standard market:
    - Atom 1.0 Syndication format
    - Atom 1.0 Publishing Protocol
- Cons:
  - Very new technology so not so much test on it
  - Using a https certificate to ensure the transport between the web service and the end points.

In these 3 scenarii, it’s difficult to say that the solution is only in one of these three. All have pros and cons. It really depends on what you can do and what are the expectations. From myself I decided to show you the data feed option.

To expose your data in asp.net, the best solution is to use a WCF service. This service will be able to render the data in the desired format (here ATOM). Nothing new? Agreed! The trick here is to use the new WCF Data Service. The WCF Data Service enable you a RESTful exposition of your data. In another words, using the OData protocol!

To achieve this, we need to have an application web where we will add a new item : “ADO.Net Entity Data Model” connected to your SQL Server database.

In case of your dashboard are based on the order system, you have to connect to the <site commerce server>_Transactions database.

Then, add an new WCF Data Service item.

In the new created service, addin the InitializeService method the code below:

config.SetEntitySetAccessRule("DiscountsApplied", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("LineItems", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("OrderAddresses", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("OrderForms", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("PurchaseOrderPayments", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("PurchaseOrders", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("Shipments", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("ShippingDiscounts", EntitySetRights.AllRead);

This code, will allow the service to retrieve the table content in read only mode.

You can now call you service in your favorite browser and get the below Atom XML feed. In this feed, you get all the entities that you give the read permission.

As it’s a RESTful service, you can play with the URI to navigate into the data. So for example, add LineItems at the end of the URL (ex: http://localhost//CommerceServerOrdersDataService.svc/LineItems)

If you call again your service, you will see a RSS feed like:

To switch to the xml view, in Internet Explorer, go to Internet Options > Content > Feeds and web slicers settings and untick the feed reading view checkbox.

And voilà! The development part is now finished and you’re able to consume this data feed in Power Pivot:

In Commerce Server, all the web services access are managed by an azman file with integrated security. I will show you in a next post how to do a dedicated one for this data feed service Smile .

For more information:

Oct 28

Don’t forget the ProductConfiguration / CategoryConfiguration in your Commerce Server development

Commerce Server, Development 3 Comments »

These last few weeks, I have been involved with several Commerce Server reviews, and I have noticed that a lot of CS implementations was done without 2 important settings, which are:

These two classes inherit from Microsoft.CommerceServer.Catalog.CatalogItemConfiguration, and they are used to load only the objects you use. Why? Because when you want to display the product price or its description for example, you do not need to get its ancestors, variants, related products so you can have a simple and lighter Product or Category…

Besides, if you have a look into the API via Reflector, you will see that the CategoryConfiguration or ProductConfiguration class is always instantiate if you’re not setting up one (ie, the CS framework choose for you the objects to populate):

To demonstrate to importance of those classes, let’s go through a simple exemple of product implementation where you only need to show the productID, the price and the display name:

Without a ProductConfiguration, we use:

Product product = catalogCtx.GetProduct("Adventure Works Catalog", "AW029-03");
Console.WriteLine("Pid: " + product.ProductId
                + " - " + product.DisplayName
                + " - " + product.ListPrice);

With the SQL Profil we are able to get the SQL calls and discover that 2 stored procedures are call:

Read the rest of this entry »

Oct 26

How to sort different Commerce Server properties in a different order

Commerce Server, Development No Comments »

To follow up a thread in Commerce Server forum: How could i sort by property1 ASC and property2 DESC in a catalog search?, I was surpised that one of the basic Commerce Server feature was not understanding.

It’s a common task in e-Commerce to sort a products’ list (or for a category) in different way for different properties, ie property1 ascending and property2 descending and keep the paging stuck to those properties ordering.

The Ravi’s solution was to use a dataview. But the main issue here is that you can only sort the result returning by the CatalogSearch! What append if you have multiple pages? And on top of that, the poor performance to sort with a DataView…

To achieve this, you simply need to use the common search CatalogSearch class. You just have to surround the propery with sqaure brackets and then the order way:

CatalogSearch catalogSearch = catalogContext.GetCatalogSearch();
catalogSearch.CatalogNames = "MyCatalog";
catalogSearch.SearchOptions.ClassTypes = CatalogClassTypes.ProductFamilyClass;
catalogSearch.SearchOptions.PropertiesToReturn = "ProductId, cy_list_price";
catalogSearch.CategoriesClause = "CategoryName = 'CatId";
catalogSearch.SqlWhereClause = "Display = 'OK'";
catalogSearch.SearchOptions.SortProperty = "[ProductId]ASC, [cy_list_price]DESC";

Be careful, it’s very important to haven’t got any space between the ordering way and the right square bracket.

For information, our Commerce Server developer senior, Anoir, have already wrote a post about it (in french ): Recherche d’un produit avec plusieurs colonnes de tris différenciés in the Altima’s blog.