Finding a suitable PCL profile for a cross-platform library

I recently posted on StackOverflow a question about finding a right profile for a .NET portable class library. For a long time I have packaged Simple.OData.Client, an open source OData client library that I wrote and maintain, as a PCL using one of the broadest possible profiles (Profile147):

Image

Since Profile147 supports .NET framework starting from the version 4.0.3, I had to include in the NuGet package a platform-specific version for .NET 4.0. Later I also added a platform-specific version for Silverlight 5, because I experienced problems with using HttpClient from a PCL targeting Silverlight. So in practice I could change the profile for my library to a more narrow one (Profile78):

Image

But I kept the original profile, mainly to ensure my PCL can be referenced by a largest possible set of other PCLs. When a portable class library references another PCL, it can not target a wider set of platforms than the refenced PCL. So I wanted to cover a case when somebody would write a PCL that targets Silverlight 5 and references Simple.OData.Client.

But targeting legacy platforms comes at some costs. Legacy platforms may lack or have different API signatures (e.g. reflection API), and use of async/await keywords requires installing Microsoft.Bcl.Async NuGet package. More packages, more dependencies, different API revisions. And there’s more. Microsoft’s David Kean, answering a question about .NET portable subsets, uncovered some of the internals: “For legacy platforms (Phone 7.x, SL4/5, .NET 4, Xbox), when we come up with the intersections between multiple platforms, we need to physically generate actual assemblies that represent the common APIs. For example, when you combine Windows Phone 7 and .NET Framework, we generate (these are generated on our side in Microsoft) an actual mscorlib, system, system.core, etc that contain the APIs that these share. This is not only very time consuming, it’s also extremely problematic in that it can generate not very useful subsets.”

David also answered my original question and among other things wrote the following: “Looking at the data internally, I can see that “Profile78” (.NET Framework 4.5, Windows 8, Phone 8, Xamarin.Android, Xamarin.iOS) is overwhelming the most popular target (~45% of portable projects target this). This is not surprising. It targets large breadth of modern platforms with a reasonable large feature support. These platforms that this profile targets, also support what we consider our “modern surface area”, hence have way more shared API surface vs profiles that target platforms that support our legacy surface area (Silverlight, .NET Framework 4).”

It all convinced me that it’s time to revise my profile selection strategy and target the modern surface area. The fact that 45% of portable libraries choose Profile78 also indicates that my original intention to support wider set of platforms for the sake of transitive PCL support wasn’t practically justified: developer move on and target newer platforms.

And to illustrate some other consequences of choosing the legacy surface area, I made a few tests using Xamarin HttpClient sample. The sample contains a core PCL that demonstrates HttpClient capabilities and two mobile applications (for iOS and Android) using that PCL. The original sample uses Profile7 (.NET 4.5 + Windows Store). I built it and successfully ran on both iOS and Android simulators. Then I changed the profile to Profile78 (added Windows Phone 8). Tried to build again, but compilation failed:

xamarin-samples\HttpClientPortable\HttpPortable\NetHttp.cs(18,46,18,56): error CS0234: The type or namespace name 'HttpClient' does not exist in the namespace 'System.Net.Http' (are you missing an assembly reference?)
xamarin-samples\HttpClientPortable\HttpPortable\NetHttp.cs(19,13,19,62): error CS1502: The best overloaded method match for 'HttpPortable.IRenderer.RenderStream(System.IO.Stream)' has some invalid arguments
xamarin-samples\HttpClientPortable\HttpPortable\NetHttp.cs(19,29,19,61): error CS1503: Argument 1: cannot convert from 'void' to 'System.IO.Stream'

So adding Windows Phone 8 requires installing HttpClient NuGet package (Microsoft.Net.Http). I did it from a Package Manager window:

PM> install-package Microsoft.Net.Http
Attempting to resolve dependency 'Microsoft.Bcl (≥ 1.1.3)'.
Attempting to resolve dependency 'Microsoft.Bcl.Build (≥ 1.0.10)'.
Installing 'Microsoft.Bcl.Build 1.0.14'.
Successfully installed 'Microsoft.Bcl.Build 1.0.14'.
Installing 'Microsoft.Bcl 1.1.7'.
Successfully installed 'Microsoft.Bcl 1.1.7'.
Installing 'Microsoft.Net.Http 2.2.19'.
Successfully installed 'Microsoft.Net.Http 2.2.19'.
Adding 'Microsoft.Bcl.Build 1.0.14' to HttpPortable.
Successfully added 'Microsoft.Bcl.Build 1.0.14' to HttpPortable.
Adding 'Microsoft.Bcl 1.1.7' to HttpPortable.
Successfully added 'Microsoft.Bcl 1.1.7' to HttpPortable.
Adding 'Microsoft.Net.Http 2.2.19' to HttpPortable.
Successfully added 'Microsoft.Net.Http 2.2.19' to HttpPortable.

As you see, Microsoft.Net.Http dragged two other packages: Microsoft.Bcl and Microsoft.Bcl.Build. Then applications successfully built and ran.

Next I added Silverlight to the set of supported platforms (Profile158). Silverlight belongs to a legacy surface area, which causes physical generation of code representing the common APIs, as David wrote above. And it requires installing a new NuGet package, because at first the code didn’t compile:

xamarin-samples\HttpClientPortable\HttpPortable\NetHttp.cs(19,29,19,61): error CS4001: Cannot await 'System.Threading.Tasks.Task<System.IO.Stream>'
xamarin-samples\HttpClientPortable\HttpPortable\NetHttp.cs(19,13,19,62): error CS1502: The best overloaded method match for 'HttpPortable.IRenderer.RenderStream(System.IO.Stream)' has some invalid arguments
xamarin-samples\HttpClientPortable\HttpPortable\NetHttp.cs(19,29,19,61): error CS1503: Argument 1: cannot convert from 'void' to 'System.IO.Stream'

Of course, async/await weren’t originally available on Silverlight. Later Microsoft released another BCL package (Microsoft.Bcl.Async) that I also had to install:

PM> install-package Microsoft.Bcl.Async
Attempting to resolve dependency 'Microsoft.Bcl (≥ 1.0.19)'.
Attempting to resolve dependency 'Microsoft.Bcl.Build (≥ 1.0.10)'.
Installing 'Microsoft.Bcl.Async 1.0.166'.
Successfully installed 'Microsoft.Bcl.Async 1.0.166'.
Adding 'Microsoft.Bcl.Async 1.0.166' to HttpPortable.
Successfully added 'Microsoft.Bcl.Async 1.0.166' to HttpPortable.
After this I was able to build and run the sample applications. I also tried even wider profile (Profile147 with support for .NET 4.0.3) that I was using with Simple.OData.Client, and it built fine using the same set of NuGet packages as Profile158.
 
And I think I’ve made my choice now. I will continue support .NET 4.0 and Silverlight 5 by offering platform-specific assemblies in the Simple.OData.Client NuGet package, but the core PCL assembly will only support .NET 4.5, Windows Store, Windows Phone 8 (and of course Xamarin iOS and Android). This will ensure it will use modern surface area without dragging large amount of extra code.

12 reasons to consume OData feeds using Simple.OData.Client

In his great blog post about NuGet dependency monitor Mauricio Scheffer devoted a whole section to WCF Data Services criticism. And it’s well deserved criticism. Anyone who used WCF Data Services client for anything more than a simple demo knows what Mauricio is talking about. It’s been about three years since I was exposed to WCF Data Services, and I remember how pleased I was at first to discover that I could send LINQ expressions to WCF Data Services client that was converting them to OData HTTP requests. But I also remember my disappointment over later discoveries: I couldn’t write LINQ expressions the way I was used to when working with SQL databases. I had to carefully think about the LINQ expression syntax, and test, test, test. They all compiled but I never knew for sure if they are not going to explode at runtime, either on a client or a server side.

Read last section of Mauricio Scheffer’s blog post, and you will understand how fragile WCF Data Services LINQ expressions are. If you want to project query result to a single column, you can’t just write “x.Id” – you need to write “new { x.Id }”. Negate a boolean expression – and you will receive a cryptic error message. Swap Where and Select clauses – and previously working statement no longer works. And all these failures have nothing to do with OData protocol restrictions (such as no support for JOINs). They are all limitations of WCF Data Services LINQ expression parser – mostly on a client side, sometimes on a server side.

My own dissatisfaction with WCF Data Services LINQ support was so big that I even wrote my own OData client. It was originally packaged as a Simple.Data OData adapter, adding to a collection of adapters for an excellent micro-ORM developed by Mark Rendle. However, Simple.Data API is modeled around relational databases thus leaving out some less common OData scenarios, in addition I wanted my client to support Mono and non-Windows mobile platforms. So I extracted from Simple.Data OData adapter a platform independent core and packaged it as Simple.OData.Client (internally used by Simple.Data OData adapter). And of course when I read Mauricio’s post, I was immediately provoked to test on Simple.OData.Client commands that WCF Data Services client failed to execute.

Since you are reading this blog post, you may guess that I was pleased with results. Not only Simple.OData.Client supports features not implemented in WCF Data Services, it offers syntax alternatives, and I will further present variations of its fluent interface: typed and dynamic. If you’re already using Simple.Data, you may stick to more familiar Simple.Data OData adapter, although Simple.OData.Client supports more advanced OData protocol features and is packaged as portable class library that can be used on non-Windows platforms, such as iOS and Android. It also gives a choice of either typed or dynamic API (so it effectively offers multiple API flavors). I will show examples using both typed and dynamic Simple.OData.Client syntax, but every example will be originated with its WCF Data Services counterpart. And you can judge yourself which API works better.

Libraries used in comparison

WCF Data Services client

Visual Studio has built-in support for generating OData client access code: simply choose “Add Service Reference…” from project context menu and enter a URL of the existing OData service. You will get proxy code that contains entities (OData service resources), and you will be able to manipulate them using Entity Framework-alike API. Well, as long as you manage to get LINQ statements right. In order to get most recent WCF Data Services client components, install them from NuGet.

Simple.OData.Client

Simple.OData.Client (also available from NuGet) is a portable class library that once has been extracted from Simple.Data OData adapter. It has its own Wiki pages at GitHub, but I believe the easiest way to understand its features is to check its tests. As I mentioned earlier, it can target various platforms including .NET 4.x, Windows Store, Silverlight 5, Windows Phone 8, iOS and Android. If you need to consume OData feeds consisting of interlinked resource sets I believe it provides the simplest yet powerful API with typed and dynamic flavors.

Retrieving data

In .NET world data sources are expected to have LINQ providers, and sometimes LINQ providers are written even for databases that don’t expose its data in tabular or relational form. So it was quite logical for Microsoft developers to offer a LINQ provider for clients of its OData protocol, and this provder has grown from a project “Astoria” to become WCF Data Services. They deliberately made client interface look like to LINQ to Entities, so a simple LINQ statement is indistinguishable from a LINQ to Entities call:

var product = ctx.Products.Where(x => x.ProductName == “Chai”).Single();

I am using here fluent LINQ syntax, but of course all these statements can be written using query comprehension. The variable “ctx” represents the data context that is responsbile for translating commands to HTTP requests and sending them to an OData service.

As I already mentioned, Simple.OData.Client offers several syntax flavors, we will focus on two of them: a typed and a dynamic ones. A typed request to an OData service will be like like this:

var product = client.For<Products>().Filter(x => x.ProductName == “Chai”).FindEntry();

And here’s a dynamic one:

var x = ODataDynamic.Expression;
var product = client.For(x.Products).Filter(x.ProductName == “Chai”).FindEntry();

Note that when using dynamic API you need first to define an instance of a dynamic expression object (“x”) that can be used for the rest of the code block (even in multiple calls) in forms of various expressions that are evaluated at runtime and converted to parts of OData commands.

Now that we have a basic idea of how to query OData services using different OData client libraries, let me guide you through some specific examples and show why WCF Data Services client may lose its attractiveness.

Reason 1: projection to a single column

WCF Data Services

This is a widely used scenario, therefore it comes as a complete surprise that it’s not supported by WCF Data Services. Here’s an example:

var productIDs = ctx.Products.Select(x => x.ProductID);

If you run it it will fail with the following error:

System.NotSupportedException : Individual properties can only be selected from a single resource or as part of a type. Specify a key predicate to restrict the entity set to a single instance or project the property into a named or anonymous type.

“As part of a type”? Let’s select it as a part of an anonymous type. Then it will work:

var productIDs = ctx.Products.Select(x => new { x.ProductID });

Like Mauricio Scheffer, I don’t see any logic in this restriction. LINQ supports single column projections, WCF Data Services server side supports single column projections. This is a pure client side LINQ expression handler limitation which should have been fixed long ago.

Simple.OData.Client (typed)

Here’s the code using that uses typed entities:

var productIDs = client.For<Products>().Select(x => x.ProductID).FindEntries(); 

Simple.OData.Client (dynamic)

Finally dynamic version of the code:

IEnumerable<dynamic> productIDs = client.For(x.Products).Select(x.ProductID).FindEntries();

Note that we can use “Products” instead of “Product”. Like Simple.Data, Simple.OData.Client has a simple built-in word matcher that accepts both singular, multiple and underscore-separated names and tries to find the matching metadata element.

Reason 2: reduntant query conditions

This is perhaps not a common scenario, however it may occur when LINQ expression is built in a loop, and it’s a perfectly valid LINQ expression. It just contains redundant clause.

WCF Data Services

This code will fail:

var product = ctx.Products.Where(x => x.ProductID == 1 && x.ProductID == 1).Single();
System.NotSupportedException : Multiple key predicates cannot be specified for the same entity set.

Sure you can’t look up an entity using multiple keys. But if the second key clause is identical to the first one, why not simple optimize the expression and remove redundancy? Simple.OData.Client performs it fine.

Simple.OData.Client (typed)

var product = client.For<Product>()
    .Where(x => x.ProductID == 1 && x.ProductID == 1).FindEntry();

Simple.OData.Client (dynamic)

var product = client.For(x.Products)
    .Select(x.ProductID == 1 && x.ProductID == 1).FindEntry();

Reason 3: deceiving LINQ extension methods

WCF Data Services

Let’s have a look at the statement similar to the one used in the previous example:

var product = ctx.Products.Where(x => x.ProductID == 1).Single();

If you have ReSharper installed on your machine, you should a green wavy line under that statement. Hover the mouse, and ReSharper will display a tooltip suggesting replacing Where and Single clauses with just one call:

LinqWarning

This is a natural code simplification supported by most LINQ providers. Unfortunately LINQ to OData provider doesn’t support such syntax optimization, and if you press Alt+Enter to accept the suggestion and replace the code above with the command

var product = ctx.Products.Single(x => x.ProductID == 1);

… you will get the following error:

System.NotSupportedException : The method 'Single' is not supported.

Too bad. Single, First, Last, SingleOrDefault etc. – all these LINQ extension methods must be applied to the result collection after the query execution completes, you can not send them as a part of a server request. To make the matter worse, you won’t get any warning if you use one of these statements as a server request – quite opposite, as we just saw, you will get a ReSharper notice if you don’t use them. The error will happen later, at runtime.

Simple.OData.Client use different methods to retrieve single and multiple results to ensure server-side result optimization:

Simple.OData.Client (typed)

// Returns first result 
var products = client.For<Product>().Where(x => x.ProductID == 1).FindEntry(); 
// Returns all results
var products = client.For<Product>().Where(x => x.ProductID == 1).FindEntries();

Simple.OData.Client (dynamic)

// Returns all results
var product = client.For(x.Products).Where(x.ProductID == 1).FindEntries();
// Returns all results
var product = client.For(x.Products).Where(x.ProductID == 1).FindEntry(); 

When retrieving multiple results, you can of course append Single/First/Last LINQ extension methods to the statements as you would do when using LINQ provider. This statements will be applied to the result collection after it has been retrieved from the server.

Reason 4: swapping command clauses

There should be no reason why you couldn’t swap Where and Select clauses? Let’s try.

WCF Data Services

var productID = ctx.Products
    .Where(x => x.ProductID != 1).Select(x => new { ProductID }).First();

The statement above works fine (note that we had to use anonymous type in the Select clause, otherwise it would have failed). Now we’ll apply Where to the projection:

var productID = ctx.Products
    .Select(x => new { ProductID }).Where(x => x.ProductID != 1).First();

ReSharper immediately comes with the suggestion to merge Where and First into a single clause, but we know it’s going to fail. Unfortunately the whole statement is going to fail anyway:

System.NotSupportedException : The filter query option cannot be specified after the select query option.

Simple.OData.Client doesn’t have this problem:

Simple.OData.Client (typed)

var productID = client.For<Product>()
    .Where(x => x.ProductID != 1).Select(x => x.ProductID).FindEntry();
var productID = client.For<Product
    .Select(x => x.ProductID)>().Where(x => x.ProductID != 1).FindEntry();

Simple.OData.Client (dynamic)

var productID = client.For(x.Products)
    .Where(x.ProductID != 1).Select(x.ProductID).FindEntry();
var productID = client.For(x.Products)
    .Select(x.ProductID)).Where(x.ProductID != 1).FindEntry(); 

Reason 5. Expanding related entities

Although OData protocol does not support joins, it has a notion of related entities that can be navigated and expanded. So if Products entities has columns CategoryID referring to Categories entities and there is a Products.Category association defined in service metadata, it should be possible to navigate to Categories from Products via Category association.

WCF Data Services

WCF Data Services LINQ provider has a custom method Expand that can be used for eager loading of related entities. So if we want to expand products with associated categories we might use the following statement:

var products = ctx.Products.Where(x => x.ProductName == “Chai”).Expand(x => x.Category);

Nope! It doesn’t even compile. And the reasons for this is custom Expand method applied after Where clause. “Where” is a standard extension method that returns IQueryable, but Expand is defined on DataServiceQuery class that is a part of WCF Data Services, so it can only be applied to generated proxy collection classes (“Products”, “Categories” etc.), and can’t be interchanged with standard LINQ extension methods.

Again, we are facing a problem with swapped LINQ command clauses, now due to use of custom method. Here’s the statement that works:

var products = ctx.Products.Expand(x => x.Category).Where(x => x.ProductName == “Chai”);

But what if we want to add a projection clause to this statement and only select the category? Let’s try:

var categories = ctx.Products.Expand(x => x.Category).Where(x => x.ProductName == “Chai”)
    .Select(x => x.Category);

Hmm, an error:

System.NotSupportedException : Can only specify query options (orderby, where, take, skip) after last navigation.

Maybe this is because we didn’t use anonymous type in the Select clause? Alright, another attempt:

var categories = ctx.Products.Expand(x => x.Category).Where(x => x.ProductName == “Chai”)
    .Select(x => new { x.Category });

Oops, this time the error message is different:

System.NotSupportedException : Cannot create projection while there is an explicit expansion specified on the same query.

But at least it gives us some clue. It’s not permitted to combine Expand and Select clauses. Out of my understanding, but let’s drop Expand then:

var categories = ctx.Products.Where(x => x.ProductName == “Chai”)
    .Select(x => new { x.Category });

It worked now, but you just saw how fragile the WCF Data Services LINQ provider. You just have to keep on trying different syntax variations until you find something that won’t fail at runtime.

For eager loading of relations Simple.OData.Client defines a method Expand that we can freely swap with other methods of its fluent interface, including filtering and projection:

Simple.OData.Client (typed)

var product = client.For<Products>()
    .Expand(x => x.Category).Filter(x => x.ProductName == "Chai").FindEntry();

var product = client.For<Products>().Filter(x => x.ProductName == "Chai")
    .Expand(x => x.Category).FindEntry();

var product = client.For<Products>().Expand(x => x.Category).Filter(x => x.ProductName == "Chai")
    .Select(x => new { x.ProductID, x.Category.CategoryID }).FindEntry();

Simple.OData.Client (dynamic)

var product = client.For(x.Products)
    .Expand(x.Category).Filter(x.ProductName == "Chai").FindEntry();

var product = client.For(x.Products)
    .Filter(x.ProductName == "Chai").Expand(x.Category).FindEntry();

var product = client.For(x.Products).Expand(x.Category).Filter(x.ProductName == "Chai")
    .Select(x.ProductID, x.Category.CategoryID).FindEntry();

Reason 6: navigating to a related entity

In addition to expanding relations OData protocol supports navigation to related entities, so it should be possible to bypass the parent entry and go directly to its relations. There is a constraint on search conditions: the parent entry should be looked up by its key, so the navigation is performed from a single entry.

WCF Data Services

WCF Data Services doesn’t support navigation in a strict sense, but as we saw in the previous section, it’s possible to simulate navigation using projection on a property that represents the relation:

var categories = ctx.Products.Where(x => x.ProductID == 1)
    .Select(x => new { x.Category });

This is not a real navigation to a related entity for two reasons. First, the statement returns a collection of anonymous types, not category objects, so we need to access the actual Category object through a Category property of the anonymous type. Second, the statement generates a different OData command: filter instead of a key lookup. So in principle you can retrieve categories for multiple products, for example by issuing the following command:

var categories = ctx.Products.Where(x => x.ProductName != 1)
    .Select(x => new { x.Category });

Unlike WCF Data Services client, Simple.OData.Client has a proper support for navigation using method NavigateTo:

Simple.OData.Client (typed)

var category = client.For<Product>()
    .Where(x => x.ProductName == “Chai”).NavigateTo<Category>().FindEntry();

Simple.OData.Client (dynamic)

var category = client.For(x.Product)
    .Where(x.ProductName == “Chai”).NavigateTo(x.Category).FindEntry();

Reason 7: chained navigation

WCF Data Services

Due to lack of proper navigation support WCF Data Services client can’t be used to fetch results of multi-level navigation. You can only retrieve the information that belongs to an immediate descendant, an attempt to reach deeper levels won’t work:

var product = ctx.Products
    .Where(x => x.ProductID == 1).Select(x => new { x.Category, x.Category.Products })

The above statement will succeed, but product.Category.Products will contain no elements. For the same reason the following statements will both fail with “sequence contains no elements” exception:

var employee = ctx.Employees.Where(x => x.EmployeeID == 14)
    .Select(x => new { x.Superior })
    .Select(x => new { x.Superior.Superior }).First();

var employees = ctx.Employees.Where(x => x.EmployeeID == 14)
    .Select(x => new { x.Superior })
    .Select(x => new { x.Superior.Subordinates });

Simple.OData.Client is another story: it fully supports navigation, so all the above scenarios can be easily expressed using both typed and dynamic syntax:

Simple.OData.Client (typed)

var products = client.For<Products>().Filter(x => x.ProductID == 1)
    .NavigateTo(x => x.Category)
    .NavigateTo(x => x.Products)
    .FindEntries();

We can perform deeper navigation, navigating from an employee to a superior of superior:

var employee = client.For<Employees>().Filter(x => x.EmployeeID == 14)
    .NavigateTo(x => x.Superior)
    .NavigateTo(x => x.Superior)
    .FindEntry();

Or we can fetch multiple results if we request subordinates of the superior:

var employees = client.For<Employees>().Filter(x => x.EmployeeID == 14)
    .NavigateTo(x => x.Superior)
    .NavigateTo(x => x.Subordinates)
    .FindEntries();

Simple.OData.Client (dynamic) Here are similar statements for dynamic API:

var products = client.For(x.Products).Filter(x.ProductID == 1)
    .NavigateTo(x.Category)
    .NavigateTo(x.Products)
    .FindEntries();
var employee = client.For(x.Employees).Filter(x.EmployeeID == 14)
    .NavigateTo(x.Superior)
    .NavigateTo(x.Superior)
    .FindEntry();
var employees = client.For(x.Employees).Filter(x.EmployeeID == 14)
    .NavigateTo(x.Superior)
    .NavigateTo(x.Subordinates)
    .FindEntries();

Reason 8: selecting a derived type entity

Let’s have a brief look at rather advanced (and rarely used in OData services) scenario: subclassing OData resource types. Personally I don’t fancy bringing OOP to RESTful (or REST-alike) Web services, but since OData protocol has support for base and derived resource types, it’s fair to examine how our libraries deal with it.

So far I have been using classical Northwind model, but it doesn’t have derived classes. So I extended it with a Transport and Ship entity type: Ship subclasses Transport and add ShipName property. TransportType property on a base class is used as a type discriminator.

WCF Data Services

Selecting a Transport instance without regards to its type is straightforward:

var transport = ctx.Transport.First();

Next is to select only ships:

var ship = ctx.Transport.Where(x => x is Ships).First();

To gain access to ShipName property we need to cast the result:

var shipName = ctx.Transport.Where(x => x is Ships).First() as Ships;

What if we want to search by ship name? This is also possible:

var ship = ctx.Transport
    .Where(x => x is Ships && (x as Ships).ShipName == “Titanic”).First();

From the last example you may imagine that you can cast the results to the derived type using Select clause, so you will get back properly typed results:

var ships = ctx.Transport.Where(x => x is Ships).Select(x => x as Ships);

But this won’t work:

System.NotSupportedException : Unsupported expression '(x As Ships)' in 'Select' method. Expression cannot end with TypeAs.

You always obtain results as a base type and will need to cast them to a derived type on a client side. However, comparing to problems we experienced with other queries, derived type management in WCF Data Services works almost as expected. But let’s check how it is implemented in Simple.OData.Client, and I believe it will become clear which syntax is more intuitive.

Simple.OData.Client (typed)

Simple.OData.Client has a generic method As<T> that redefines the type of entities being processed. Once the cast is performed, all subsequent calls on the fluent chain will be made using that type:

var transports = client.For<Transport>().FindEntries();
var ships = client.For<Transport>().As<Ships>().FindEntries();
var ship = client.For<Transport>().As<Ships>().Filter(x => x.ShipName == "Titanic").FindEntry();

Simple.OData.Client (dynamic)

In a similar manner, dynamic Simple.OData.Client API has a method As, it’s just that this method is not generic and the derived entity type is evaluated based on the dynamic expression parameter:

var transports = client.For(x.Transport).FindEntries();
var ships = client.For(x.Transport).As(x.Ships).FindEntries();
var ship = client.For(x.Transport).As(x.Ships).Filter(x.ShipName == "Titanic").FindEntry();

Modifying data

WCF Data Services client use same methods as Entity Framework to add, update and delete OData resources, it also has a notion of data context, so all operations are queued until the call to SaveChanges and only then are sent to an OData service. Simple.OData.Client doesn’t have a concept of data context (although multiple operations can be packed in batches according to OData protocol), but eventually data modification methods provided by both libraries are mapped to equivalent HTTP requests.

Here’s an example of creating an entry using WCF Data Services client:

var product = new Products { ProductName = "Milk" };
ctx.AddToProducts(product); 
ctx.SaveChanges();

And here’s Simple.OData.Client counterparts:

var product = client.For<Products>().Set(new { ProductName = “Milk” }).InsertEntry();

or

var product = client.For<Products>().Set(new Products { ProductName = “Milk” }).InsertEntry();

or

var product = client.For(x.Products).Set(x.ProductName = “Milk”).InsertEntry();

The important difference is that in WCF Data Services API a central role is given to the context. It’s a container, a stack of operations and entities. Whatever you intend to do must go via the context, it needs to track all entities prior them to be sent to the server, and it’s developer’s responsibility to ensure the entities are tracked. This is what makes it quite different from Entity Framework that is capable of auto-tracking the affected entities. As long as you are dealing with isolated entities, it shouldn’t be inconvenient. But relationship management has some pitfalls. Let’s have a look at some of them and compare with how this can be done using Simple.OData.Client.

Reason 9: Assigning a linked entity

WCF Data Services

The following code looks reasonable except that it doesn’t work:

var category = ctx.Categories.Where(x => x.CategoryName == "Beverages").Single();
var product = new Products { ProductName = “Chai”, Category = category };
ctx.AddToProducts(product);
ctx.SaveChanges(); 

The code doesn’t fail and the product is created, but it is not linked to a category. Unlike Entity Framework that detects assignments of association properties, the context (“ctx”) needs to be explicitly notified about the intention to link the product to the category. So we need to modify the code, and if you search for available options you will find several choices: AddLink, AddRelatedObject and SetLink. Which one is the right in every particular scenario depends on relation cardinality. In the example above SetLink is the right one, so the code that works looks like this:

var category = ctx.Categories.Where(x => x.CategoryName == "Beverages").Single();
var product = new Products { ProductName = “Chai”, Category = category };
ctx.AddToProducts(product);
ctx.SetLink(product, "Category", category);
ctx.SaveChanges();

There are several reasons why I dislike such approach:

  • A library could have been smarter to figure out that the object has an association property assigned and take the burden of additional linking code from developers;
  • The service metadata has all information about relation cardinality, so there is no need to force developers to be explicit about what kind of relation they are establishing;
  • Use of magic strings sucks. WCF Data Services uses lambda expressions in other methods, it should have allowed lambdas here too.

Now let’s see how linking is managed by Simple.OData.Client

Simple.OData.Client (typed)

var category = client.For<Categories>().Filter(x => x.CategoryName == "Beverages").FindEntry();
var product = client.For<Products>()
    .Set(new { ProductName = "Chai", Category = category }).InsertEntry();

Couldn’t be easier, could it? And here’s a dynamic version:

Simple.OData.Client (dynamic)

var category = client.For(x.Categories).Filter(x.CategoryName == "Beverages").FindEntry();
var product = client.(x.Products)
    .Set(new { ProductName = "Chai", Category = category }).InsertEntry(); 

Reason 10: Detaching a linked entity

WCF Data Services

I guess you are already prepared that detaching a link won’t be easy, and I can assure you it won’t be. In fact it was even more difficult than I thought. No need to say that the following code won’t work:

var product = ctx.Products.Where(x => x.ProductName == "Chai").Single();
product.Category = null;
ctx.UpdateObject(product);
ctx.SaveChanges();

The code doesn’t fail, it just doesn’t affect anything. Again, due to the unaware (or should I say ignorant) context. But having learned about SetLink method in the previous section we might try that:

var product = ctx.Products.Where(x => x.ProductName == "Chai").Single();
ctx.SetLink(product, "Category", null);
ctx.UpdateObject(product);
ctx.SaveChanges();

Nope. Doesn’t work either. The WCF Data Services context doesn’t treat a null association object as an intention to detach the link. But there is a DetachLink method, let’s try that instead:

ctx.DetachLink(product, "Category", category);

No, the relationship is still there. Perhaps DeleteLink?

ctx.DeleteLink(product, "Category", category);

This time the code fails at runtime:

System.InvalidOperationException : AddLink and DeleteLink methods only work when the sourceProperty is a collection.

Let’s change it then so the link is deleted from a category:

ctx.DeleteLink(product, "Category", category);

No, if I reload the product it still shows the linked category. But apparently this is due to the internal cache in the context object. If I clear the context and reload the product the link will disappear. So after some frustrations here is the code that works:

var category = ctx.Categories.Where(x => x.CategoryName == "Beverages").Single();
var product = ctx.Products.Where(x => x.ProductName == "Chai").Single();
ctx.DeleteLink(category, "Products", product);
ctx.UpdateObject(product);
ctx.SaveChanges();
// Without this call if you reload the project it will still show a linked category!
ctx.DetachProduct();

Well, you can judge yourself how intuitive the above code is. And now how the same thing can be achieved using Simple.OData.Client.

Simple.OData.Client (typed)

var product = client.For<Products>().Filter(x => x.ProductName == "Chai").FindEntry();
var category = client.For<Categories>().Filter(x => x.CategoryName == "Beverages").FindEntry();
client.For<Products>().Key(product.ProductID).UnlinkEntry(x => x.Category);

Note use of Key clause. I could use Filter clause instead as long as the filter condition represented a key lookup.

Simple.OData.Client (dynamic)

var product = client.For(x.Products).Filter(x.ProductName == "Chai").FindEntry();
var category = client.For(x.Categories).Filter(x.CategoryName == "Beverages").FindEntry();
client.For(x.Products).Key(product.ProductID).UnlinkEntry(x.Category);

An alternative to using UnlinkEntry method is to set the Category property to null and call UpdateEntry.

Reason 11: Modifying multiple entries

Neither Entity Framework nor WCF Data Services client has built-in support for update of multiple entries based on search condition. OData protocol doesn’t offer this feature either – you have to fetch query results (or at least its key values) and then go through them and update one after another. But this is not uncommon scenario when you need to change or delete all entities matching certain criteria, so Simple.OData.Client offers this functionality.

WCF Data Services

As I said, WCF Data Services client doesn’t expose a method to modify or delete multiple results of a query execution in a single class, so the you will have to do something like this:

foreach (var product in ctx.Products.Where(x => x.ProductName.StartsWith(“A”))
{
    ctx.DeleteObject(product);
}
ctx.SaveChanges();

Simple.OData.Client offers methods UpdateEntries and DeleteEntries that takes care of querying the OData service and traversing the result set. Below are the examples.

Simple.OData.Client (typed)

client.For<Products>().Filter(x => x.ProductName.StartsWith(“A”)).DeleteEntries();

Simple.OData.Client (dynamic)

client.For(x.Products).Filter(x.ProductName.StartsWith(“A”)).DeleteEntries();

Final thoughts

Reason 12: REST is not SOAP

We have gone through several examples that demonstrate obstacles of reading and updating OData feeds with WCF Data Services clients. However there is one more, on a conceptual level. Microsoft has chosen to present OData services to Visual Studio users as if it were old fashioned SOAP services described with its WSDL files. I believe that majority of developers who are first introduced to OData services when they choose “Add Service Reference” from a project menu don’t even realize that they are exposed to REST communication. From within Visual Studio it all looks like a traditional WCF service with a contract and data types. Sure, OData resource types may be looked upon as classes with DataContract attribute, and HTTP verbs GET, POST, PUT/MERGE and DELETE are similar to WCF service operations. But analogy with WCF, SOAP and event Entity Framework is misleading. I’d rather see better exposure of HTTP verbs, so developers have a clear picture of what operations are safe, idempotent or neither of them.

While Simple.OData.Client API doesn’t include methods named after HTTP verbs, it has methods with straightforward mapping to them. FindEntries and FindEntry correspond to GET, they retrieve either all or first matching result. InsertEntry, UpdateEntry and DeleteEntry are mapped to POST, PUT/MERGE and DELETE respectively. LinkEntry and UnlinkEntry provide higher abstraction over calls HTTP verbs to establish or remove relationships between entities. These methods (all named <Verb>Entry or <Verb>Entries) are always final clauses in the chain of Simple.OData.Client fluent interface calls. All but last elements in the call chain incrementally build an OData request, and the final call to <Verb>Entry invokes the corresponding HTTP command. I believe that such API gives a better understanding of underlying HTTP communication and help developers make efficient use of OData protocol.

We need better interoperability between dynamic and statically compiled C#

It’s 2013 now, .NET DLR has been out for more than three years, and there are plenty of great libraries around with clever mix of statically compiled and dynamically evaluated code. I am guilty of one myself.

Unfortunately ‘dynamic’ keyword doesn’t only bring joy to the art of programming. Its power often results in unintended behavior that can usually be explained by digging deep into language implementation, but still looking unreasonable.

Look at the interface below. its first method returns a reference to the interface itself, and the second method returns a result, so both methods can be chained:

public interface IFluentClient
{
    IFluentClient SaveText(string text);
    string GetText();
}

Now in order to be able to use this interface we need a class that implements it. Here it is:

class FluentClient : IFluentClient
{
    private string _text;

    public IFluentClient SaveText(string text)
    {
        _text = text;
        return this;
    }

    public string GetText()
    {
        return _text;
    }
}

This class is not public and we can’t instantiate it in a different assembly, so we will create a factory for it:

public class ClientFactory
{
    public IFluentClient GetFluentClient()
    {
        return new FluentClient();
    }
}

Finally we can write a test for the code above:

[TestMethod]
public void SaveAndGetText()
{
    var client = new ClientFactory().GetFluentClient();
    var text = client.SaveText("a").GetText();
    Assert.AreEqual("a", text);
}

So far so good. The test passes of course. Now we will send a dynamic object to SaveText:

[TestMethod]
public void SaveAndGetText()
{
    dynamic str = "a";
    var client = new ClientFactory().GetFluentClient();
    var text = client.SaveText(str).GetText();
    Assert.AreEqual("a", text);
}

Let’s run the test and it will fail with the following error:

Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: 'object' does not contain a definition for 'GetText'

Out of curiosity I tested this code on Mono, and it failed too, with a stranger error:

FluentClient.GetText() is inaccessible due to its protection level

Obviously a string casted to a dynamic object confused the runtime. But why? Eric Lippert in a blog post explained some of the reasons why the DLR-to-CLR interoperability was implemented in a way that makes the runtime treat statically compiled code as dynamic after a dynamic object enters the scene. He defends the decision by stating that “if you and I and the compiler know that overload resolution is going to choose a particular method then why are we making a dynamic call in the first place?” This argument was probably fair enough some time ago when developers were still new to DLR and used dynamic C# only on special occasions. I’d say this is no longer a good argument. It’s easy and convenient to mix statically compiled and dynamic C#, and the language should make their interoperability more natural.

How can we make the failing test work? For example by breaking a fluent call chain:

[TestMethod]
public void SaveAndGetText()
{
    dynamic str = "a";
    var client = new ClientFactory().GetFluentClient();
    client = client.SaveText(str);
    var text = client.GetText();
    Assert.AreEqual("a", text);
}

Suddenly it all works, even with a use of dynamic object. But we can even make the original code work without changing a single line in the test code! The strange error message on Mono that complained about protection level gave me a hint: what if I change the visibility of FluentClass changing its protection level from private to public? Bingo! Suddenly all tests passed. Both on Windows and Mono platforms. But who could expect such behavior?

Use of dynamic code requires more thorough testing. But as you could see in examples above, it sometimes requires rewriting some calls or class definitions without changing any functionality, only because .NET runtime accepts one but not other perfectly valid syntax. This makes interoperability between CLR and DLR fragile. I believe there is a need to make it more predictable and developer-friendly.

Announcing Simple.OData.Client NuGet package for Mono iOS/Android

Release of NuGet 2.5 opens possibilities for targeting Mono iOS/Android platform. The Simple.OData.Client that I maintain is a portable class library, so it was natural step to extend the number of supported frameworks with Xamarin Mono offerings. But as it often happens, I hit a few problems before I could publish the updated package (version 0.15).

One problem I had to deal with was forwarding of the types that are packaged differently in Windows and .NET. Luckily, this problem was addressed earlier by other people, so big thanks to Stuart Lodge (@slodge) and Daniel Plastied (@dsplaisted) for investigating this topic and describing workarounds. Those interested in more details should read this article at StackOverflow. I am now using the same approach that Stuart implemented in MvvmCross, and here is how the file sets for Mono platforms look:

<!—Droid –>
<file src="Simple.OData.Client.Core\bin\Release\Simple.OData.Client.Core.dll" 
      target="lib\MonoAndroid16\Simple.OData.Client.Core.dll" />
<file src="lib\Droid\System.Net.dll" target="lib\MonoAndroid16\System.Net.dll" />

<!-- Touch –>
<file src="Simple.OData.Client.Core\bin\Release\Simple.OData.Client.Core.dll" 
      target="lib\MonoTouch40\Simple.OData.Client.Core.dll" />
<file src="lib\Touch\System.Net.dll" target="lib\MonoTouch40\System.Net.dll" />

Next challenge came from a couple of places in my code where I was using async/await pattern. Since Simple.OData.Client is targeting wide range of platforms including .NET 4.x and Silverlight, I installed Microsoft.Bcl.Async NuGet package that enabled use of async/await keywords when targeting legacy platforms. However, for the time being the files from this package can not be deployed on iOS and Android devices (legal reasons), and Xamarin async/await support is a work in progress. After struggling for a few days with various async issues (only compile-time issues, I haven’t had a single runtime problem with asynchronous operations), I reviewed the code and figured out that it’s so few places where I was using await, that I can rewrite this code replacing await with TPL API almost in no time. So now Simple.OData.Client does not bear any dependency on Microsoft.Bcl.Async.

Finally, I had to write new tests to verify functionality on new platforms. I used an excellent little helper library MonoDroidUnitTest for Android tests, and for iOS there was a Unit Test project template in Xamarin tools. Since I don’t have Mac, I signed in to MacInCloud to test using iOS simulator. It kind of works, but the performance is sluggish so I don’t think this is a good alternative for a serious iOS development (but OK for occasional validation). It was easier with Android – I could simply start a Droid simulator, loaded an app with a few tests, and in a few seconds (well, actually minutes) I could see the following picture:

Android2 

So in case you are doing OData development and need a library that you can use on multiple platforms, have a look at Simple.OData.Client. Now with Mono support.

Cross-platform design-time view models using portable class libraries

Stuart Lodge is working on a fantastic series of videos and blog posts showing how to build cross-platform mobile applications with MvvmCross. One of his tips is about exposing design-time data. His method is straightforward and efficient, but in case you don’t want to copy sample files to a location inside Windows Program Files folder, you may consider other alternatives, such as storing design-time information as project content files or embeding it into an assembly.

This task becomes more complicated in case you want to show some images at design-time, and BitmapImage class is not part of portable class libraries. I used to solve it by representing view model image properties using opaque object class and setting its values in platform-specific part of the view models. The unfortunate consequence of this approach was declaring a view model per platform serving design-time purposes. The platform view models inherited from a common portable view model and only added a tiny bit of non-portable logic – like reading image resources. Here is how I used to show design-time data in my ODataPad application:

public class ServiceViewModel 
{ 
    public string Name { get; set; } 
    public string Description { get; set; } 
    public string Url { get; set; } 
    public object Image { get; set; } 
} 

public class WinRTDesignHomeViewModel : DesignHomeViewModel 
{ 
    public WinRTDesignHomeViewModel() 
    { 
        foreach (var service in this.Services) 
        { 
            service.Image = new BitmapImage(
                new Uri("ms-appx:///Samples/" + service.Name + ".png")); 
        } 
    } 
} 

public class Net45DesignHomeViewModel : DesignHomeViewModel 
{ 
    public Net45DesignHomeViewModel() 
    { 
        foreach (var service in this.Services) 
        { 
            service.Image = new BitmapImage(
                new Uri(@"pack://application:,,,/ODataPad.UI.Net45;component/Samples/" +
                    service.Name + ".png")); 
        } 
    } 
}

XAML files for Windows Store and WPF applications included Image element with binding to an Image property of ServiceViewModel.

<Image Source="{Binding Image}" Stretch="UniformToFill" />

Here are the screenshots of design-time views:

ODataPadWinRT

ODataPadNet45

ODataPadWP8

This method works but as I already pointed out, it feels heavy because each design-time data set is backed with its own design-time view model – due to non-portability of image .NET types and image management methods.

But there is another way – storing image properties using portable data types, and what can be more portable than a standard string? Enter base64 strings.

Here’s a revised ServiceViewModel class with image data stored as strings:

public class ServiceViewModel 
{ 
    public string Name { get; set; } 
    public string Description { get; set; } 
    public string Url { get; set; } 
    public string ImageBase64 { get; set; } 
}

We no longer need WinRTDesignHomeViewModel and Net45DesignHomeViewModel classes. Instead we will store all design-time information in a portable class library as an embedded resource. Not that Assembly.GetManifestResourceStream method is part of most of portable class libraries profiles, so we can share the following PCL code between all platforms:

public DesignHomeViewModel()
{
    IEnumerable services = null;
    var namespaceName = typeof(DesignHomeViewModel).Namespace;

    var stream = typeof (DesignHomeViewModel).Assembly
        .GetManifestResourceStream(string.Join(".", namespaceName, "SampleServices.xml"));
    using (var reader = new StreamReader(stream))
    {
        this.Services = SamplesService.ParseSamplesXml(reader.ReadToEnd());
    }
}

Wait, but that can’t be sufficient – our XAML views contain Image elements, we can’t just throw base64 strings at them. Well, almost: it’s all about converters.

Here’s revised XAML code:

<Image Source="{Binding ImageBase64, Converter={StaticResource Base64ToImage}}" 
       Stretch="UniformToFill"/>

And these are converers for Windows Store, WPF and Windows Phone applications:

namespace ODataPad.UI.WinRT.Common
{
    public sealed class Base64ImageConverter : IValueConverter
    {
        public object Convert(object value, Type targetType, 
                              object parameter, string language)
        {
            return ConvertAsync(value, targetType, parameter, language).Result;
        }

        public object ConvertBack(object value, Type targetType, 
                                  object parameter, string language)
        {
            return null;
        }

        public async Task ConvertAsync(object value, Type targetType, 
                                       object parameter, string language)
        {
            var image = new BitmapImage();

            if (value != null)
            {
                var bytes = System.Convert.FromBase64String((string)value);

                var ras = new InMemoryRandomAccessStream();
                using (var writer = new DataWriter(ras.GetOutputStreamAt(0)))
                {
                    writer.WriteBytes(bytes);
                    await writer.StoreAsync();
                }

                image.SetSource(ras);
            }
            return image;
        }
    }
}

namespace ODataPad.UI.Net45.Common
{
    public class Base64ImageConverter : IValueConverter
    {
        public object Convert(object value, Type targetType, 
                              object parameter, CultureInfo culture)
        {
            var image = new BitmapImage();
            if (value != null)
            {
                var bytes = System.Convert.FromBase64String((string)value);

                image.BeginInit();
                image.StreamSource = new MemoryStream(bytes);
                image.EndInit();
            }
            return image;
        }

        public object ConvertBack(object value, Type targetType, 
                                  object parameter, CultureInfo culture)
        {
            return null;
        }
    }
}

namespace ODataPad.UI.WP8.Common
{
    public class Base64ImageConverter : IValueConverter
    {
        public object Convert(object value, Type targetType, 
                              object parameter, CultureInfo culture)
        {
            var image = new BitmapImage();
            if (value != null)
            {
                var bytes = System.Convert.FromBase64String((string)value);
                image.SetSource(new MemoryStream(bytes));
            }
            return image;
        }

        public object ConvertBack(object value, Type targetType, 
                                  object parameter, CultureInfo culture)
        {
            return null;
        }
    }
}

Now all my design-time data are stored in a single portable class library and I display them using a single view model also declared in a PCL. All platform-specific data such as images are stored as strings and only converted to bitmaps when rendering views using value converters.

Introducing Simple.OData.Client: a portable OData client library for .NET4.x, Windows Store, Silverlight 5 and Windows Phone 8

Last week I was busy with my home project: creating an OData portable class library. This project originated from the Simple.Data OData adapter when I needed a library for Windows Store application and didn’t find any suitable one. And since Simple.Data only supports .NET 4.x platforms, I started extracting parts of the adapter to make a PCL.

As it always happens, I spent far longer time to complete this task, but it was worth it even the only outcome was code cleanup. I thought the quality of Simple.Data OData adapter code was pretty good, but when I was forced to separate non-portable portions of it, I realized that this was a separation of concerns. When you are writing code that targets multiple platforms, it’s no longer just your subjective decision of how much responsibility you can delegate to a certain class or module. If your occasionally brought stuff that does not meet portability criteria – logging, creating user credentials, external data access – you will have to remove it. Otherwise your code won’t compile.

Of course running the code through portability check won’t expose all violations of clean code principles, moreover – there may be plenty of reasons for code to not be portable. But in many cases it’s a good check, and we should probably ask yourself a question “why this code is not portable?” more often.

But enough for lessons and principles. The portable OData client library is here, and Mark Rendle was kind enough to let me keep the “Simple” prefix in the library name, although this word is kind of his trademark now. (His remark was “as far as I’m concerned it all grows the brand Smile“). And I hope that simplicity of Simple.OData.Client API is on the level of other members of the family of Simple frameworks.

Simple.OData.Client is an open source library, available at GitHub. In addition, it has a NuGet package. If you install it from NuGet, depending on your project target framework you will get a reference to one of two assemblies:

  • Simple.OData.Client.Net40.dll: for target frameworks .NET 4.0, .NET 4.0.3, .NET 4.5;
  • Simple.OData.Client.Core.dll: for portable libraries targeting .NET 4.0.3, .NET 4.5, Windows Store, Silvelight 5, Windows Phone 8 and for target frameworks Windows Store, Silverlight 5 and Windows Phone 8.

Support for Android and iOS (via Xamarin Mono) is in my plans (in fact, AFAIK the portable version may be used to target Mono for Android).

The project has Wiki pages with many examples, it has two API flavors: basic and fluent. Below is an a example of using its fluent API to retrieve data from NuGet OData feed:

var client = new ODataClient("http://packages.nuget.org/v1/FeedService.svc/");
var x = ODataFilter.Expression;
var packages = client
    .For("Packages)
    .Filter(x.Title == "Simple.OData.Client")
    .FindEntries();

foreach (var package in packages)
{
    Console.WriteLine(package["Title"]);
}

Simple.OData.Client supports all HTTP methods used by OData protocol (GET/POST/PUT/MERGE/DELETE) including support for batch requests.