Advanced Evaluation Example: WCF vs XcoAppSpace

One of the reasons why we started researching API usability was that we ourselves wanted to know more about how to create usable APIs, and how to show that one API is easier to use than another one. The Xcoordination Application Space is our attempt of creating an easy to use middleware API. In the given example, we want to use the API Concepts Framework to compare it to WCF.

The task we use for comparison is quite simple: Transferring a single string message from a client to a server. In the following sections we first show the evaluation for WCF and then for XcoAppSpace. Finally, we present a short interpretation of the results.

WCF

To implement the scenario with WCF, first a service contract needs to be defined, which is annotated with ServiceContract and OperationContract. On the server side, this contract needs to be implemented, and the implementation needs to be annotated with the ServiceBehavior attribute. For publishing the service, a ServiceHost needs to be instantiated and configured. On the client side, a ChannelFactory needs to be created to get access to the service. On both sides, the binding that is used for communicating (in this case a NetTcpBinding) must be specified.

While using XML is quite prominent with WCF, we are showing a code-only approach here for better comparability - but this doesn't have any negative influence on usability, since the same configuration stuff would just have been written in XML instead (which is definitely NOT easier to use than code, see our related publication).

Contract:
[ServiceContract]
public interface IService {
	[OperationContract]
	void Ping(string text);
}

Server:
//service
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MyService : IService {
	public void Ping(string text) {
		Console.WriteLine("received: " + text);
	}
}

//main method
using (var svh = new ServiceHost(typeof (MyService), new Uri("net.tcp://localhost:9001")))
{
	svh.AddServiceEndpoint(typeof (IService), new NetTcpBinding(), "service");
	svh.Open();

	Console.ReadLine();
}

Client:
using (var cf = new ChannelFactory<IService>(new NetTcpBinding(), "net.tcp://localhost:9001"))
{
	IService s = cf.CreateChannel();
	s.Ping("hello, world!");
	Console.ReadLine();
}

Evaluation:
The following code shows feeding the stuff into the API concepts framework:
var eval = new ApiEvaluator(typeof(ServiceHost).Assembly);
eval.SetUsings("System", "System.ServiceModel");
eval.Instantiation(() => new ServiceContractAttribute());
eval.Instantiation(() => new OperationContractAttribute());
eval.Instantiation(() => new ServiceBehaviorAttribute());
eval.PropertyAccess((ServiceBehaviorAttribute s) => s.InstanceContextMode);
eval.Instantiation(() => new ServiceHost(typeof(string), new Uri(""))).RequiresInformationLookup();
eval.MethodCall((ServiceHost s) => s.AddServiceEndpoint("", null, ""), returnParameterIsUsed: false);
eval.MethodCall((ServiceHost s) => s.Open());
eval.Instantiation(() => new NetTcpBinding(), 2);
eval.Instantiation(() => new ChannelFactory<string>(null, "")).RequiresInformationLookup();
eval.MethodCall((ChannelFactory<string> c) => c.CreateChannel());

And this is the result of the evaluation:
Cost S Cost U Cost Sum Concept Usages Prev Usages Description
30.0 29.0 59.0 ClassUsage 1 0 ServiceContractAttribute
0.0 4.0 4.0 Instantiation 1 0 [ServiceContractAttribute()]
30.0 29.0 59.0 ClassUsage 1 0 OperationContractAttribute
0.0 4.0 4.0 Instantiation 1 0 [OperationContractAttribute()]
30.0 29.0 59.0 ClassUsage 1 0 ServiceBehaviorAttribute
0.0 4.0 4.0 Instantiation 1 0 [ServiceBehaviorAttribute()]
11.6 7.0 18.6 FieldAccess 1 0 ServiceBehaviorAttribute.InstanceContextMode
30.0 21.0 51.0 ClassUsage 1 0 ServiceHost
1.0 19.5 20.5 Instantiation 1 0 new ServiceHost(Type,Uri[])
InformationLookup: 0 / 8 (lookup of external information (1 lookups))
21.1 20.0 41.1 MethodCall 1 0 ServiceHost.AddServiceEndpoint(String,Binding,String) (return value not used)
13.6 5.0 18.6 MethodCall 1 0 ServiceHost.Open()
30.0 29.4 59.4 ClassUsage 2 0 NetTcpBinding
0.0 5.6 5.6 Instantiation 2 0 new NetTcpBinding()
30.0 21.0 51.0 ClassUsage 1 0 ChannelFactory<>
5.0 19.5 24.5 Instantiation 1 0 new ChannelFactory<>(Binding,String)
InformationLookup: 0 / 8 (lookup of external information (1 lookups))
11.1 13.0 24.1 MethodCall 1 0 ChannelFactory<>.CreateChannel()
0.0 InformationLookup: Some concepts require lookup of external information - this should be prevented where possible.
503.2 Overall


XcoAppSpace

Message transfer is very easy with the AppSpace. On both sides first an instance of the class XcoAppSpace is created. On the server side, the Run method is called to create a port and map it to a function that is called for incoming messages. On the client side, the Connect method is called to connect to the port, and Post is used to post a message which is then sent to the server. Since only a message of a simple type (string) is transferred, no explicitly defined contract is necessary. For more information see the XcoAppSpace homepage.

Contract:
No contract is required for this scenario.

Server:
using (var space = new XcoAppSpace("tcp.port=8000"))
{
	space.Run<string>(m => Console.WriteLine("Msg received: " +  m));
	Console.ReadLine();
}

Client:
using (var space = new XcoAppSpace("tcp.port=0"))
{
	var port = space.Connect<string>("localhost:8000");
	port.Post("Hello, World!");

	Console.ReadLine();
}

Evaluation:
The following code shows feeding the stuff into the API concepts framework:
var eval = new ApiEvaluator(typeof(XcoAppSpace).Assembly, typeof(IPort).Assembly);
eval.SetUsings("System", "XcoAppSpaces.Core", "Microsoft.Ccr.Core");
eval.Instantiation(() => new XcoAppSpace("address"), 2);
eval.ConfigString("tcp.port=8000", 2);
eval.MethodCall((XcoAppSpace s) => s.Run<string>(m => Console.WriteLine("Message received: " + m)), returnParameterIsUsed: false);
eval.MethodCall((XcoAppSpace s) => s.Connect<string>("localhost:8000"), returnParameterIsUsed: true).RequiresInformationLookup();
eval.MethodCall((Port<string> p) => p.Post("test"));

And this is the result of the evaluation:
Cost S Cost U Cost Sum Concept Usages Prev Usages Description
30.0 29.4 59.4 ClassUsage 2 0 XcoAppSpace
2.0 9.1 11.1 Instantiation 2 0 new XcoAppSpace(String)
15.0 7.5 22.5 MethodCall 1 0 XcoAppSpace.Run<>(Handler<>) (return value not used)
14.0 23.5 37.5 MethodCall 1 0 XcoAppSpace.Connect<>(String)
InformationLookup: 0 / 8 (lookup of external information (1 lookups))
10.7 7.5 18.2 MethodCall 1 0 Port<>.Post(String)
0.0 ConfigString
9.0 11.3 20.3 ConfigString 2 Config string: tcp.port=8000 (2 usages)
0.0 InformationLookup: Some concepts require lookup of external information - this should be prevented where possible.
169.0 Overall


Interpretation of Results

The evaluation clearly shows that the XcoAppSpace is much easier to use than WCF. When comparing the lists of concepts, it can be seen that the main reason for WCF's high complexity is the large amount of classes that need to be used. While XcoAppSpace requires only a single class to be explicitly used, WCF requires 6 different classes.
So, when we in theory would like to improve the usability of WCF for this use case, what could be done? When thinking about how some of the classes could be removed, some ideas easily come up:
  • The attributes could be made optional, and for simple use cases defaults could be assumed, so that the attributes are only needed when special behavior is required
  • There could also be a default binding (e.g. NetTcpBinding), removing the necessity to specify it
  • There are two different classes used for hosting and connecting (ServiceHost, ChannelFactory) - it could be changed to using only a single class on both communication sides, like the XcoAppSpace does


We hope that this example could give you an idea of how to use the API Concepts Framework. You are welcome to have a look into our solution code for more usage examples.

Last edited Sep 7, 2014 at 10:10 AM by thomass, version 6