Retuning Custom Collections (Complex Typed Objects) Over .Net Web Services.

My personal preference is to use custom typed collections (read portable types) as oppose to datasets, especially when trying to achieve an SOA. However this is easier said than done when working with .Net web services…or should I say .Net web references. Let me explain…

I have a class that looks like this…

///

/// Holds a collection of 's.
///

[Serializable()]
[XmlRoot("LayerOptionChoiceCollection")]
public class LayerOptionChoiceCollection : BaseEntityCollection
{
public virtual void Add(LayerOptionChoiceEntity item)
{
List.Add(item);
}

public virtual void AddRange(LayerOptionChoiceEntity[] items)
{
foreach (LayerOptionChoiceEntity item in items)
{
List.Add(item);
}
}

public virtual LayerOptionChoiceEntity this[int index]
{
get
{
return (LayerOptionChoiceEntity)List[index];
}
set
{
List[index] = value;
}
}
}

… BaseEntityCollection inherits from CollectionBase giving it an ArrayList like functionality with its ‘List’ member, as well as some simple data binding capabilities.

cmbQuoteView.DataSource = this.LayerOptionChoiceCollection
cmbQuoteView.ValueMember = "DuckCreekQuoteID"
cmbQuoteView.DisplayMember = "DisplayText"

LayerOptionChoiceEntity is simply a class with private intrinsic members and public get/set properties to expose them.

I simply wanted to pass this typed collection over my web service to be able to bind it to a drop down in the UI. I created the following web method.

_
Public Function GetAllLayerOptions(ByVal programID As Integer) As LayerOptionChoiceCollection
Dim svc As New LayerOptionService()

Return svc.GetLayerOptionChioces(programID)
End Function

I added a web reference from my service façade, and I figured I would be on my way. I compile the app and I get the first error in my service façade class.

CS: Cannot implicitly convert type 'Endurance.WorkBench.WebService.Test.DuckcreekWS.LayerOptionChoiceEntity[]' to 'Endurance.Duckcreek.Data.LayerOptionChoiceCollection'

VB: Value of type '1-dimensional array of Endurance.WorkBench.ServiceFacade.DuckcreekWS.LayerOptionChoiceEntity' cannot be converted to 'Endurance.Duckcreek.Data.LayerOptionChoiceCollection'.

My first question, what is a Endurance.WorkBench.WebService.Test.DuckcreekWS.LayerOptionChoiceEntity[]?

Looking at the reference.cs file generated by wsdl.exe I see the following signature has been created for my method.

public LayerOptionChoiceEntity[] GetAllLayerOptions(int programID) {
object[] results = this.Invoke("GetAllLayerOptions", new object[] {
programID});
return ((LayerOptionChoiceEntity[])(results[0]));
}

Very interesting, so it wants to return an array of my entities instead of my collection type. Apparently this is by design. See the post from 2006-02-14, 3:59 am http://www.codecomments.com/archive321-2006-2-805600.html.

“…the client webservice proxy generate the types
according to the webservice's WSDL document. And WSDL document use XML
schema's standard types to represent different object types used in
webservice methods, this is for interop consideration since the webservice
will be consumed by different client platform (c++, java, php.....). And
for list/collection like type, WSDL will always use SOAP Array to represent
it, thus the .NET client proxy will generate the Array for such
objects(parameter or return type). If you do need to use .NET specific
type, you need to manually change the client genernated proxy code…”

The only exception to the rule, is, ironically, the DataSet (from the 2006-02-15, 3:58 am post).

“…So the further question you mentioned all aims at make the autogenerated
client proxy use our custom class type(.NET specific). I'm afraid this is
not available through the buildin webservice client proxy genenration
tool(wsdl.exe). The DataSet class you mentioned is a particular class(which
is somewhat hardcoded in the wsdl.exe tool's generation logic since there
is not particular attributes in description for DataSet as we can see).
I've tried DataTable class also which can not be recognized.
I think for your scenario, it is more likely that you're going to build a
custom webservice client proxy generation tool rather than extend the
current one. Anyway, currently the simplest means is to modify the
autogenerated proxy class code…”

Ok so, for my purposes I make the decision that I will change the generated web reference.

I dive into the reference.cs file and change all instances of the LayerOptionChoiceEntity[] to return my collection of LayerOptionChoiceCollection, and add the necessary namespace.

Build succeeded. Now the return type of my web service and web reference match. I run the application, an exception occurs. I check my event viewer and I see the following list of nested exceptions:

Type : System.InvalidOperationException, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Message : Method DuckCreekService.GetAllLayerOptions can not be reflected.

Type : System.InvalidOperationException, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Message : There was an error reflecting 'GetAllLayerOptionsResult'.

Type : System.InvalidOperationException, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Message : There was an error reflecting type 'Endurance.Duckcreek.Data.LayerOptionChoiceEntity'.

Type : System.InvalidOperationException, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Message : Types 'Endurance.Duckcreek.Data.LayerOptionChoiceEntity' and 'Endurance.WorkBench.ServiceFacade.DuckcreekWS.LayerOptionChoiceEntity' both use the XML type name, 'LayerOptionChoiceEntity', from namespace 'http://bm.endurance/Endurance.WorkBench.WebService/DuckCreekService'. Use XML attributes to specify a unique XML name and/or namespace for the type.

“My first question, what is a Endurance.WorkBench.WebService.Test.DuckcreekWS.LayerOptionChoiceEntity[]?”

Well, I have an answer. You have to remember that part of idea, if not most, behind a web service is that it can be consumed by another platform in-specific application. This works in reverse as well, when we create a web reference and wsdl.exe generates a proxy class, it assumes no knowledge of the ‘type’ (read assembly and class) that it is consuming. So it generates a class definition based on the xml it is consuming and it places this definition in the References.cs (or vb) file. And that is what a Endurance.WorkBench.WebService.Test.DuckcreekWS.LayerOptionChoiceEntity[] is. My assembly and class definition was Endurance.WorkBench.Duckcreek.Data.LayerOptionChoiceEntity, wsdl.exe generated a new definition Endurance.WorkBench.WebService.Test.DuckcreekWS.LayerOptionChoiceEntity, using the name of the web reference.

The solution: delete this type from the reference.cs (or vb) class and no more namespace collision.
Point of interest: wsdl.exe appears to generate its types using xsd.exe. http://softwareink.blogspot.com/2006/05/xsdexe.html

Article of interest http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnservice/html/service07162002.asp

Unit Testing with VSTS (Visual Studio Team System) and Ent Lib (Enterprise Library)

As a huge fan of Nunit I was quite excited to give VSTS unit testing a try. I figured I would start out with a simple test to verify that one my data access methods was returning information correctly. To do this you can simply right click on your method, or class, and go to ‘Create Unit Tests…’.

You will then be prompted with a dialog from which you can specify which methods to stub out tests for and in which project to create them.

I chose to create a new project - a method was stubbed out for me.

///

///A test for GetAllCarriers ()
///

[TestMethod()]
public void GetAllCarriersTest()
{
DataRetrievalDAL target = new DataRetrievalDAL();

CarrierCollection expected = null;
CarrierCollection actual;

actual = target.GetAllCarriers();

Assert.AreEqual(expected, actual, "Endurance.Duckcreek.Persistence.DataRetrievalDAL.GetAllCarriers did not return th" +"e expected value.");

Assert.Inconclusive("Verify the correctness of this test method.");
}

For simplicity sake, all I really care about is the fact that stored procedure behind this method executes and that it does not return null, so I changed the assertions.

///

///A test for GetAllCarriers ()
///

[TestMethod()]
public void GetAllCarriersDALTest()
{
DataRetrievalDAL target = new DataRetrievalDAL();
CarrierCollection actual;

actual = target.GetAllCarriers();

Assert.IsNotNull(actual);
}

I compile the application and it succeeds. I open the Test View window (from the menu bar Tests Windows Test View) and I see my recently created test.


/


/



Right clicking the test I go to Run Selection to execute my test.

Under the test results window I can see my test failed. I right click the test and go to View Test Results Detail.

Error Message.








"Test method Endurance.Duckcreek.Persistence.Test.DataRetrievalDALTest.GetAllCarriersDALTest threw exception: System.TypeInitializationException: The type initializer for 'Endurance.Duckcreek.Persistence.BaseDAL' threw an exception. ---> System.Configuration.ConfigurationException: Invalid section name. The section 'dataConfiguration' does not exist in the requested configuration file 'C:\Endurance\Projects.Dev\Endurance.WorkBench\TestResults\glennj_XP-GLENNJ1 2006-06-25 22_25_08\Out\vstesthost.exe.config' or the file 'C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Config\machine.config'. Make sure that the 'enterpriseLibrary.configurationSections' configuration section exists in one of the files and that the section 'dataConfiguration' is defined.."

Out data access layer uses Microsoft Enterprise Library which runs off of configuration files. This specific error is saying that it can not find the configuration file for the data access component. From past experiences with NUnit I know that in order to allow a .dll to use a .config file, you need to name the .config file the fully qualified name of the .dll.config

Company.Namespace.Class.dll
Company.Namespace.Class.dll.config

So I rename the web.config file to the name of my dll.config, Endurance.DuckCreek.Persistence.Test.dll, and I place it in the bin directory of unit tests project. Ideally it would be placed on the root and copied to the bin directory as a post build event.

I recompile the application and I run my test again. New error message

Test method Endurance.Duckcreek.Persistence.Test.DataRetrievalDALTest.GetAllCarriersDALTest threw exception: System.TypeInitializationException: The type initializer for 'Endurance.Duckcreek.Persistence.BaseDAL' threw an exception. ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging, Version=1.1.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

It can’t find the ent lib dll’s. So I copy the ent lib dll’s to the bin directory of my unit tests project and I try again. No luck. It still can’t find the ent lib dll’s. I double check and I verify that they are in fact in the bin directory, specifically Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging. So I decide to fire up FileMon and see what’s going on. Using FileMon I capture all of the events during a test run and I search for the name of that .dll. Found it.

After a unit test run, the output from the test is put within a time stamped directory within the TestResults folder in under your solution. TestResults\glennj_XP-GLENNJ1 2006-06-25 23_05_47\Out. What needs to happen is that the EntLib dll’s need to make it to this directory after each test run. I gave some thought to attempting to build some sort of a post build event, but I figured it would be easiest to just add project references to the EntLib dll’s and hope for the best.

I recompile the solution run the test and receive another error.

Test method Endurance.Duckcreek.Persistence.Test.DataRetrievalDALTest.GetAllCarriersDALTest threw exception: System.TypeInitializationException: The type initializer for 'Endurance.Duckcreek.Persistence.BaseDAL' threw an exception. ---> System.Configuration.ConfigurationException: Can not find the configuration file C:\Endurance\Projects.Dev\Endurance.WorkBench\TestResults\glennj_XP-GLENNJ1 2006-06-25 23_11_48\Out\dataConfiguration.config. You must check the 'path' attribute in the section 'dataConfiguration'.

Its locating the main .config file and the dll’s but EntLib can not find the other .config files it needs. Given that this is unit test project, for time purposes, I choose to point my test projects config file to an existing dataConfiguration.config on my machine.

<configurationSection xsi:type="ReadOnlyConfigurationSectionData" name="dataConfiguration" encrypt="false">
<storageProvider xsi:type="XmlFileStorageProviderData" name="XML File Storage Provider "path="C:\Endurance\Projects.Dev\Endurance.WorkBench\ Endurance.WorkBench.Web ervice\dataConfiguration.config" />
<dataTransformer xsi:type="XmlSerializerTransformerData" name="Xml Serializer Transformer">
<includeTypes />
</dataTransformer>
</configurationSection>

I run the test and it passes.

Working with EntLib and VSTS unit testing has it difficulties, but it’s worth the trouble in my opinion. More to come on Code Coverage and Refection based testing for private members.

ASP.Net 2.0 References

Due to the deprecation of a project file for the ASP.Net 2.0 web site model [1], references are handled differently. One way is through a file reference. A file reference is maintained by placing a file with a .refresh extension in the Bin directory of the web site. Within this file you place a path pointing to the location of the file you want to maintain a reference to.

File: Software.Ink.dll.refresh
Content: '..\Software.Ink\bin\Software.Ink.dll'

During a build this file will be analyzed and the appropriate reference updated accordingly.
The second option is through a project reference. A project reference is added through the property pages of the web site and a change will be made to .sln file to indicate that the project reference exists against the web site. So the solution file now handles what the project file use to…..

FYI - Project references and file references can not always co-exist [2].

[1]Wap
[2]Project References

Patterns and Anti Patterns for Service Oriented Architectures

So far Tech Ed has most definitely exceeded my expectations. I’ve been spending some of my time in the break out sessions around architecture and agile development. Ron Jacobs’ (Architect Evangelist – Microsoft) session, Patterns and Anti Patterns for Service Oriented Architectures, proved to be quite insightful and relevant to the challenges at my current client.

Ron Said - Understanding the goal of service orientation is key. The goal is fiction free interaction between systems.
Glenn Says – Today we build a lot of our custom .net applications with an n-tiered distributed architecture. Let’s say we facilitate the distribution through web services. Does this mean we get an SOA for free? I mean, if we expose some or all of our services another system could discover how to and then interact with them. Then we have two (loosely coupled) systems consuming a common set of services. SOA? I think yes and no. Yes from an academic perspective, no because if the services were not designed correctly, they may not bring their intended value. How difficult was it for someone to consume those services? Did it require any changes on your end? Will future maintenance on either end require changes from both? Those are some of the questions that I will be asking prior to the development of my next web service. Its easy to claim an SOA, but not so easy to have it provide value.

Ron Said - Service Orientation is a Style of Architecture Which Emphasizes Standards Based Integration

Glenn Says – SOAP.

Ron Said - Services are Autonomous

Glenn Says – They need to be free to think and act as they like. Drive the consumer’s interaction with your services; don’t let them dictate things to you. At my current client I’m tasked with integrating our existing custom .net app with a new black box system. The communication will be Xml over Http and there will be some complex business processes that will be executed. I plan on taking the autonomy factor into account when developing the different services. When the black box system communicates back to our application, I’m going to try to not allow the incoming data itself to drive the processes. Instead I will expose different independent services that the black system can interact with at will. Using this approach should hopefully minimize future changes to the functioning services. The option of ‘I’ll put ‘abc’ in this field and you can do ‘xyz’ won’t be available, instead I’d rather see a new service.

Ron Said - Services Share Contract and Policy Only

Glenn Says – While ACA offers some wonderful configurable ways to change your transport mechanism (In Proc, WS, Remoting), this is not a requirement to have a successful SOA. If your policy is Xml over Http that’s fine. If you’re only on .Net and your contract is .net specific thats fine as well. Your policy simply needs to suite your business needs. Other systems will have a policy which they will need to follow to consume your services.

Ron said - Anti-Pattern: Loosey Goosey, How do I factor my service so that it is flexible and highly adaptable?

Glenn Says – We don’t want ‘god classes’ and we don’t want ‘god services’. Web methods are cheap go ahead and use them.

Ron Said - Tip: Use Portable Types

Glenn Says – Not going there (j/k). Is a DataSet a portable type? I don’t know. What I do know is that I’ve seen the very SOAP content from a DataSet going over a web service…and I’m not sure how simple that would be for a non .net system to interact with. At my current client I’m taking a sample Xml document that I would like to send over the wire. I’m inferring a schema and generating a typed object off of that. I can now leverage the xml serializer and quite rapidly have a way to receive an Xml Post of data, deserialize it into an object, manipulate the data through a simple DOM like fashion, and be able to seriliaze the changes back to xml.

I’ll put the Power Point on the blog site as well.

I’d love to hear everyone thoughts!

- Glenn

Bermuda

I was staffed in Bermuda last week =). My client needed some assistance pushing out a few features before there busy season. Not a bad deal. I’ll post some pics after I download them off my RAZR.

Xsd Inference

As it turns out the Xsd inference feature of Xsd.exe is to be desired. It was making rather poor guesses at how to type my elements; it took the safe path and made them all strings actually. It was rearranging the order of my elements. The ordering of the elements upon deserialization is apparently driven by the very hierarchical fashion in which the public properties are structured within the class(es). I’m not sure how a dataset acts. The final straw was that it was actually generating an object based on the schema it inferred from the xml that was not even serializable! It was throwing an InvalidOperation exception when I tried to instantiate the XmlSerializer with its type.

{"Unable to generate a temporary class (result=1). error CS0030: Cannot convert type 'Avanade.East.serverRequestsOnlineDatasetAgencyDetailsRqAgencyDetails[]' to 'Avanade.East.serverRequestsOnlineDatasetAgencyDetailsRqAgencyDetails' error CS0029: Cannot implicitly convert type 'Avanade.East.serverRequestsOnlineDatasetAgencyDetailsRqAgencyDetails' to 'Avanade.East.serverRequestsOnlineDatasetAgencyDetailsRqAgencyDetails[]'"}

Xsd Inference came to the rescue. After generating the typed object using the schema inferred by Xsd Inference, all went well. I would highly recommend Xsd Inference over the inference feature of Xsd.exe.

Xsd.exe

On my project I’m pretty much running the show in terms of integrating two applications. Full SDLC. Our app, is a .Net Winform with a distributed architecture facilitated via web services. The application I need to integrate with…well I don’t know much about and I don’t need to, well call it Application B. It can send and consume Xml and that’s all I need to know.
Xml. Naturally I’m going to SOA-ize the web services to allow Application B to tap into our system. SOA (service Oriented Architecture). I’m using this term lightly here. I think it qualifies because at the very basic level we have two loosely coupled systems consuming a common set of services. If I could have designed this thing from the beginning I would have made the architecture a little more elegant (ACA.net?). The ability to change the transport and a more generic interface (ie. xml layout.) etc. I probably also would have gotten rid of the binary serialization (don’t ask) =)
So – back to the integration. Given the fact that I am going to have to be able to consume and produce various xml document structures, as well as manipulate their various data values within C# I decided one of the first things I needed was a rapid way to accomplish this generation and consumption of xml again and again. And I turn to Xsd.exe for .Net 2.0.
This is wonderful little tool that comes with the .Net Framework (1.1 & 2.0) that allows you to generate schemas and typed objects from xml documents. Typed objects as in custom classes/collections (Yeah!) and Datasets (Boo!). Xml => Schema => Typed Object. Basically I’m slapping a C# win form front end on this bad boy and allowing it to build the correct arguments for the command line. I can then easily create objects based on an XML doc, and serialize and deserialize them into typed classes. My primary goals are to (1. In one click, have the ability to generate the schema from an xml doc and output the custom classes. (2. Have the ability to output the schema to the input window, modify, and regenerate the classes from the modified schema. (3. Be able to specify a namespace. (4. Be bale to specify an output location. (5. be able to specify the location of Xsd.exe itself. Long term goals are to have C# (Yeah!) and VB.Net (Boo!) support, as well as custom class and dataset generation options. NUnit…or actually VS2005 Unit Testing, fully NDoc commented, and submitted as an Avanade Intellectual Asset. Toolbar instead of buttons and a status bar. As well as the ability to use the XSD Inference tool which seems to produce a more accurate schema. Eventaully…hopefully. Still need to pass my last MCSD.Net test, create a presentation on SOAP and read up on .Net 2.0. When it’s done I’ll post the executable and source code somewhere and post a link here.

A Little Bit About Me

For money, I write software for a consulting firm called Avanade and I am based out of our Manhattan office (13th & 8th). I’ve been there a little over two years and I’ve learned a lot since joining. I attented the Chubb Institute and SetFocus to learn programming and landed my first software job at age 20 in NYC.