Building a Reusable Builder: An internal DSL in C#

Sitting and enjoying the OpenSpace Coding Day at Conchango, and in particular enjoying Ian Coopers talk on Internal DSLs in C# I came to thinking how it might be quite simple to create a reusable builder object.

I’ve done some posts on this sort of thing before, and have spent quite a bit of time recently at work constructing the beginnings of a Language Workbench (I say beginnings as I’m following YAGNI and there’s only one DSL that uses it so far).

Anyway here’s what I’ve just sketched out at the back of the class.

public class Builder<T>
{
    public BuildingInProgress<T> With(T targetObject)
    {
        return new BuildingInProgress<T>(targetObject);
    }
}

public class BuildingInProgress<T>
{
    private readonly T _target;

    public BuildingInProgress(T targetObject)
    {
        _target = targetObject;
    }

    public BuildingInProgress<T> Set<TValue>(Expression<Func<T, TValue>> property, TValue value)
    {
        PropertyInfo info = ExpressionHelpers.GetPropertyInfo(property);
        info.SetValue(_target, value, null);
        return this;
    }
}

public static class ExpressionHelpers
{
    public static string GetMemberName<TSubject, TMember>(Expression<Func<TSubject, TMember>> subjectMember)
    {
        PropertyInfo propertyInfo = getMemberExpressionMember(subjectMember.Body);
        return propertyInfo.Name;
    }

    public static PropertyInfo GetPropertyInfo<TType,TMember>(Expression<Func<TType,TMember>> property)
    {
        return getMemberExpressionMember(property.Body);
    }

    private static PropertyInfo getMemberExpressionMember(Expression expression)
    {
        if (expression == null) return null;

        switch (expression.NodeType)
        {
            case ExpressionType.MemberAccess:
                {
                    MemberExpression memberExpression = (MemberExpression)expression;
                    return (PropertyInfo)memberExpression.Member;
                }
            default:
                throw new InvalidOperationException(string.Format("The Expression Type was not expected: {0}", expression.NodeType));
        }
    }
}

And that’s basically as far as I have got in ten minutes (whilst paying attention as well of course;). Obviously it has weaknesses, the static reflection would need much building out to handle more than simple MemberExpressions for example.

Here is a really trivial example of its intended use:

[TestFixture]
public class BuilderFixture
{
    [Test]
    public void Where_SetIsCalled_Then_AValueShouldBeSet()
    {
        const string testValue = "testValue";
        const int otherTestValue = 12;
        Foo foo = new Foo();
        Builder<Foo> fooBuilder = new Builder<Foo>();
        fooBuilder.With(foo)
            .Set(f => f.Bar, testValue)
            .Set(f => f.Baz, otherTestValue);
        Assert.AreEqual(testValue,foo.Bar);
        Assert.AreEqual(otherTestValue, foo.Baz);
    }
}

public class Foo
{
    public string Bar { get; set; }
    public int Baz { get; set; }
}

Trying to Bind Form Elements in the View to a Dictionary in the Model with ASP.NET MVC

I've got a class that has, instead of statically typed properties, a dictionary that is used to store different values depending on what is needed for any given instance.

    public class Baz

    {

        public Baz()

        {

            Foo = new Dictionary<string, string> { { "Bar", null } };

        }

        public IDictionary<string, string> Foo { get; set; }

    }

I'm using ASP.NET MVC and I want to bind the object to the View. The View is dynamically generated based on a template object, like this:

<% foreach (Field field in ViewData.Model.Template.Fields)

    {%>

        <%= Html.Encode(field.DisplayText) %>

        <%= Html.TextBox(string.Format("Foo[{0}]", field.Name)) %>

 <% } %>

In the above, the field.Name will correspond to the Key of a KVP placed into the dictionary of the Foo instance during its' manufacture.

I am using the DefaultModelBinder, and if I look at the ASP.NET MVC (Beta) source code and in particular the BindModelCore method I see that the first thing it does is to find out if the type being dealt with is a Dictionary type, if it is then the UpdateDictionary method gets called. The first line of the UpdateDictionary method assigns the result of a call to CreateSubPropertyName passing in the BindingContext.ModelName as the prefix, and the string 'index' as the property name.

Now I can't see how this works. The result of this code seems to be that, where the ModelName is foo, a key of 'foo.index' is looked for in the route data, query string and form values (DefaultValueProvider.GetValue). Now this isn't found unless the id's for my form elements that tie in to the Dictionary are called 'foo.index' (yep, same id for multiple elements - already smells very wrong). That means altering my View code to look like this:

 <% foreach (Field field in ViewData.Model.Template.Fields)

    {%>

        <%= Html.Encode(field.DisplayText) %>

        <%= Html.TextBox("Foo.index")) %>

 <% } %>

And, giving each the name of foo.index, I get back the values of each as the attempted values and progress into the code which iterates through these values (now stored in a string[] called 'indicies'. The first line of code in the foreach loop (still in DefaultModelBinder.UpdateDictionary) calls CreateSubIndexName, passing in the ModelName and one of the attempted values, so if the attempted value was 'test', it will return foo[test]. Now this key is formatted how I'd want, but uses the attempted value as the key, which is not what I'd want.

The very next thing that happens is a call to DefaultModelBinder.BindProperty which calls CreateSubPropertyName passing in the ParentContext.ModelName, which right now is 'foo[test]' and the string 'key'. This results in a 'newName' of 'foo[test].key'. The consequence of this is that DefaultValueProvider.GetValue is called and looks for an id called 'foo[bar].key', unsurprisingly this doesn't exist, as I had to give all of the dictionary bound form elements an id of 'foo.index', when the null value is returned up then an exception results and the attempt to bind the elements stops.

What I'd like is for the form element id to read Foo[Bar] (assuming no prefix required, otherwise Baz.Foo[Bar]), and for the binding process to use this to get the attempted value, and to then Try and update the KeyValuePair in the Dictionary Foo where the key is Bar with the attempted value. Is that too much to ask?

Now it's entirely possible (probable?) that I've not understood how to make the UpdateDictionary method tick, Googling has revealed nothing, so I'd be appreciative of any information anyone reading this might be able to impart. Otherwise, if I get a chance, I may look to implement my own ModelBinder that has this behaviour (but i'd rather not).

Refactored: Fluent Test Data Builders Using Generic Extension Methods and Lambda Expressions

Working further with the Fluent Test Data Builder I was working on yesterday has led me to refactor what I had, and the result is, I think, even nicer!

I've added a new method SetValue, and changed the from getting the Member.Name to getting the PropertyInfo from the Member. The other significant change is that I'm now wrapping the SetLength and SetValue methods in try catch blocks so that I can intercept a TargetInvocationException and throw the InnerException in its place, this helps to ensure that when I use the ExpectedExceptionAttribute I am catching the exception I really expect. The drawback with this is the stack trace, but I'm content to live with that for now.

publicstatic T SetLength<T>(this T obj, Expression<Func<T, string>> expression, int length)

{

try

    {

        PropertyInfo member = getMemberExpressionMember(expression.Body);

        string currentValue = expression.Compile().Invoke(obj) ?? string.Empty;

        string newValue = currentValue.PadRight(length);

        member.SetValue(obj, newValue, null);

        return obj;

    }

    catch (TargetInvocationException tie)

    {

        if (tie.InnerException != null) throw tie.InnerException;

        throw;

    }

}

public static TObj SetValue<TObj, TVal>(this TObj obj, Expression<Func<TObj, TVal>> expression, TVal value)

{

    try

    {

        PropertyInfo member = getMemberExpressionMember(expression.Body);

        member.SetValue(obj, value, null);

        return obj;

    }

    catch (TargetInvocationException tie)

    {

        if (tie.InnerException != null) throw tie.InnerException;

        throw;

    }

}

So how am I making use of this fluent interface? Well the methods above are two of those I have written, and probably the most interesting, but I have others too. These fall into two main categories:

1. setting default valid values;


2. creating default valid objects.

For setting default valid values, these are done as ExtensionMethods on the type I'm interested in. For example:

        public static Extract WithDefaultTitle(this Extract extract)

        {

            extract.QualificationTitle = "Mr";

            extract;

        }

For creating default valid objects, this is also done with an ExtensionMethod on the type that I'm interested in. As my need for different default starting points increases then I can easily write more. Here's an example:

        public static Extract AsValid(this Extract extract)

        {

            return extract

                .WithDefaultCode()

                .WithDefaultTitle()

                .SetValue(e => e.Id, 111)

        }

With these methods available to me I can combine them with my SetValue and SetLength methods to then alter the state of any given property so that I can test the implications of the value set in a degree of isolation (at least consistency).  In particular I am using this to test that behaviour to validate my object works correctly given different combinations of property values on the object under test. Here's an example of this:

        [Test]

        [ExpectedException(typeof(InvalidOperationException), "An Id must be present.")]

        public void NullIdShouldThrowAnInvalidOperationExceptionOnValidate()

        {

            Extract extract = new Extract()

                .AsValid()

                .SetValue(e => e.Id, null);

            extract.Validate();

        }

        [Test]

        public void A_SurnameWithLessThan35CharactersShouldBePaddedTo35CharactersInFormattedProperties()

        {

            Extract extract = new Extract()

                .AsValid()

                .SetLength(e => e.Surname, 34);

            Assert.AreEqual(34, extract.Surname.Length);

            Assert.AreEqual(35, extract.FormattedProperties["Surname"].Length);

        }

Personally I'm happy with how this is progressing, but we'll see how it goes as I begin to push the current envelope a bit more.

Fluent Test Data Builders Using Generic Extension Methods and Lambda Expressions

Whilst writing some extension methods this evening to make creating objects in unit tests simpler I found myself writing methods that looked sufficiently similar to each other that I felt the code lacked a little DRYness.

The code being written has to output a data extract in a fixed field length formatted text file. As such the length of the formatted fields is very important, so I had created some methods to initialise the properties of my object with given lengths so that I could ensure the desired behaviour during formatting (such as trimming, padding, exception throwing, etc...). These methods looked much like this:

public static DataExtract WithSetLengthSurname(this DataExtract dataExtract, int length) 

{ 

    dataExtract.Surname = string.Empty.PadRight(length); 

    return dataExtract; 

}

public static DataExtract WithSetLengthForenames(this DataExtract dataExtract, int length) 

{ 

    dataExtract.Forenames = string.Empty.PadRight(length); 

    return dataExtract; 

} 

As you can see these methods are very similar. What I wanted I felt was to take this fluent interface for creating my DataExtract object, but to make the whole thing more generic.

This is what I came up with:

public static T SetLength<T>(this T obj, Expression<Func<T,string>> expression, int length) 

{ 

    string memberName = getMemberExpressionMemberName(expression.Body); 

    string currentValue = expression.Compile().Invoke(obj) ?? string.Empty; 

    string newValue = currentValue.PadRight(length); 

    obj.GetType().GetProperty(memberName).SetValue(obj, newValue, null); 

    return obj; 

} 

private static string getMemberExpressionMemberName (Expression expression) 

{ 

    if (expression == null) return string.Empty; 

    switch (expression.NodeType) 

    { 

        case ExpressionType.MemberAccess: 

            { 

                MemberExpression memberExpression = (MemberExpression) expression; 

                return memberExpression.Member.Name; 

            } 

        default: 

            throw new InvalidOperationException(string.Format("The Expression Type was not expected: {0}", expression.NodeType)); 

    } 

}

With this code then I can now make a call like this:

DataExtract extract = new DataExtract().SetLength((e) => e.Forenames, 10);

Now it's not unreasonable to look at this and wonder what has been gained, after all I could have written this:

DataExtract extract = new DataExtract { Forenames = "".PadRight(10) };

I suppose for me it comes into its own when combined with other extension methods as a part of a wider ranging Fluent Interface based test object builder, like this:

CandidateExtract extract = new CandidateExtract() 

    .SetLength((e) => e.Surname, 36) 

    .WithValidQualificationCode() 

    .WithValidQualificationTitle() 

    .SetLength((e) => e.Forenames, 36);

As can be seen in the above some property specific methods still exist, because the behaviour that each implements is sufficiently unique, but where the behaviour is very similar (such as setting a string to a default length) this seems to me to provide a number of benefits:

• the number of methods appearing in Intellisense drops making it easier to find what you want quickly
  
• the number of builder methods to be written drops (in my case significantly)
  
• by using a fluent interface readability is enhanced.

WCF With The Castle Windsor Facility

Towards the end of a rather busy Saturday of coding in the office I decided to take on the exposing of some services at the boundary of a system I am working on. We're using the Castle project's Windsor container for our IOC on all of our new projects and so I figured that it would make sense to do a short spike into the WCF facility that ships with it to see whether this would be worth using going forwards. The short answer is that I think it is.

It proved very simple to get going. I'd recommend anyone looking to use this facility gets the latest version of the source code in the trunk before starting and has a look at the demo project in there as this proved very helpful.

I quickly defined an interface and set it up as a contract (as per normal with WCF), I then created a class that implemented the interface. At this point in the Global.asax.cs I configured my Windsor container mappings like this:

ServiceDebugBehavior returnFaultsAndMex =

                new ServiceDebugBehavior

                    {

                        IncludeExceptionDetailInFaults = true,

                        HttpHelpPageEnabled = true

                    };

            ServiceMetadataBehavior metadata =

                new ServiceMetadataBehavior {HttpGetEnabled = true};

            container = new WindsorContainer()

                .AddFacility<WcfFacility>()

                .Register(

                          Component.For<INHibernateSessionManager>()

                              .Named("NHibernateSessionManager")

                              .ImplementedBy<NHibernateSessionManager>(),

                          Component.For<IBarRepositoryFactory>()

                              .Named("BarRepositoryFactory")

                              .ImplementedBy<BarRepositoryFactory>()

                              .DependsOn(Property.ForKey("sessionFactoryConfigPath")

                                             .Eq(Path.Combine(

                                                     Path.GetDirectoryName(

                                                         GetType().Assembly.Location),

                                                     "Hibernate.cfg.xml"))),

                          Component.For<BarService>()

                              .Named("BarDomainService")

                              .ImplementedBy<BarService>(),

                          Component.For<IBarEnterpriseService>()

                              .Named("BarEnterpriseService")

                              .ImplementedBy<BarEnterpriseService>());

Next up was the .svc file:

<% @ServiceHost Service="BarEnterpriseService" 

    Factory="Castle.Facilities.WcfIntegration.DefaultServiceHostFactory, Castle.Facilities.WcfIntegration" %>

With a bit of work in the web.config file then, with a press of F5, I can navigate to the svc and get the MEX page:

  <system.serviceModel>

    <services>

      <service name="Foo.Bar.EnterpriseServices.BarEnterpriseService" behaviorConfiguration="ReturnFaultsAndMex">

        <endpoint contract="Foo.Bar.EnterpriseServiceContracts.IBarEnterpriseService"

                          binding="basicHttpBinding"/>

      </service>

    </services>

    <behaviors>

      <serviceBehaviors>

        <behavior name="ReturnFaultsAndMex" >

          <serviceDebug includeExceptionDetailInFaults="true" />

          <serviceMetadata httpGetEnabled="true" />

        </behavior>

      </serviceBehaviors>

    </behaviors>

  </system.serviceModel>

This is great and very nicely integrates the whole WCF experience into my IOC centric application. I do still have a couple of areas where I have questions. In the global.asax file I included details of two behviours, for error details and to enable the MEX service. This code was lifted from the sample in the Castle trunk. I still needed to add these behaviours explicitly into the web.config though. Present or removed these registrations seem to have no effect, and I find the same to be the case with the demo app.

Continuous Integration with CI Factory part 2

I've just finished setting up a new build instance with CI Factory. This time I used the latest version (1.0.1.5 at the time of writing). I followed the steps here, together with some from my last CIFactory post.

I tried this time to add a property in Arguments.xml of this:

< property name="MSBuild.Framework.Version" value="${framework::get-framework-directory('net-3.5')}" />

In addition then I edited Compile.Target.xml to use this property like so:

program="${MSBuild.Framework.Version}\msbuild.exe" workingdir="${ProductDirectory}"

After running the run.bat file the only thing that failed was that I got the same old error about needing to set the path to the share for VSS so I went in to the ccnetproject.xml file and edited the ssdir setting to provide the appropriate path to my network share that holds my vss ini file (as described in my last CIFactory post).

I ran the CCNETServer.bat and that problem went away, but then I saw that the first build failed. On checking the log what I saw was that the MSBuild.Framework.Version property was not being set. To resolve this I went to the Compile.Properties.xml file for the MSBuild package and added the property in there, not ideal, but it did the job.

I now have a working build for the project on this server, alongside the build from the previous post (which is still to be upgraded). Next up the upgrade of the old one!

Defending Scrum

Recently Scrum has been taking a bit of a battering in the wider community, being made to take responsibility for some of the perceived failings of Agile. Under particular attack has been the scrum based certifications which are seen to churn out 'experts' on the basis of short courses. But who then in many cases lack the necessary expertise, of which experience is an important constituent, to actually make Scrum work. This is a view that I have much sympathy with, but I guess I distinguish between Scrum the methodology and Scrum the business and so am not required to hold the methodology to account for its', damn near literal, selling out by the associated business(es). That said, it was refreshing today then to read a defence of agile, and scrum in particular, on Robert C. Martin's 'Uncle Bob' blog.

http://blog.objectmentor.com/articles/2008/11/16/dirty-rotten-scrumdrels

I find myself in agreement with Uncle Bob's sentiment and arguments. The thing that I find a little surprising is the implication that teams can adopt XP or Scrum, maybe I have misread. Personally I have always thought of Scrum as a kind of management wrapper that works very well when surrounding XP practices. Another interesting post I read today was by Jeremy Miller.

http://codebetter.com/blogs/jeremy.miller/archive/2008/11/16/thoughts-on-the-decline-and-fall-of-agile.aspx

I agree with most of what Jeremy has to say, and for me it comes back to the twelve agile principles (http://agilemanifesto.org/principles.html). If we as a development team lack the discipline to ensure that each of these twelve principles are adhered to then we invite issues of the sort that Scrum's accusers make reference. Scrum provides a framework that I find makes the introduction of Agile into an organisation easier. It is not the endpoint to my mind, as always there is a maturation process and with the right people, and good use of retrospectives, the teams practices may evolve out of the strictures of Scrum and into something different. What it does do though is provide some initial norms that the team can work to. In a project I am managing at work we have brought together a number of contractors, all highly skilled and some with excellent agile experience, to develop some software where the need to 'provide early and continuous delivery of valuable software' could not be greater. Scrum, I find, accelerates the process through Tuckman's four stages of team development, enabling the team to reach the stages of norming and performing far more quickly that they might were they to have to deal with project management issues during the forming and storming phases. Quite frankly, given the timescales, it is quite enough that they come to a sense of group ownership of the architecture, design and requirements, I am more than happy to begin with a Scrum based imposition of how the team will work, and allow them through retrospectives (starting after at least a couple of sprints) to begin with debating, and incorporating, improvements. This to me parallels closely the idea from the 5-S model that Shitsuke only begins once the initial 4 S's have been established.

This parallel with the 5S's brings me to Lean. Something that I've not been able to entirely figure out yet is the way that Scrum seems to be set off against Lean so much at the moment. If I look at the 5-S model for example I struggle to see how these would not be carried into a successful Scrum implementation. Perhaps it is because a lot of the Scrum material typically focuses on the project management, but the requirement for every feature developed to be of quality (even if deciding on quality can be left to the team) is still there. If quality becomes a relative concept where it can be 'good enough' then this is up to the team. If the team is committed to agile then the hope would be that there definition of quality includes the principle that 'Continuous attention to technical excellence and good design enhances agility.' Kanban were introduced to me whilst I was working at Immediacy and have been an essential in my Scrum toolkit since, Scrum in its essence seems to me to be as much about promoting visibility as anything else and Kanban are an excellent way of achieving this. Like I would imagine any agile system to be, Scrum is centred around a Pull system vision where the software developed is closely driven by its end users. The system is also a self-Pull system where the team pulls in detail and direction as required, pulls in the removal of impediments or resources where required.

In all of these respects and many more Scrum provides an COTS for Agile teams, where the customisations may lead to the team no longer doing 'Scrum' in the strictest sense, it may be a more open 'Lean' style where sprints are not timeboxed for example. At least some of the roots of Scrum (at the least its' name) are in the work of Nonaka and Takeuchi, as well as in the Agile software movement, and their text 'The Knowledge Creating Company' is one that I keep close as a constant source of interest. This text with its' examinations of companies such as Honda, Canon and Matsushita provides valuable insights into the practices of management in Japanese companies which typically come under the banner of 'Lean'. As a point in case, let us again return to the 5-S model which was first developed by Canon (Japanese Management Association, 1987) who were studied by Nonaka and Takeuchi, who in turn provided, at the least, inspiration to Ken Schwaber. A tenuous link perhaps, but one that perhaps makes it less surprising that this model fits so well with Scrum. The common ancestry, commonality of practice, shared principles, and current open warfare between Lean and Scrum always put me in mind that often the intra-denominational arguments can be at least as fierce as extra-denominational ones.

As a final thought, when in his post James Shore states that 'when people say "Agile," they usually mean Scrum' I am left wondering if this is true. A great many people (particularly in project management and related capacities) who I deal with know little of Scrum, and many have not heard of it. They are as likely to think of Agile Unified Processes or DSDM, and in many cases are so imbued with Waterfallesque thinking that they apply Scrum as a label to their clearly non-Agile UP practices. Agile is not something that they currently grok. Whilst I think it is true that Scrum has gained a very real momentum, in no small part due to the business that promotes it and certifies some of its' practitioners, I do not think that this is in any way a bad thing despite this. As Scrum implementations fail some will no doubt use it as an excuse to write it, and possible Agile with it, off. To my mind such people are unlikely to be successful whatever they do, harsh perhaps, but not reflexively analysing and learning from failures is a sure way to stunt professional growth and progression (and probably a reason for a failure to make Scrum work). If, however, good professionals (and this is the vast majority of those whom I have been privileged to work with) reflexively analyse the reasons for a perceived failure of Scrum then in many cases I suspect they will see the problem as not being with the methodology, but rather with its' interpretation and implementation in their context. With the adoption of methods not compatible with it, and with the failure to adopt methods required by it, in short the failure to work in the Scrum way.