Defending Scrum

Recently Scrum has been taking a bit of a battering in the wider community, being made to take responsibility for some of the perceived failings of Agile. Under particular attack has been the scrum based certifications which are seen to churn out 'experts' on the basis of short courses. But who then in many cases lack the necessary expertise, of which experience is an important constituent, to actually make Scrum work. This is a view that I have much sympathy with, but I guess I distinguish between Scrum the methodology and Scrum the business and so am not required to hold the methodology to account for its', damn near literal, selling out by the associated business(es). That said, it was refreshing today then to read a defence of agile, and scrum in particular, on Robert C. Martin's 'Uncle Bob' blog.

http://blog.objectmentor.com/articles/2008/11/16/dirty-rotten-scrumdrels

I find myself in agreement with Uncle Bob's sentiment and arguments. The thing that I find a little surprising is the implication that teams can adopt XP or Scrum, maybe I have misread. Personally I have always thought of Scrum as a kind of management wrapper that works very well when surrounding XP practices. Another interesting post I read today was by Jeremy Miller.

http://codebetter.com/blogs/jeremy.miller/archive/2008/11/16/thoughts-on-the-decline-and-fall-of-agile.aspx

I agree with most of what Jeremy has to say, and for me it comes back to the twelve agile principles (http://agilemanifesto.org/principles.html). If we as a development team lack the discipline to ensure that each of these twelve principles are adhered to then we invite issues of the sort that Scrum's accusers make reference. Scrum provides a framework that I find makes the introduction of Agile into an organisation easier. It is not the endpoint to my mind, as always there is a maturation process and with the right people, and good use of retrospectives, the teams practices may evolve out of the strictures of Scrum and into something different. What it does do though is provide some initial norms that the team can work to. In a project I am managing at work we have brought together a number of contractors, all highly skilled and some with excellent agile experience, to develop some software where the need to 'provide early and continuous delivery of valuable software' could not be greater. Scrum, I find, accelerates the process through Tuckman's four stages of team development, enabling the team to reach the stages of norming and performing far more quickly that they might were they to have to deal with project management issues during the forming and storming phases. Quite frankly, given the timescales, it is quite enough that they come to a sense of group ownership of the architecture, design and requirements, I am more than happy to begin with a Scrum based imposition of how the team will work, and allow them through retrospectives (starting after at least a couple of sprints) to begin with debating, and incorporating, improvements. This to me parallels closely the idea from the 5-S model that Shitsuke only begins once the initial 4 S's have been established.

This parallel with the 5S's brings me to Lean. Something that I've not been able to entirely figure out yet is the way that Scrum seems to be set off against Lean so much at the moment. If I look at the 5-S model for example I struggle to see how these would not be carried into a successful Scrum implementation. Perhaps it is because a lot of the Scrum material typically focuses on the project management, but the requirement for every feature developed to be of quality (even if deciding on quality can be left to the team) is still there. If quality becomes a relative concept where it can be 'good enough' then this is up to the team. If the team is committed to agile then the hope would be that there definition of quality includes the principle that 'Continuous attention to technical excellence and good design enhances agility.' Kanban were introduced to me whilst I was working at Immediacy and have been an essential in my Scrum toolkit since, Scrum in its essence seems to me to be as much about promoting visibility as anything else and Kanban are an excellent way of achieving this. Like I would imagine any agile system to be, Scrum is centred around a Pull system vision where the software developed is closely driven by its end users. The system is also a self-Pull system where the team pulls in detail and direction as required, pulls in the removal of impediments or resources where required.

In all of these respects and many more Scrum provides an COTS for Agile teams, where the customisations may lead to the team no longer doing 'Scrum' in the strictest sense, it may be a more open 'Lean' style where sprints are not timeboxed for example. At least some of the roots of Scrum (at the least its' name) are in the work of Nonaka and Takeuchi, as well as in the Agile software movement, and their text 'The Knowledge Creating Company' is one that I keep close as a constant source of interest. This text with its' examinations of companies such as Honda, Canon and Matsushita provides valuable insights into the practices of management in Japanese companies which typically come under the banner of 'Lean'. As a point in case, let us again return to the 5-S model which was first developed by Canon (Japanese Management Association, 1987) who were studied by Nonaka and Takeuchi, who in turn provided, at the least, inspiration to Ken Schwaber. A tenuous link perhaps, but one that perhaps makes it less surprising that this model fits so well with Scrum. The common ancestry, commonality of practice, shared principles, and current open warfare between Lean and Scrum always put me in mind that often the intra-denominational arguments can be at least as fierce as extra-denominational ones.

As a final thought, when in his post James Shore states that 'when people say "Agile," they usually mean Scrum' I am left wondering if this is true. A great many people (particularly in project management and related capacities) who I deal with know little of Scrum, and many have not heard of it. They are as likely to think of Agile Unified Processes or DSDM, and in many cases are so imbued with Waterfallesque thinking that they apply Scrum as a label to their clearly non-Agile UP practices. Agile is not something that they currently grok. Whilst I think it is true that Scrum has gained a very real momentum, in no small part due to the business that promotes it and certifies some of its' practitioners, I do not think that this is in any way a bad thing despite this. As Scrum implementations fail some will no doubt use it as an excuse to write it, and possible Agile with it, off. To my mind such people are unlikely to be successful whatever they do, harsh perhaps, but not reflexively analysing and learning from failures is a sure way to stunt professional growth and progression (and probably a reason for a failure to make Scrum work). If, however, good professionals (and this is the vast majority of those whom I have been privileged to work with) reflexively analyse the reasons for a perceived failure of Scrum then in many cases I suspect they will see the problem as not being with the methodology, but rather with its' interpretation and implementation in their context. With the adoption of methods not compatible with it, and with the failure to adopt methods required by it, in short the failure to work in the Scrum way.

I Wish the Framework Team Used Interfaces More

One of the things that often bugs me about the .NET Framework is that so many of the classes are written without an interface. A case in point is the System.Net.Mail.SmtpClient class. This is an excellent class and very useful. It extends System.Object, but it does not implement an interface. This is extremely frustrating!

It's not that I would like to write my own version and swap it in, but I would like to be able to Mock it nicely and follow that development 101 practice of coding to the Interface. Not having an interface is very limiting, what if I want to use a Decorator pattern to implement logging (yes I know that it does some of this internally) or authorisation, no chance!

How hard would it have been to write it to an interface? It takes me no more than a couple of clicks and movements of the mouse.

Keeping code clean - Decorators for Error Handling?

I read a review a few days ago of the new book by Robert C Martin (Uncle Bob) 'Clean Code'. I'm going to order a copy as soon as I clear a couple of books from my current reading list backlog as it sounds interesting and the Uncle Bob blog on Object Mentor is one that I always enjoy to read. One of the things that stuck in my mind from the review was that the 'Do one thing' principle applies even to error handling.

Hmmm, if you see the example in the review (I don't know if its' from the book) then this is shown as not evaluating the exception and acting according to its' type, but rather just logging it. This has been nagging in my mind since I read it. Its' not that I disagree with the example, using exceptions for control of flow has always struck me as being a thoroughly poor practice in every place I've seen it done. It was more the idea that perhaps I shouldn't even have the try-catch block around my code in the first place. Now I know that the where's and when's of exception handling (let alone the how's and why's - any of Kiplings six honest serving men in fact) are a very contested country and I'm certainly not positing what follows as 'the answer', just an idea.

What struck me was that I could use the decorator pattern to implement my error handling. For example I have an interface:

public interface ICandidateRepository
{
Candidate GetCandidate(Guid candidateId);

Candidate RegisterNewCandidate();

void SaveCandidate(Candidate candidate);

List<Candidate> FindCandidates(ICandidateCriteria criteria);

}

I already have a concrete class that implements this interface and uses NHibernate to perform the functions that are described. Instead of wrapping these calls in try-catch blocks I can create a new concrete class that implements this interface and looks like this:

public class CandidateRepositoryExceptionDecorator : ICandidateRepository
{
private readonly ICandidateRepository _nestedCandidateRepository;
private ILogger _logger;

public ILogger Logger
{
get { return _logger ?? NullLogger.Instance; }
set { _logger = value; }
}

public CandidateRepositoryExceptionDecorator(ICandidateRepository candidateRepository)
{
_nestedCandidateRepository = candidateRepository;
}

public Candidate GetCandidate(Guid candidateId)
{
try
{
return _nestedCandidateRepository.GetCandidate(candidateId);
}
catch (Exception ex)
{
_logger.ErrorFormat(ex, "Exception when calling GetCandidate({0}).", candidateId);
throw;
}
}

// ... etc ...

}

I use my DI framework of choice to get the parameters into the constructor and to organise the nesting of my objects. In my case this is with the Windsor container. The components section is left like this:

<components>
<component id="candidateRepositoryErrorHandler" service="Semeosis.Examinations.Model.Interfaces.ICandidateRepository, Semeosis.Examinations.Model.Interfaces" type="Semeosis.Examinations.Model.Repositories.Decorators.CandidateRepositoryExceptionDecorator, Semeosis.Examinations.Model.Repositories.Decorators" />
<component id="candidateRepository" service="Semeosis.Examinations.Model.Interfaces.ICandidateRepository, Semeosis.Examinations.Model.Interfaces" type="Semeosis.Examinations.Model.Repositories.NHibernate.CandidateRepository, Semeosis.Examinations.Model.Repositories.NHibernate" />
</components>

The order in which the components are listed determines the order in which the container chains the objects. See this post by Oren for more info.

The upshot of this is that when I call:

ICandidateRepository repository = container.Resolve<ICandidateRepository>();

I get an instance of my CandidateRepositoryExceptionDecorator and this contains my CandidateRepository class from my NHibernate repositories project.

Now my concerns are separated and I can keep the functions that deal with the creational and persistence concerns separate from the error handling concerns. If in the future I need to change the way that my exception handling is manager, or I need to perform other actions than just logging when an exception is logged then I can very easily extend this code. Also unit testing this is incredibly trivial using mocking, whilst also reducing the number of tests on the functions that actually 'do the work'. Separation of concerns in one layer helps keep things clean and tidy across many other layers of the onion! I guess that's why TDD can have such a beneficial effect on model design - but the benefits can be two way.

Looking for a good developer

Recently where I work we have been interviewing candidates for developer roles in our team. I have been involved in conducting some of these interviews and this has been causing me to reflect on other interviews I have been involved in, both as candidate and interviewer.

Most of the candidates that I have seen, both recently and over the years, have been fairly strong when it came to their knowledge of C#, SQL, and the .NET Framework. Where typically they fall down is in their understanding of the ways in which these tools can be used to create quality software.

An analogy that has been rolling around in my mind is that of the developer as a builder (I'm sure it's not original to me). Now I'm not a builder (as my g/f would happily attest I'm not even much good at DIY) but it strike me that a good builder has certain key areas of expertise.

A good builder will know the materials that are key to their trade. She will understand the differences between materials, such as between different types of wood, or cement, girders etc...

But there are another set of skills that are important for a builder to possess before you would want them to work for you. Beyond this knowledge of materials a good builder will also know the techniques required to build things from these materials that are fit for purpose.

To build a wall it's not enough to know about bricks and cement, its' important to also know at least some of the types of bond, that's to say the commonly accepted strategies for building the wall.

Of course, what I'm saying and the parallel is not new. The GoF design patterns were inspired by the work of Alexander in the field of architecture (not software). But it is, at least to my mind, a useful analogy. You wouldn't want a joiner (carpenter) that didn't know about types of joints, so why would you employ and trust a developer that didn't know (and couldn't discuss to some degree) some of the common design patterns, or fundamentals of object oriented development.

The Price of .Net's Diversity

.Net, since its' inception, has always tried to be a friend to everyone. A truly general purpose set of software development technologies. It offer both RAD and enterprise level development tools, dynamic languages, functional languages, type-safe languages, and OOP languages. With the Entity framework, Astoria, the DLR, dynamic data, and more shortly on the way the incredible breadth of the .Net offerings just continues to expand - and that's without even mentioning any of the open source technologies.

But whilst with this incredible diversity we gain a lot of flexibility we have to pay a heavy price. Perhaps the biggest cost is the discipline we must exercise when we approach problems. .Net makes it very easy, and even seem rewarding, to do things badly.

Whilst something like Ruby on Rails makes it difficult to work following patterns and practices which are not standard in its community. .Net at best does little to encourage what are generally considered good practices (separation of concerns, low coupling, explicit intent, unit testing, etc...), and at worst encourages bad practice.

Take the provider mechanism introduced in ASP.NET 2.0. Yes this makes it quick to bind a data grid to a data source, but the moment that you need to do something beyond a demo app it start to lose its shine and make life difficult. Unfortunately a lot of developers struggle on anyway and concerns that ought to belong to a data or repository layer are placed in the markup of an aspx page. Yes there is an object provider, and as CLSA.NET does, this can be usefully put to work.

Another example is the dataset, invidious things that seem to be an overt attempt to subvert OOP, AOP, or any other good practices.

I could list many more examples. Often these things are a fools gold that promise much but deliver nothing. They lead to more code being written. They make unit testing at best very difficult, and often impossible. They inhibit changes in functionality. They make maintenance and code handovers very taxing.

But what concerns me almost more is that they seem to breed 'wrong thinking' and the proliferation of anti-patterns and code smells. They become a bad influence under whose spells developers, and perhaps even more teams, can fall and remain trapped.

I like the flexibility, and the diversity I get being a .NET developer. I would just like MS to adopt a stronger lead in making it easier for me, and others, to follow good practices. Happily there does seem to be some sight of some cavalry coming over the hill. In the web world the new ASP.NET MVC framework could go a long way to meet this critique, and perhaps Prism in the WPF world.