Continuous Integration with CI Factory part 2

I've just finished setting up a new build instance with CI Factory. This time I used the latest version (1.0.1.5 at the time of writing). I followed the steps here, together with some from my last CIFactory post.

I tried this time to add a property in Arguments.xml of this:

< property name="MSBuild.Framework.Version" value="${framework::get-framework-directory('net-3.5')}" />

In addition then I edited Compile.Target.xml to use this property like so:

program="${MSBuild.Framework.Version}\msbuild.exe" workingdir="${ProductDirectory}"

After running the run.bat file the only thing that failed was that I got the same old error about needing to set the path to the share for VSS so I went in to the ccnetproject.xml file and edited the ssdir setting to provide the appropriate path to my network share that holds my vss ini file (as described in my last CIFactory post).

I ran the CCNETServer.bat and that problem went away, but then I saw that the first build failed. On checking the log what I saw was that the MSBuild.Framework.Version property was not being set. To resolve this I went to the Compile.Properties.xml file for the MSBuild package and added the property in there, not ideal, but it did the job.

I now have a working build for the project on this server, alongside the build from the previous post (which is still to be upgraded). Next up the upgrade of the old one!

Defending Scrum

Recently Scrum has been taking a bit of a battering in the wider community, being made to take responsibility for some of the perceived failings of Agile. Under particular attack has been the scrum based certifications which are seen to churn out 'experts' on the basis of short courses. But who then in many cases lack the necessary expertise, of which experience is an important constituent, to actually make Scrum work. This is a view that I have much sympathy with, but I guess I distinguish between Scrum the methodology and Scrum the business and so am not required to hold the methodology to account for its', damn near literal, selling out by the associated business(es). That said, it was refreshing today then to read a defence of agile, and scrum in particular, on Robert C. Martin's 'Uncle Bob' blog.

http://blog.objectmentor.com/articles/2008/11/16/dirty-rotten-scrumdrels

I find myself in agreement with Uncle Bob's sentiment and arguments. The thing that I find a little surprising is the implication that teams can adopt XP or Scrum, maybe I have misread. Personally I have always thought of Scrum as a kind of management wrapper that works very well when surrounding XP practices. Another interesting post I read today was by Jeremy Miller.

http://codebetter.com/blogs/jeremy.miller/archive/2008/11/16/thoughts-on-the-decline-and-fall-of-agile.aspx

I agree with most of what Jeremy has to say, and for me it comes back to the twelve agile principles (http://agilemanifesto.org/principles.html). If we as a development team lack the discipline to ensure that each of these twelve principles are adhered to then we invite issues of the sort that Scrum's accusers make reference. Scrum provides a framework that I find makes the introduction of Agile into an organisation easier. It is not the endpoint to my mind, as always there is a maturation process and with the right people, and good use of retrospectives, the teams practices may evolve out of the strictures of Scrum and into something different. What it does do though is provide some initial norms that the team can work to. In a project I am managing at work we have brought together a number of contractors, all highly skilled and some with excellent agile experience, to develop some software where the need to 'provide early and continuous delivery of valuable software' could not be greater. Scrum, I find, accelerates the process through Tuckman's four stages of team development, enabling the team to reach the stages of norming and performing far more quickly that they might were they to have to deal with project management issues during the forming and storming phases. Quite frankly, given the timescales, it is quite enough that they come to a sense of group ownership of the architecture, design and requirements, I am more than happy to begin with a Scrum based imposition of how the team will work, and allow them through retrospectives (starting after at least a couple of sprints) to begin with debating, and incorporating, improvements. This to me parallels closely the idea from the 5-S model that Shitsuke only begins once the initial 4 S's have been established.

This parallel with the 5S's brings me to Lean. Something that I've not been able to entirely figure out yet is the way that Scrum seems to be set off against Lean so much at the moment. If I look at the 5-S model for example I struggle to see how these would not be carried into a successful Scrum implementation. Perhaps it is because a lot of the Scrum material typically focuses on the project management, but the requirement for every feature developed to be of quality (even if deciding on quality can be left to the team) is still there. If quality becomes a relative concept where it can be 'good enough' then this is up to the team. If the team is committed to agile then the hope would be that there definition of quality includes the principle that 'Continuous attention to technical excellence and good design enhances agility.' Kanban were introduced to me whilst I was working at Immediacy and have been an essential in my Scrum toolkit since, Scrum in its essence seems to me to be as much about promoting visibility as anything else and Kanban are an excellent way of achieving this. Like I would imagine any agile system to be, Scrum is centred around a Pull system vision where the software developed is closely driven by its end users. The system is also a self-Pull system where the team pulls in detail and direction as required, pulls in the removal of impediments or resources where required.

In all of these respects and many more Scrum provides an COTS for Agile teams, where the customisations may lead to the team no longer doing 'Scrum' in the strictest sense, it may be a more open 'Lean' style where sprints are not timeboxed for example. At least some of the roots of Scrum (at the least its' name) are in the work of Nonaka and Takeuchi, as well as in the Agile software movement, and their text 'The Knowledge Creating Company' is one that I keep close as a constant source of interest. This text with its' examinations of companies such as Honda, Canon and Matsushita provides valuable insights into the practices of management in Japanese companies which typically come under the banner of 'Lean'. As a point in case, let us again return to the 5-S model which was first developed by Canon (Japanese Management Association, 1987) who were studied by Nonaka and Takeuchi, who in turn provided, at the least, inspiration to Ken Schwaber. A tenuous link perhaps, but one that perhaps makes it less surprising that this model fits so well with Scrum. The common ancestry, commonality of practice, shared principles, and current open warfare between Lean and Scrum always put me in mind that often the intra-denominational arguments can be at least as fierce as extra-denominational ones.

As a final thought, when in his post James Shore states that 'when people say "Agile," they usually mean Scrum' I am left wondering if this is true. A great many people (particularly in project management and related capacities) who I deal with know little of Scrum, and many have not heard of it. They are as likely to think of Agile Unified Processes or DSDM, and in many cases are so imbued with Waterfallesque thinking that they apply Scrum as a label to their clearly non-Agile UP practices. Agile is not something that they currently grok. Whilst I think it is true that Scrum has gained a very real momentum, in no small part due to the business that promotes it and certifies some of its' practitioners, I do not think that this is in any way a bad thing despite this. As Scrum implementations fail some will no doubt use it as an excuse to write it, and possible Agile with it, off. To my mind such people are unlikely to be successful whatever they do, harsh perhaps, but not reflexively analysing and learning from failures is a sure way to stunt professional growth and progression (and probably a reason for a failure to make Scrum work). If, however, good professionals (and this is the vast majority of those whom I have been privileged to work with) reflexively analyse the reasons for a perceived failure of Scrum then in many cases I suspect they will see the problem as not being with the methodology, but rather with its' interpretation and implementation in their context. With the adoption of methods not compatible with it, and with the failure to adopt methods required by it, in short the failure to work in the Scrum way.

Continuous Integration with CI Factory

For the project that I'm currently working on I've needed to set up an automated build process, no surprises there. Having worked in environments previously where the development and maintenance of the build occupied a large amount of effort, and required a reasonable amount of specialised knowledge I was keen to ensure that this would not be the case (it wouldn't be sustainable here). That in mind I chose to use CI Factory. There is some excellent documentation on how to install CIFactory on their site, and I will not repeat this here. Rather I will focus on what I had to do to get my application up and running. The application in question is a .NET 3.5 application.

I downloaded CI Factory following the link from the CI Factory website. This is to version 1.0.0.76 and this is the version that I used. I have just noticed that if you click to view the full download list of all versions then the current version is 1.0.1.5. I will be doing my next setup with this and it looks like I'll be following the upgrade instructions from here, which should help me to better understand this tool.

The first thing I did was to get VS2008 Team Suite installed on the Build machine, and VSS 2005 (yes we are still using VSS - TFS should be along in the next couple of weeks but we need our CI in place now).

I opened CIFactory.sln, converting it 2008 along the way using the Wizard, then opened the Arguments.xml file. Editing this file I:

1. Renamed the project and pointed the projects location to the D drive. This is because, for SAN reasons, we have limited storage space on the C drive but the D drive can grow plenty.
2. Entered the build master and an initial set of developer details.
3. Removed the SVN details.
4. Removed the unwanted packages (for now it is very stripped down, I will look to add more packages in the future when we buy licences for things like NDepend)
5. Added the VSS package.
6. Added the VSS arguments.

I opened the Properties.xml file for the MSBuild package and edited it to point to .net 3.5 instead of .net 2.0 (more of this later).

I then opened the Properties.xml file for the Visual Source Safe package and edited:

1. The VSS.ExePath to point to the correct VSS install location, for me this is Program Files\Microsoft Visual SourceSafe\ss.exe
2. The VSS.TemplateDB so that it used the D drive

At this point I ran run.bat and it went off and created the build system, the project structure in VSS, the build solution and its' files all by itself and all very nicely.

I still had some extra work to do though.

I got an error that VSS needed to set the ssdir directory. A bit of playing soon discovered that doing this did not help. What I did to resolve this was to open the ccnetproject.xml file and edit it in two places:

1. Change the cruisecontrol\sourcecontrol\ssdir element so that it points to the correct path for the VSS share.
2. Change the project\sourcecontrol\sourcecontrolprovider\ssdir element points to the correct path for the VSS share.

The final thing to sort was in response to an error message "External Program Failed: C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\msbuild.exe (return code was 1)]]".

To resolve this I opened the Compile.Target.xml file in the MSBuild package folder and edited it to alter the version of the framework that it refers to from 2.0 to 3.5, so that look like this:

<target name="Compile.CompileSource">
<exec
program="${framework::get-framework-directory('net-2.0')}\msbuild.exe" workingdir="${ProductDirectory}"
failonerror="false"
resultproperty="Private.Compile.Result"
verbose="true"
>

Having to change this was a bit annoying as I thought that changing this initially would have prevented my needing to repeat this step.

With this change I got a green light from the build, happy days, and a nicely packaged set of files. I still have a bit to do, getting some more packages added in. I also have to get some builds setup for some other projects, and upgrade this install. Additionally once I've upgraded I'll see if any of the issues I've had look like bugs and raise them if they are.

So I guess over the next couple of weeks I may blog more on this.

Blog posts that I found useful to get this far are:

http://www.cifactory.org/joomla/
http://geekswithblogs.net/twickers/archive/2007/04/03/110691.aspx
http://geekswithblogs.net/twickers/archive/2007/04/03/110693.aspx
http://codebetter.com/blogs/jeffrey.palermo/archive/2007/11/28/upgrade-nant-for-use-with-vs2008-solutions-and-net-3-5.aspx

Finally a big thanks to Jay Flowers for putting this all together.

Visual Studio 2010 (Rosario) - First Impressions (1)

I've just gone through the pain of downloading each of the files required to extract the Visual Studio 2010 aka Rosario September CTP VPC. Why couldn't they use the Microsoft Download Manager??

Anyway, first things first, I opened Visual Studio 2010 and went to the New Project dialog. The first thing that I noticed was that the MVC Framework is missing. Now I'm not surprised as it is yet to be released, and I don't doubt that it'll make it to the final release, but it's a bit disappointing all the same as it means I'll have to settle for bad old asp.net when I get to the web stuff.

The new options that stand out to me are the WiX project options, nice to see that getting shipped with Visual Studio, and Modelling Projects (which currently just has an empty project). Being curious to see what this is this Modelling Project is my first point of call.

Choosing to create a new Empty Modelling Project presents me with this:

I get a Model Explorer on the left and in the Solution Explorer I can see that I have been given a ModelDefinition folder. Opening this folder reveals a single file - ModelDefinition.uml. When I right click and choose to Add a File I have a number of different types of model presented to me. These are:

• Activity Diagram
• Component Diagram
• Layer Diagram
• Logical Class Diagram
• Sequence Diagram
• Use Case Diagram

My first thoughts are that it's a useful enough set of diagrams to work from. I'm a little surprised not to see a State Machine available, especially given the way that Workflow Foundation might have some obvious synergy with this type of model. There's also no sign of support for OCL, but I'm less surprised by this (and certainly not disappointed). I've also never come across a 'Layer Diagram' diagram before, and a quick look at the OMG UML Superstructure specification doesn't seem to mention it - though maybe I just can't spot it there. That being the case the Layer Diagram seems to be like a logical place to start playing.

The Layer Diagram seems to be about showing the various layers that will make up an application, and the dependencies between those layers. It has 3 patterns available out of the box that you can drag and drop on; 3 layer, 4 layer, and MVC. Seeing the MVC pattern there is nice, but I'm not sure how useful this is. How many applications end up wth just these three layers, hmmm, maybe if you're following the ActiveRecord pattern but otherwise you're likely to have some form of Repository/DAO layer I would think. So being a generally DDD guy the first 'layer' pattern that I've tried to put together follows an Onion/Hexagonal pattern.

Ok, so why is this better than what I can do with basic shapes in Visio. Well, one thing is that I can specify acceptable namespaces for each layer (and unacceptable ones), drag and drop projects onto the layer from the solution explorer, which associates the project with that layer. Now if I specify an invalid namespace in a class file then when the model is validated I get a series of errors telling me of the bad namespace naming. Now I've got to say that this feature on its' own is not exactly compelling. Yes, I like to ensure compliance to conventions across the dev team(s) that I work with, but I wouldn't create this diagram just for that purpose. Also, I haven't created this in the way that I normally would - I typically follow the examples provided by Kyle Bailey and Jeffrey Palermo. With this I wasn't sure of how to represent the utilities that would normally be present. I was also tempted to break some of the layers up and make this more like a project diagram, a temptation I resisted. Another feature I spotted was the ability to put layers inside of layers, so I could have dropped the MVC layer inside of my UI layer, if I did though then suddenly for me my UI layer would contain layers that are typically just folders inside of a project for me. Maybe I'm just not 'getting it', or I'm just not working on big enough systems)(though I'd question the ability to manage the complexity of a system with many more layers that represented collections of projects. I guess that when more features are added to this type of model then its' real potential for usefulness will become clear. At the moment I'm not really sold on it.

Looking at the xml that is used to represent the diagram behind the scenes makes me worried about the potential merge conflicts that might emerge should two people ever work on the same artifact at the same time. This is something that I think I'll definitely check out soon as it has a real impact on how practical something like this can be when you are working as a part of a team.

Next up is a sequence diagram. These are diagrams that I use a lot and typically I use Enterprise Architect to create them. I've tried using Visio and always just end up incredibly frustrated at the lack of even basic support for this type of artifact.

Well, the less than headline news is that this is nowhere near being Enterprise Architect so Sparx Systems can sleep easy for at least another release of Visual Studio. On dragging a lifeline on to the surface I am able to specify that I am dealing with an actor. I am not able to specify that it is a boundary however which is something that I use quite a lot. Support for messaging seems to be ok with synchronous and asynchronous messages easily added into the diagram. Self messaging is also nicely catered for. Creating objects is handled ok, I prefer it when the arrow points to the box rather than to the lifeline itself, but I'm not that fussy. I can't see that it is possible to denote deletion from another object but I guess that given garbage collection in managed applications this doesn't ever really happen as such. I also can't see that it's possible to add constraints (OCL or otherwise). Frames seem to be supported through the 'Interaction Use' which I am assuming is an interaction frame.

Currently Guard clauses don't seem to be possible which is a bit of a miss. This is probably because the only operator currently available for the interaction frame is 'ref' and I'd need at least support for 'alt', 'loop', 'opt', and 'par' before I could make much practical use of the sequence diagram facility. Still there's at least a year to go before release so I hope that they do expand this feature. Also it would be nice to be able to drag a class on to the designer and then use it. With Enterprise Architect I can reverse my code base into a model and then get to select from classes and their available methods when I am putting the model together. I don't expect this feature to be comparable to Enterprise Architect when it releases, but if it is to be more than a gimmick there is a very long way to go. The good news for Microsoft (I think) is that I'm already more inclined to use this CTP than the UML features of Visio 2007 (which are just bloody awful), they may not be as feature rich yet, but they already feel more right.

Another post to follow as I continue to explore this CTP.