Recognised roles

Any project needs all sorts of expertise. In the picture 1 I have illustrated a typical, yet simple, scenario where the project needs some management, architecture, user interface design, development and it operations. The easiest and arguably most common solution is to assign one or more people into each expertise.

Person per role

This seems obvious: if we have a work to do, we need a person to do it. But this leads to problems well known: people are not collaborating effectively, they have their own silos, most of the decisions need to be done upfront which leads to waterfallish development no matter what your official methodology is. Most challenges in real world have cross-cutting concerns. Solving any such problem needs broad understanding of many aspects and in this setup this means you need many people work together. Collaboration is not bad in itself, but when you have two specialists who don’t understand each other’s expertise, it is not as efficient as it should be.

The solution to this problem has been examined before (see for example my problems in current resourcing). But as I see this one of the major hurdles to efficient (dare I say, agile?) development and I don’t see it being handled properly I need to keep refreshing on it. So, what we should aim for?

Person per gap between role: 10 required

Instead of using simple “person(s) per expertise” -method, we should probably use “minimal number of people that is enough to fill all the gaps between silos” –method. First you need to identify all the expertise areas that are needed to accomplish the desired end result. Then you need to find such persons that fulfill every gap between any two expertise areas. In our example, we had identified the five expertise areas so we have 10 gaps to fulfill.

Again, each of these gaps does not directly map into one person. Soon it becomes evident that if each team member only has one or two areas she is comfortable working in, you have a problem. Should each member have only two areas of competence, you’d really need 10 people. But, if even one of them possesses three different skills, the number of required people counts down to 8. If you have one “superstar” that is competent in four areas, you need only two people to fulfill every gap.

Persons with overlapping skills: only 3 required!

In any real-life project you have probably more than just 5 types of expertise needed: concept design, usability design, graphic design, requirements gathering, database programming, UI programming, testing, it operations, mathematical analysis, domain-specific knowledge etc. And as we know, the more people there are in the project, the more demanding the communication. And communication happens to be one of the most important aspects of any project, especially one of software deveopment (see: software development as a cooperative game). This reminds us about the importance of multi-competence.

So as a summary, the suggested method to project staffing is:

• identify the required expertise areas
• find the least amount of people that cover all the required areas
• if you need more firepower into any area, only then can you add a specialist

=> Sami’s rule of project staffing: Build the project on top of generalists, only add specialists if needed.

What we want to achieve:

• We do not want to duplicate validation rules into many places
• We want to have as much as possible validated on the client
• We do not want to have tons of boilerplate custom javascript

And the solution consists of the following elements:

• ValidationAttributes (System.ComponentModel.DataAnnotations)
• jQuery Validate
• Unobtrusive extension to jQuery Validate

The basics of attribute-based validation

Namespace System.ComponentModel.DataAnnotations contains many attributes that you can use to validate user input. Specifically, you have DataTypeAttribute, RangeAttribute, RegularExpressionAttribute, RequiredAttribute and StringLengthAttribute. I guess the names are pretty self-explanatory.

In addition, System.Web.Mvc namespace contains CompareAttribute which allows you to define that the values of two different fields must be the same (for example, password and passwordConfirmation) and RemoteAttribute which allows you to have complex validation rules executed on the client. In practice, you can define a controller and action which is executed by an AJAX call.

ScottGu explains the basics pretty well: http://weblogs.asp.net/scottgu/archive/2010/01/15/asp-net-mvc-2-model-validation.aspx.

Unobtrusive client validation

ScottGu’s example uses Microsoft’s own validation framework (MicrosoftMvcValidation.js) but in ASP.NET MVC 3 Microsoft allows us to use jQuery Validate and has implemented a nice addition on top of that. Brad Wilson posted a nice article about enabling unobtrusive validation: http://bradwilson.typepad.com/blog/2010/10/mvc3-unobtrusive-validation.html.

Extending the validation

It’s likely that sooner or later you find yourself having a custom validation rule that is not available in the previously mentioned framework. Most probably you’ll be missing the attribute on server side as jQuery Validate plugin actually contains a lot more validation methods than .NET Framework’s attributes represent, for example email validation for which the support is implemented below. You can find the list of all supported validation rules on the jQuery Validate’s documentation.

Custom attribute

It turns out that implementing a custom attribute is really an easy task. You implement your own class that inherits System.ComponentModel.DataAnnotations.ValidationAttribute and implements System.Web.Mvc.IClientValidatable. So you need to do three things.

1) Override public bool IsValid(object value)
This method will be run when the validation is done on the server (for example, if the client does not have javascript enabled). This is all you need to do if you don’t need client validation.

2) Create a class that inherits from ModelClientValidationRule. This is usually very simple. Here’s an example how to enable email validation on the client:

public class ModelClientValidationEmailRule : ModelClientValidationRule
{
    public ModelClientValidationEmailRule(string errorMessage)
    {
        base.ErrorMessage = errorMessage;
        base.ValidationType = "email";
    }
}

3) Implement public IEnumerable<ModelClientValidationRule> GetClientValidationRules(ModelMetadata metadata, ControllerContext context)

This is also usually very easy to implement, here’s the example on email validation:

public IEnumerable<ModelClientValidationRule> GetClientValidationRules(ModelMetadata metadata, ControllerContext context)
{
    yield return new ModelClientValidationEmailRule(FormatErrorMessage(metadata.GetDisplayName()));
}

This is all you need do to write your own attribute to enable validation using the readymade validation rules on jQuery Validate plugin. But there will be times when you invent new ways to do validation on client so you’ll need to write your own validation method for jQuery Validate.

Custom client validator

As you can guess, custom validators are not that hard to implement either. In the simple case (your validator doesn’t have any parameters) here’s all you need:

(function ($) {
    $.validator.addMethod(
        "your-method-name",
        function (value, element) {
            return (!value && this.optional(element)) || /*place your validation logic here*/;
        },
        "your own error message");
    $.validator.unobtrusive.adapters.addBool("your-method-name");
} (jQuery));

And to use this, you need to write your own custom attribute (and when implementing the ModelClientValidationRule, you need to set base.ValidationType = “your-method-name”).

NOTE: Usually it is good practice to first check whether the input is empty and whether the element is not marked as required. Otherwise using your own validator would also make the field required which is unexpected. The same applies to implementing server-side attribute’s IsValid method.

Conclusion and FAQ

This way we have a single solution for performing validation both on the client and on the server, and the usage of HTML 5 (yay!!!) data attributes for this kind of tasks is simply beautiful.

Q1: I have everything set up, but I still don’t get the validation happening on the client, it still goes to the server everytime.

A1.1: You do not have client validation enabled. Check that you have the following values on your web.config:

<appSettings>
    <add key="ClientValidationEnabled" value="true" />
    <add key="UnobtrusiveJavaScriptEnabled" value="true" />
</appSettings>

A1.2: You do not have the required javascript libraries loaded. You need to have jquery, jquery.validate and jquery.validate.unobtrusive included on your page.

<script src="/Scripts/jquery-1.4.3.min.js" type="text/javascript" />
<script src="/Scripts/jquery.validate.min.js" type="text/javascript"/>
<script src="/Scripts/jquery.validate.unobtrusive.min.js" type="text/javascript"/>

Q2: I have validation rules that I cannot express by static attributes. How to use more complex validation rules?

A2: On your action method you can have arbitrary validation logic. To put the error messages into the same output system, use ModelState.AddModelError("<fieldname>", "ErrorMessage");

The actual operation will begin 30.8.2010. If you want to be the first who will enjoy our march…. info@offbeat.fi.

• The services and products we will provide will surely have something to do with .NET, agile and conscious development
• We are four people together: Jouni, Riikka and Lauri.
• The company name still remains to be seen

More info about the company and services will follow soon, now it is time to find some clients and get the old job finished. We are living interesting times!

Iterative development

Wikipedia has the definitive explanation on what it means to iterate:

Iteration means the act of repeating a process usually with the aim of approaching a desired goal or target or result. Each repetition of the process is also called an “iteration”, and the results of one iteration are used as the starting point for the next iteration.

In software development the desired goal is of course a shippable software product. So instead of having a big bang one-shot release (as known from waterfall process) the development team comes up with one version, which is immediately iterated into next version. The product is ready as soon as the product reaches good-enough quality level.

Iterative development is good for providing rapid feedback from users, but it’s weakness is that the whole product must be taken into production in one shot. The whole product is iterated all the time. (This of course is in the purest iterative, non-incremental case)

Incremental development

Let’s again borrow the definition from Wikipedia:

Incrementalism is a method of working by adding to a project using many small, incremental changes instead of a few (extensively planned) large jumps.

So what this means is that incremental is what has been called staged delivery for decades. It means you develop one part of the system and when it’s ready you develop the next part. In the purest version, you can first make the analysis and design, then develop first increment then develop the next increment and so on as long as all the features are developed.

The difference and the synergy

The elevator pitch for iterative and incremental development goes like this: iterative means making better something that has been done earlier and incremental means adding new features on top of what has been done before. These two ideas can be used without each other (as shown earlier) but they can be used together as well – which is actually a Very Good Idea as it turns out.

Iterative-incremental method space

Alternative view on iterativeness and incrementalism

As incremental development goes through all the phases of development (analysis, testing, development, design, integration, production) it is a perfect tool to expose all problems in the process. Should the integration be difficult, using incremental process you’ll face it early and repeat it many times so you’ll have to fix the problem, not postpone it into the end of the project (when you’ll be already late).

Iterative process means rework on the existing product, so it doesn’t help that much on the process-side, but it makes the product better. Usually agile teams first come up with a walking skeleton which is an end-to-end solution to the problem but may miss some features. So after getting feedback from customer the skeleton is iterated into next version by adding some muscles onto it.

So we could state that:
- incrementalism improves the process
- iterativity improves the product

Enter agile

So agile methods are said to be both iterative AND incremental. As we learned from previous chapters they are different things and actually we can think of them orthogonal axes in the method space (see picture). Plain waterfall is non-iterative, non-incremental process (have you seen such in reality?) so it is in the lowest left corner. The good-old staged delivery is incremental but not iterative so it is in the lowest right corner.

However, spiral model is highly iterative but not at all incremental so it is in upper left corner. Thus the last corner, upper right, is reserved for agile methods. I’ve drawn it as an area, not a point because there is not just one agile process but many different flavors. The process must be in that area to be agile by definition but the amount of iterativity and incrementalness can vary. This is actually one of the key questions when architecting the process for any given project.

Sprint, increments and iterations

What is a sprint anyway?

In Scrum we use the term sprint. What does it mean, is it an iteration or an increment? The term sprint is merely a fixed timebox, and Scrum doesn’t care whether you are iterating or incrementing your project. Most scrum teams keeps doing both types of work in any given sprint.

A good pattern is to think of it this way: the sprint is one increment in which you do all the activities you need to, from analysis to deployment. Think of it as a release. Inside a release it is good idea to have a few iterations – or internal releases – to make sure your increment will meet the requirements of your customer who will be analyzing your work in the sprint review.

Now, at the release of Sharepoint 2010, the lite version formerly known as Windows Sharepoint Services, will be renamed to Sharepoint Foundation.

Finally, the naming of these fundamental technologies will achieve it’s closure: The programming framework and paradigm that we originally called Lightning, was released under the name .NET Framework. Now, at the release of version 4.0 the framework will also be renamed to conform to this naming convention and as expected, the new name will be Windows Technology Foundation. To celebrate this remarkable milestone, Microsoft has acquired the well known site and will continue to bring daily news about all WTF technologies.

Skill sets

First of all, the skill sets that really are taken into consideration, are mostly technological. Like “Sally is a java programmer, Joe is database administrator specializing on Oracle and Harry is a tester while Sophia is a UI designer”. Someone has somehow calculated how many units of each resource type project needs and thus a project team is formed.

This model doesn’t take into account what kind of personality types the specific project needs. Should we take Sally the java programmer who is not comfortable with iterating but usually produces error-free code from decent design, or Scott the java programmer who is at his best at trying different things out but doesn’t want to wait for design to be finished? Should we take Sophia the UI designer, who is considered more senior and competent on her own but is unable to work with customers or should we take Chris the UI designer who shines with customers but who often gets to argue with other team members?

Furthermore, this model completely ignores the fact that a team is more than the sum of it’s parts. A team may vastly benefit from a person who is complete junior on every technological aspect but really makes the people work as a team.

If these decisions are not made consciously, the project will most probably meet unexpected challenges in getting into working effectively.

Productivity differences

In any given resource pool you have many people with similar skill sets. Are each of them similarly productive? We have significant evidence that between two programmers there can be 20-time (the multiplier varies from 10 to as far as 50 on different studies) difference in their productivity. Do you know the productivity characteristics of each member on your resource pools?

Team is more than a group of people

By assigning project teams merely resourcing each member from their respective pools you do not get an actual team. Instead you’ll just put up a group of people who are not enjoying the empowering effects of a well-working team. A team in which every member knows each other will have productivity that is far more than the sum of its each individual member.

Teams are not something that any single person can mandate to exists: a group of people decides to work as a team or the do not. Finally, if you happen to come up with a jelling team, resource pooling often breaks each team after every project. So you’ll face the team building phase on every project. It’s not the most efficient way of working.

So what to do about it?

1)       Forget resources

The first step into the right direction is to quit speaking of people as resources. Considering people as resources proposes the usage of resource pools and ignores the notion of a well-functioning team. Instead of focusing on setting up resource pools and optimizing individual utilization we should focus on building highly efficient teams that are capable of organizing themselves at all times. Teams should be big enough to allow assigning the best person to any given task but small enough so that the number of communication channels does not overwhelm the benefits of people knowing each other.

2)      Consider organizational structure

Do you have your workforce organized by functional expertise? Project management team, UI design team and development team, database team and testing team? This grouping doesn’t encourage self-organization and introduces many possible bottlenecks to your workflow. People working together should stay together and have a common organizatory goal.

3)      Let teams build themselves

A jelling team can typically consist of 10 or less people. Bigger than that and the communication channels get far too many and the diversity of the persons get in the way of efficient collaboration. You may have organization units that are bigger, but inside units you may have more teams. If you as a unit manager see some people getting along especially well, why not letting them work together as a team?

4)      Emphasize specializing generalists, not pure specialists

Each person who has strictly limited expertise is a bottleneck to productivity. Specialists need a well-defined role, they need a project that can use a specialist of his profession. When your people are capable, and willing, to act on a multitude of roles, your staffing problems get a lot easier.

A group of specialists can make – and usually do – products in which any given area is crafted well, but the product as a whole doesn’t work at all. To make anything remarkable you need a group of people who understand what others are doing, people who understand the product as a whole.

Surely you need the specializing part, too. A group of pure generalists will surely do something that works somehow, but it won’t shine on any aspect. The optimal situation is having a team full of generalists, each of which could carry any role in the project, but where each has her own specialty on top of good general knowledge.

=>  Sami’s first law of productivity: Productivity is directly proportional to the amount of overlap in team members’ skill-area / expertise-level –space.

But there are sub-stories also. In physics we have had Newtonian and Einsteinian stories, and these stories constantly change. In this post I urge you to think what are your stories of software development. There are at least three different types of stories that define you as a software developer.

What is software development at very high level -story

In the last decades of software development we have not been in lack of paradigms to choose from. Many have come, many have gone. And I agree, we still are trapped in nonfunctional paradigms. Here are a few different paradigms that Alistair Cockburn pointed out:

Of course, these are a bit philosophical paradigms from methodologists and as such do not define very strictly how you will behave in every situation. But it is a foundation on top of which the other stories build. Do you have a different metaphor for software development that describes your style better? Let me know!

What is a software project –story

This is the type of story that has caused most process wars, manifestos to be written, different schools of thought to be born and continuous flux on the science side.

Educated 90´s software development

Perhaps the most common story is what was teached at school in the 90´s, being a mixture of Meyer´s engineering, Humphrey´s discipline and Jacobson´s model building. This most often resulted in the traditional waterfall-development in which each activity is carried out sequentially by specialists of each phase.

This story encourages for big, upfront design, comprehensive documentation and project management as a separate task. And testing postponed to the end of the project.

Software project is a collaborative game

In many parts, this is the original way of writing software – that is doing an reincarnation in the names of lighter-weight methodologies. You basically take a cross-functional team (preferably small) of people and trust them to get the job done. This philosophy is behind the many agile methods that coming with varying set of practices, principles and rules on the game.

Usually this story comes with iterations, incremental development, continuous integration, behavior driven design and test driven development etc.

Alistair Cockburn has even composed a manifesto for this style:

“Software development is a (series of) cooperative game(s), in which people use markers and props to inform, remind and inspire themselves and each other in getting to the next move in the game. The endpoint of the game is an operating software system; the residue of the game is a set of markers to inform and assist the players of the next game. The next game is the alteration or replacement of the system, or creation of a neighboring system.”

Cowboys on the desert

In practice, many software teams are not willing – or capable of – carrying out monstrous project framework and processes that 90´s style requires. Further, they do not have the discipline neither responsibility that collaborative game needs. Instead they work like cowboys on an unknown land: without any idea where to go, they take each confronting sack, changing direction continuously and ultimately ending somewhere leaving a disastrous mess behind.

In software terms this means that there is a bunch of individuals who are not collaborating fluently, requirements drop unexpectedly from customer and there´s virtually no communication between the customer and developing team except what is absolutely required to hand off the requirements and at the end, deploying some piece of somehow working software.

Requirements change wildly and they are implemented right away without thinking the architecture and keeping the code clean.

Systems thinking

Some go as far as saying you should not have projects at all, but think development artifacts as products. Usually projects have up-front funding, scope fixed at the beginning and success is measured in terms of meeting the scope in given time at given cost. What´s worse, the team is often disbanded and each member goes on to new projects. A new team will step in and take care of the maintenance.

Alternatively, in systems thinking the development is funded incrementally, scope is expected to evolve along the way and success is measured in terms of market share and profitability. The same development team often stays with product.

What am I as a developer –story

Finally, after each person´s beliefs about development in general and reckoning about projects (or products) each still has a different attitude towards his work

Stonecutters, living earners and cathedral builders

There´s is a great story about three stonecutters in Kitchen Table Wisdom.

It’s a story offered by psychologist Roberto Assagioli, who tells of a man who goes to interview three stonecutters engaged in building a very large cathedral. When he asks the first stonecutter what he is doing, the man bitterly replies that he is cutting stones into blocks, a foot by a foot by three quarters of a foot. With frustration, he describes a life in which he has done this over and over, and expects to continue to do so until he dies.

The second stonecutter also says that he is cutting stones into blocks, a foot by a foot by three quarters of a foot, but he adds an important piece. He lets the man know that he is earning a living for his family; through his work his children will have clothes to keep them warm and food for them to grow strong, and that he and his wife will be able to maintain their home, which they built with so much love.

But it is the third man’s response that astounds the interviewer. In an enthusiastic voice, he tells the man how joyous and grateful he is to have the privilege of building a great cathedral, so strong that it will stand as a holy beacon for a thousand years.

http://clf.uua.org/quest/2008/12/janamanchi.html

Are you just typing some freakin´ code, putting up darned CSS styles, <put here your current day-to-day activity> or are you part of  team who is building an exciting new product? Or are you just doing whatever you´re told to do so that you can feed your kids? When you are annoyed by your job what do you do? Complain about it? Ignore it? Fix the problem?

Technology lovers, tool blamers and superheroes

Besides the overall attitude to your work there are some characterizing ministories that you may have absorbed.

Are you a superhero that often works night´n´days, sacrificing your own life because “it´s the only way to save the project”. Do you think the other people on your team are inferior and you are the only competent guy?

Or do you think that you and your team are perfectly competent and suitable for the project – except that the tools are always wrong. Be it the operating system, version control software, time-tracking software, internet browser… You are doing your job perfectly but the tool just prevents you to be productive. And there is nothing you can do about it.

You may also have fallen in love with technology. Be it all the new technology in general or any given brand that has dazzled you. Anything you think is from the point of view of the newest technology.

Different types of developers

Alex Love has written a nice overview of different types of developers. Do you recognize one of the types to describe yourself?

Afterthoughts

While it may be hard to find a single story that works for you all the times, it´s important to understand your own way of thinking. Equally important is to understand the stories that your team mates carry with them. Conscious team leaders make up teams that consist of people with compatible inner stories.

Have you noticed different types of stories that define software developers? Feedback is welcome!

During the 50 years of software development we have seen a plethora of software development methods. Every ten years comes a new methodology, a new solution, to the same old problems we have been fighting for the whole 50 years. And will be fighting, dare I say, the next 50 years. One weapon used in the war is development methodologies. But what really is the essence of a methodology?

A lone coder

Imagine you are developing software alone for yourself. You know perfectly well, WHAT it is you want to achieve. You are effectively a business analyst. Also, you alone specify HOW you are going to implement the software. Naturally you write the code and test the solution yourself.

I bet anyone who has wrote any non-trivial software for himself, did find some opportunities to make the software better than originally thought. When you work alone it’s usually not a problem, you change your mind and are happy with that.

You don’t need much ceremony for this to work.

Enter external client

Things get a lot more difficult when another person comes into the game. Especially if the other guy is your client who is going to tell you WHAT to do.

When two or more people work together (you are working together, aren’t you?) they need a way to coordinate their work, share knowledge and find a way to make decisions.

This comes extremely important when you need to agree on WHAT to build.

Bigger team, more roles

Usually as one person turns to an organization and projects grow bigger, more roles become involved in the game. Most often first person to enter is an hybrid of project manager and analyst taking care of the client and telling the developer WHAT to do. What effectively occurs, is more loss of information (according to many studies ca. 50% of information gets lost on a handoff).

Next people to come into an project are architects, designers (telling developer HOW to do) and testers (trying to validate whether the developers did what the analyst told them to do, no matter is it what the client wants, no matter was it done how designer thought). More coders. More clients with conflicting ideas on WHAT to do. More problems. Many methodologies have been born to solve these questions, many theories about teamwork have existed many decades, so we are perfectly suited for a successful project?

Each new person presents new demands for the development methodology.

And the ultimate question is…

So now we have a team of people (without going into discussion on what is the essence of a team) consisting of many specialists perfectly suited to do their own craft. And the team has a client who has a business process to be improved using software. The only remaining problem is that the team should produce what the client needs. And it needs answering the ultimate question WHY. Why is the software being developed? Why is client asking for what she asks? Why is designer proposing a given solution?

To answer that question (and also to further specify the WHAT) business processes are modeled and various diagrams are drawn.

The problem

Developers have their own methods, business has it’s own methods and IT operations has it’s own methods. Unluckily, there’s very little common ground between the different stakeholders. There are huge gaps between the different sides. And after a few thoughts, we find that our specialists-driven team has also many gaps: designer doesn’t understand what each decision means development-wise, developer doesn’t understand the UI designer etc.

We need something to help us bridge the gap or even better, remove them totally.

The solution

Each project has it’s very own characteristics: it has a unique combination of skills, unique value definitions, unique scheduling, unique state of the world and – what’s most important – unique human beings. This means that the project needs to find a way to work effectively together under these unique circumstances.

I have written before on cross-functional teams. But it doesn’t stop on the software development team. We need a cross-functional business development team. Most software is written because of business change, and as business changes continuously, so do software.

For any project to succeed it needs all the stakeholders speaking the same language, sharing objectives, collaboration, building trust and enabling business agility. Real methodologies always try to find practices that bridge the gaps between end users and client, between client and analyst, between members of development team etc.

“The essence of Lean is thinking in context” someone said. I couldn’t agree more. Thinking in context is the very essence of software development. We have come too far in determining readymade methodologies. Think in context, define your own methodology for your project and bridge the gaps.

Afterthought

A football team consists of highly trained specialists. They have goalkeeper, defensemen, middle-field and the forwards. Each one is focusing primarily on his role. What is the difference between a football team and a typical software development team? Each member on a football team know how to play football! Every single player can kick the ball, they know the strategy, the are able to serve the ball, they know how to tackle the opponent etc.

Typically on a software development team you find testers who can’t write code, you have designers who can’t write code, you have coders who know nothing about user interfaces (or even the users…). Why is that?