In every team trying Scrum, there is bound to be a rule about points.  I dare you to put up a hand and say you have none.  These rules are things like “We can’t take anything over 13 points into a sprint”, “Our epics are 100 points”, “The login screen is our baseline of 3 points”, “Anything over 40 points must be broken down”.  So, I double dare you

I have different view of the humble point. A point may seem like a one dimensional thing, but it has a some facets built into it.  One facet is the “amount of effort to build something”.  Another facet is “amount of ignorance” and this has an inverse – “amount of shared knowledge”.  Sometimes I find it useful to make a judgement based on what I don’t know as opposed to what I do know.  Regardless of whether I choose to view the cup as half full or half empty, I cannot estimate effort to build something based upon what I don’t know.  So, effort tends to track the amount of knowledge, not ignorance.  As knowledge increases, my ignorance decreases and each point starts representing more and more of pure effort.

However, if I am in a state of complete ignorance, then it is completely impossible for me to make any judgement on effort to build.  I’d be simply speculating.  What I can do, though, is create a time box to explore the unknown so that I can start moving out of my state of ignorance.  This is also an estimate and I am not making an excuse for non-delivery either.  I need to understand some things and also show my understanding in some code.  Yes, the code that I produce may not have a visible user interface or some other convenient demo-friendly stuff, but I need to carefully plan my sprint review to express my understanding.

It’s all about gaining a SHARED understanding. This understanding is body of knowledge that I have learned which I need to confirm with others.  This act of confirmation can happen in several ways.  I can have a conversation and explain what I understand, I can draw a blocks and lines picture, or show a spreadsheet, and so on.  Regardless of the method of communication, I still use the opportunity of discovery to express my understanding in code as tests.  Another powerful way of expressing my understanding is to write out a story and a few scenarios.  Using BDD style grammar can be a great way of concisely expressing some things, that can be easily shared.  Yes, you heard me correctly – as a developer, I write the stories and scenarios.  When I am given a story and scenario by someone and asked to estimate, then I am attempting to estimate based on  another person’s expression of their understanding and my assumed understanding.

In a recent discussion with Jimmy Nilsson, he said that he prefered to call scenarios “examples”.  That really resonated with me.  I also do a lot of discovery by example, and then gradually introduce more a more into the examples, as I get more and more confident of my knowledge.

How do I know how much I don’t know? That’s a tough question.  What I do comes straight out of my TDD habits.  I create a list of questions – my test list.  For some questions, I will know the answer easily, some not all, and some are debatable.  The more that I can answer, the better I can estimate effort.  I can then turn the questions that I can answer into statements of fact.  The more facts I have, the less ignorant I am.

Recently, I worked with a team that wanted to get TDD going, and the most significant change that I introduced was in backlog grooming and sprint planning.  During these two ceremonies, we (as a team) threw questions madly at a requirement, regardless of whether we knew the answer or not.  We then worked through the questions (as a team) to establish how much we could answer.  The trend that emerged was that the original estimates where either half of the new estimate or double of the new estimate.  When they where halved, it was generally because we were able to negotiate some of the unknowns (the ignorant areas) to a future sprint with the product owner.  In some cases, the product owner was equally ignorant, and was reacting to the “business wants the feature” pressure.  When they were doubled, it was so much more was discovered than originally assumed.  At the end of the session, we always asked the meta-question “If we answer all these questions sufficiently, will we be done?”.  I call this style of working “test first backlog grooming” or “test first sprint planning”.

Often I discover more things I don’t know. Annoyingly, this happens in the middle of a sprint, but if it did not happen in that phase of work, then perhaps I was not digging deep enough.  When this happens, I just keep on adding them to my list of questions.  These new questions are raised at any time with others on the team, the customer or with whoever can help me understand a bit more.  Sometimes, it’s put on the table for negotiation to be dealt with at another time.  Nevertheless, standups still seem to be a good time to put new questions on the table, for discussion later.

There are several ripple effects of thinking about points in this manner – this notion of ignorance and shared knowledge gauges.

The first is about the possible shape of your sprint backlog. If you have deep understanding, then it is likely that you will be able to decompose complex problems into simple solutions, that take less effort.  The effect is that low point stories are in greater number in a sprint.

If you are highly ignorant, then the estimation points reflect that and there are more medium to high point stories in the sprint.

Sprint backlog shape with high ignorance

The second is about what you value in a story. You will find less value in the ontology of epics, themes and stories.  It is no longer about size of effort but degree of understanding or ignorance.  Instead, the shape of the product backlog is something that is constantly shifting from high uncertainty (big point numbers) to high certainty (low point numbers).  That’s what test first backlog grooming gives you.

Shape of product backlog - higher knowledge items have smaller points.

The third is about continuous flow that is the nature of discovery.  When you work steadily at reducing your degree of ignorance, then you are steadily answering questions through answers expressed in code, and steadily discovering new questions that need answering.  This process of discovery is one of taking an example based on what you know in this moment and modeling it.  Then expanding that example with one or two more additional twists, and modeling that, and so it goes.

It also touches product ownership and software development. When you work in this way, then explicit estimation of effort becomes less significant.  Moments that have been earmarked as important  points in the life of the product become more significant.  Call them milestones.  These milestones are strategically and tactically defined, and become a dominant part of product ownership.  Software development becomes the act of having long running conversations with the customer.  Those milestones give context for the content of those conversations.  Ultimately, those conversations are then expressed as a set of organised thoughts in code.  If your code is not organised well, then perhaps you also don’t understand the problem or solution or both.

This is a long story for a short message. A high priority is to resolve the tension that exists in an estimation in the form of knowlege/ignorance fighting against effort.  When you release that tension through shared understanding, then you can deal with the tension that exists in the act of creating those significant milestones.  In my opinion, that’s the real wicked problem.

No collaboration, no heroes!

I thoroughly enjoyed Karen Greave’s talk on Agile Testing.  She had just about 100% coverage (pun intended, groan).  Yet, testing is really a pain in the rear.  Testing is execution, and Karen was dead-on right, that automation is the path to follow.  Computers are very good at testing.  A computer does what it is programmed to do, and it can test the way it was programmed to test.  It’s simple: if testing is your constraint, move that constraint away from testing by automating.

Now, you have to deal with the constraint that shifted to the next point: test authoring.  While testing (i.e. execution) is just a passive, laborious effort, test authoring is a very creative, active exercise.  It is actually an exercise in confirming a shared, common understanding.  Kristin Nygard said “To program is to understand” and test authoring is a programming exercise.  That’s why outside-in, behavior driven development style scenarios are actually tests, coded in a human language.  The act of authoring a scenario proves your understanding and the expected working of the software.

This is why I separate test execution (passive) from test authoring (active).  And Karen said that early feedback is good (right again), which is why I author my tests very early.  I’m extreme about this.  I test first.

Automation creates time for collaboration

But there is another angle to Sustainable Pace.  As a developer, you need to build a rhythm, or flow.  It’s a cadence that you establish as you are writing code.  It’s a cadence that TDD helps you establish itself.  This cadence is also sustainable pace.  And one thing that kills this cadence and your pace in a flash is a mid-stream meeting.

In the panel discussion afterwards, there was a question at the end regarding ways to reduce the number of meetings which I just glossed over.  If you schedule meetings with developers for only the first hour in the morning, you not only reduce the number of meetings but, also, you don’t destroy the sustainable pace built up from the morning.

So, don’t think about pace as just working 7 hours a day, it’s about what you do in the 7 hours that matters.  Get the rhythm going and be anal about things that can kill your flow mid-stream; especially meetings.

DevDays 2010 in Cape Town was slick.  Very slick.  You guys really know how to put an a good show.

Most of the speakers were good.  Most demos were good.  Most jokes were funny.  The food was mostly good. The mood was mostly good too.   And the fan club was mostly impressed.  And most noobs were converted forever.  And the new features were mostly so good, apparently, you won’t have to write so much code anymore.  And the most underused word was design.  Most absent words were TDD, refactoring, quality, and clean!!

I understand that DevDays is a product showcase but, come on, MS South Africa, at least cater for the entire spectrum of developers, just a little bit, and in a responsible manner.  How about pitching content that shows that you do care about design, about clean code and quality.  How pitching the new features in a way that shows a trend towards agility, to being able to refactor code continuously,  to test first and other vital aspects of professional software development.

Let me give you just an example to illustrate what I mean.  The EF4 code first demo was explained completely as if the classes in the model are no different from entities in a table.  Even the language used was “entities” and “keys”.  I don’t think I heard the word “class” or “object” once!  You may have slides with the words “Code First Model”, but if you don’t actually do it for real, then you’re just leading dataset-happy developers that are marginally object-oriented further away from good code and good architecture.  You need to explain why it’s better:  that it promotes better object orientation, that POCO models disconnected from an ORM can be done test-first, and you can evolve your model, instead of designing up-front, blah, blah, blah.

Another time there was a ping-pong table with a developer at one end and tester at another with a bug being batted between them.  That pulled quite a laugh from just about everyone.  While that is reality in many organisations, there are many of us that test our own code and deal with our own bugs.  The tester that we toss our code to is our automated continuous integration server.  Tossing code downstream when it’s too late for reprise is not very professional.  How about focusing on the testing tool, as opposed to pitching it in a manner that makes everyone believe that dealing with the bug downstream is just the way its meant to be.

Sure, I know that you need to show off the latest cool things and evangelise your products, but there is a sector of developers that you are blatantly ignoring.  It is the sector that is, perhaps, the most influential amongst other developers. We are those developers that value our craft of software development.  We evangelise the craft and the value that it brings to our lives, our teams, our projects, our clients and our organisations.

Perhaps I am just delusional.

Oh well, so long and thanks for the fish.

That’s why I’m giving the TEST FIRST TDD course next week.  If you want to go beyond just learning about an xUnit API and step on the path of a personal journey to changing the way you create software, then come along.  I don’t have miracles but I can do better than just shining a light.  I will step into the darkness with you and help you move towards the light.

Really?

No!  Even though there wasn’t this big crash, someone did get a fright of their life, even I as an observer got a fright. And someone else did continue their journey quite shaken and certainly not in their same frame of mind.  The only person that seemed to be least affected was the person who jumped the red light.  But I am speculating, maybe he was under pressure and really needed to get to the end on time, and so he rushed ahead and saw in his rear view the potential destruction he left behind.  And maybe he regrets his decision.  But he certainly did not see how his actions affected the other people that were close by, myself included.

Driving through a red light can kill you and you can kill other people too.  The same thing happens when ignore the red light from your tests.  You can hurt yourself and you can hurt other people too.

Just think about that red light at the traffic light and in your code.  It’s telling you so much.

• It’s telling you to slow down a bit.
• To look around.
• To think about the moment.
• Don’t focus on the end, you will get there in good time.
• If you ignore the red light, other things will break and other people will get hurt.
• Maybe you don’t know why you must stop, but you need to stop first, before you can find out why.
• It’s telling you to do the right thing, not the rushed thing.

Driving through a red light is another suicide pattern that I will add to my growing list.

And TDD is also part of ubuntu coding.  I am not telling you anything new.  I’m just telling you the same thing from another perspective.

The one thing I can tell you is that the course is a very immersive experience.  I really hate lecturing but I enjoy probing conversations and that’s how I give the course. I don’t have answers to the practical work and concerns are addressed as we go along.  As a result, the day takes unexpected turns and routes.  But in the end I get you to the right destination.  Come along; you will leave exhausted, but inspired!

Take the Fast Track to Domain Driven Design

about the course /  course contents / should you attend? / register for the course

factor10 has expanded its services in South Africa to include our advanced and
expert level courses aimed for the software professional.  On September 8-9, 2009,
we will be offering a fast track to DDD for Architects at the BMW Pavilion in
Cape Town.

Who should attend?

This course is for software professionals that want to take the right steps towards
advanced and expert levels in their careers.  Register for this course if you want to …

• learn more than just another syntax and set of tools
• write software for large, long living systems
• increase the quality and maintainability of your design
• design high quality models and use code to represent those models effectively
• develop applications with a good APIs
• add a design edge to your skill set

Why should you learn DDD?

More and more developers and architects realise that learning every detail of a new
API just isn’t the way to deliver the best business value. It’s such a tough balancing
act; focus on the solving the business problem and focus on building working software
with your frameworks.

One way of taking a big leap in the right direction is to learn and apply domain driven
design. It is definitely not abstract and fluffy; it deals a lot with the code also. DDD
leads us to focus on understanding and to communicate that understanding very well;
in language, in design and in code. You will shift your focus away from designing for a
technology, and you will learn to design for the business domain; to the core of the
problems and solutions. Those are the most interesting parts and what your users
and customers really care about.

about the course /  course contents / should you attend? / register for the course

The feedback loop is missing.  They view the stories as a spec that has been handed down.  And they have not made the connection that spec writing is design work that is intended to clearly illustrates concepts in a domain.  It is a form of writing code.  But it’s just that this code is, maybe, non-executable.

Here’s my workflow and how I close the loop.

• write story and scenario
• Sketch a design if needed – helps when pairing to be on the same page.
• Start writing test for scenario
• ooops … test is getting complicated? stuck?
• maybe the domain is not understood enough? Dig deeper, improve scenario, design (as needed) and continue writing test
• or maybe the scenario was badly written? Ignore scenario structure, continue writing test.  Refactor scenario later.  We’re in deep discovery mode here.
• get test to pass
• refactor code
• refactor scenario
• … cycle the red-green-refactor until happy.

Acknowledging when you’re in discovery mode and knowing that you are allowed to refactor requirements is the trick.  Nothing is cast in concrete.  That’s why I like frequent feedback loops with tight turning circles.

No feedback loop, no progress.

BTW, I really don’t like explaining such things as flow-charts and sequences.  You got to find your own style.  It’s not a recipe or rules thing.  The above is something that is about as close to what I do but it changes when the need arises.  That’s also another key feature of being agile – adapt or die in the waterfall.

So, I posed the question “How far away are you from executing your requirements?”. If you are going to go through various steps and stages to get to compilation and then execution, then every step is an opportunity for valuable information being lost in translation. If you can compile your requirements immediately then nothing will be lost.

Each additional step between requirements description and compilation and execution is an opportunity to confuse the user and the developer and everyone in between.  That’s why fully dressed use cases are not so effective as fully dressed behavior driven stories.  And that’s why BDD is very agile and a great asset in DDD and use cases just don’t cut it anymore.

Right now, my favorite tool is Cucumber.  I can execute the requirements and that raises the clarity ranking of my requirements super high.

Then I started thinking about what it takes to coach external DSLs.  I got really scared – parsers, generators, abstract syntax trees.  Those were things that most people wanted to forget about the day after they wrote the exam for that horrible semester course in university.  What makes it worse is that I never did that in university – I did Electronic Engineering and we spent our coding time figuring out how to do Fast Fourier Transforms on digital signals.

But when I look at the progress that is being made in language workbenches to help us create DSLs, then I reckon we should dust off those books and start paying attention again.  Some of these language workbenches still leverage a host language such as Java or C#, but the act of using a structural editor that edits the AST directly is strangely weird.  But, some refactorings are just not a problem anymore.  For example, if you are changing the node on a tree, then all the references to that node are automatically aware of it.  Compare that to a text editor where you need refactoring wizardry in the tools to make sure all references are updated neatly.

I think that language workbenches may be a great tool for coaching DSLs because the things that made your head spin during that lex and yacc week, are made a whole lot simpler.  You can focus on designing the language and walk into ASTs with less fear.  But, be warned — language design is not easy, and you still need to know about ASTs and parsers and generators.  In fact, just write a parser and a generator and it will be a learning experience that goes beyond DSLs.  Better still, do it BDD/TDD style.

The relevancy of language oriented programming is just going to continually increase.  Those previously “irrelevant” courses are important again.  Watch out, the gap just got wider.