Making a mess less

All my models start messy.  Yet, I find that my messy models are actually quite useful.  It forces me to acknowledge my ignorance.  The longer I feign ignorance, the longer it takes for the mess to surface.  The more I appeal to my ego with “Hey, look!  My model is very cool!”, the longer I delay learning something useful.

I caught myself saying this the other day and it made me think about how I deal with a mess.  For me, a messy problem is one that has more than 2 inputs and outputs.  My brain struggles to resolve that.  I just can’t do NxM matrices in my head.

Making a mess less

The first thing I do is accept the mess  and then I play with these 3 techniques.  There are others but I find that I lean on these early in the exploration.  I also don’t have a recipe or process of which technique I use when.  I simply try something that looks interesting to me, or the focus on a part that is most obscure.

Find the range

I pick just one visible dimension in problem – a single input or output.  I play a seemingly bizarre games of extremes with that variable, and adjust the extremes of that variable towards what feels “normal”.  I want to find the range of usefulness.

Say I want to model the time it should take to open a bank account. I will ask “How about 1 minute?”, and then How about 1 month?” I play with the answers and adjust so that it sort of makes sense that it is somewhere between a few hours and a day.  Now I know that anything outside of that time range is irrelevant.  I can shift focus on other variables – progress!

Setting the range gives me a feedback loop.  When I play with other variables, I can see if they shift values outside the useful range.  Sometimes, the range is consistently being violated by other variables.  In such a case, I just discard what I thought was the useful range, and play elsewhere in the mess.  The true range will reveal itself eventually.

Find reactive pairs

I look for pairs of variables where the change in one results in a change in another.  Here, I am trying to understand moving parts, or cause and effect. Certainly, changing one may result in several other changes but I am interested in just one pair. Sometimes I just guess. Other times I just get lucky. The most interesting are the ones that take me by surprise, accidentally.

For example, I want to figure out how best to process a very long queue of insurance claims. I may start with the sort order for the queue. If I set sort order by descending monetary value then we may find that the assessment process requires different steps, different processing time. And so I learn that the sort order results in different claim assessment processes. This may then trigger a thought around batching claims by classifying incoming claims and look at size of the resulting batches.

I find that the most useful pairs are the ones that oppose each other.  These reveal a natural tension and offer a powerful starting point for finding a solution.  The solution lies in answering the question “How do I release or control that tension?”.  I am aware, though, that finding the solution will create tension elsewhere.  That’s design – focus energy on the part that has the highest yield and live with the rest.

Make linear systems

These messy problems have many inputs and outputs making it difficult to model. So, for the variables that are known to me, I fix them to sensible defaults. Then I play with just one variable. In essence, I make the system a first order system – just one input and one output. Such models are easier to handle, even in thought experiments.  Once I understand how that “linear version” of the system works, I have more options available. I can choose to fix this variable to a constant and make another “linear version” with another variable.  I can also opt to finding its range or looking for its (reactive) pair.

Using the techniques

It’s a great way to collaborate

When working in a group, each person has a different view and a different degree of knowledge about the problem.  Using these techniques with a whiteboard, or sitting in a circle and talking through thought experiments, forces conversation to small meaningful parts and drives convergence quite naturally.  Group ignorance is revealed safely, and shared understanding is the outcome.

Validate the mess early

Because the less messy parts that are actually useful are revealed.  Never forget, though, that it is still a messy model and needs validation.  Find a way to test these less messy parts.  You may model it in a spreadsheet, throw data at it, write some code.  Pause, find the cheapest way to validate, and come back for more.  Remember that it is a mess and there is still a lot to do.

Focus on learning, not the backlog

It is easy to get carried away and drift focus to building a backlog of work as the primary goal.  A backlog derived from a mess is not valuable.  The objective is to learn more, not build more.  The more we learn, the greater the chance that we will build the most useful aspects.  This is not about ignoring deadlines.  Instead, we are aiming for highest impact in each time box.  That impact comes from making the mess less.  Validating early, collaborating consistently helps keep focus.

Closing thought

The mess creates a virtuous cycle when I allow myself to acknowledge the mess, find use and value in the mess, validate the less messy useful parts early. I keep forward momentum with awareness of my ignorance.  My messy models are still a lot better than no model.

Update on Grokking Functional Programming

Writing this book has been quite a journey.  Being a personal experiment in user experience design in a non-interactive medium has proven to be quite a challenge.  The constraints are significant from page size, typography and then to consider cognitive overload and leaps from concept to concept.  Yet, at the core I am still aiming for a pairing experience.  I want to create an experience that someone is working with me at the same computer and sheets of paper.

This is what is emerged as a workflow.

  • sketch out a plan for the chapter;
  • focus on what are the main takeaway points;
  • what pre-requisite knowledge is needed for the chapter; and
  • fill in some examples and exercise, pretty much faux-code

The gap that surfaced this week with my editors is that we swirl around the main takeaway points a lot.  The fundamental question that I now ask is “What is the one super power that we receive with this chapter?”.  That’s a hard question to answer and it forces me to ask very abstract and pointed questions about the main concept of the chapter.  Often, the answer is philosophical which forces me to reduce it to something practical.

Sometimes, answering the super power question has left me admitting “Well, that’s pretty unremarkable!”.  Initially, I was disappointed when that happened.  Now, I use that as a feedback to dig further.  It forces me to seek for my own deeper understanding.  The bottom line is that I am still hindered by my own ignorance.  It takes effort to break through ignorance barriers and it is not about pounding on the same door all the time.  It is about circling around and attempting to find tiny cracks to chip into.

In addition, I have now realised is that UX can only be solved after the philosophical and abstract are reduced to concrete.  Having tight UI constraints is actually a blessing for UX.  I now appreciate the limitations as it simply removes options not worth their bandwidth consumption.

Grokking FP JourneyWhere have I taken people on this journey so far?  On the right is what I’ve covered so far.  Next up are algebraic data types and then more in depth coverage of high order functions.  Very surprising for me is that this journey of constraints, user experience challenges, asking super power questions has led me to shifting this chapter on high order functions from being chapter 2 to chapter 7.  That was unexpected.

Keep an eye out for tweets on discounts on the book.  The latest promo from Manning is for 30 October 2014.  Using discount code dotd103014au at http://manning.com/khan will give you 50% off the list price.

 

A Book is Born

Grokking Functional Programming

Over the last year I have been spending some of my time writing a book on functional programming.  Finally, it is available on Manning’s early access program.  The first three chapters have been published and my commitment is to produce new content every month or sooner.

This has got to be one the most difficult pieces of work that I have ever attempted.  In many ways, this is an experimental book.  It takes my brain on two orthogonal paths.  The first is about the topic of functional programming (FP).  And the second is about cognitive learning.  FP is a difficult subject and large too.  I’ve attempted learning it several times and threw it to one side in frustration.  So this book is my crazy attempt to apply a few aspects of learning to a complex topic.  My hope is that it is an accessible book for most software developers.

Early and gracious thanks must go to the team at Manning for putting their trust in me on this experimental journey.  And to say that my editor has been patient is a gross understatement.

If you are going to read this book, please do provide feedback on the book’s forum.  I really want to know where you got confused, which sections are too fast paced, which pages made you take ridiculously giant mental leaps.

Update: Anybody can get 50% off the book by using the discount code mlkhan at manning.com/khan/.

Event: Inclusivity and Innovation in S.African Software

I am facilitating a panel discussion as one of Cape Town’s World Design Capital 2014 events, in collaboration with the IEEE.  Attendance is free.
This is the brief that I have given to the panelists.
Software development in South Africa is still a closed society.  It is also a consumer society.  It is closed because we are not inclusive nor embracing of our cultural diversity.  It is a consumer society because we are net importers of technology, fed by global technology companies motivated by profit in a third world market.  This is the lifestyle of software development – import ideas, import technology, and restricted participation. A desired future state is a lifestyle of software developers where we are net exporters of ideas, exporters technology and encouraging inclusive participation.  In this session, we want to explore why this future state is valuable, and what we need to do to achieve this.
And the panelists are …
  • Philip Copeman of Pink Software.  He is the custodian of TurboCash, most probably South Africa’s most successful open source software
  • Brain Leke of ThoughtWorks.  Brain is a long time champion of equality of demographics in African software communities.
  • Lorraine Steyn of local startup I’m Bored.  Lor has been in the trenches of helping young developers kick start their careers for over 25 years.

When, where …

  • Date: May 20, 2014
  • Time: 12.30PM to 1.30PM.  It’s deliberately being held over lunch, and it’s a free attendance.
  • Where: 3rd Floor, Bandwidth Barn, Woodstock Exchange
  • Address: 66-68 Albert Road, Woodstock, Cape Town (here’s the map)
  • Cost: Free

Why are we doing this?

… because it is an important for the sustainability of software development as a thriving sector, and this spurs further discussion and action too; and to a much wider audience.

… and because we should care.

Freedom day and those crazy bald heads

Today is April 27, 2014.  It is 20 years and a few days since I voted for the ANC as a free and equal citizen of South Africa.  I never thought that I would live free of apartheid for almost half my life.  Some days more cherished than others, but always, I will take this freedom over apartheid.

In recent times, the leadership of the ANC, especially our State President Jacob Zuma, has been disappointing.  And this word – disappointing – is an understatement.   We cannot pause to feel sorry for ourselves, disillusioned.  The ANC has lost my vote. It is time for another struggle.  Vote with your head and your heart, but not out of sentiment and nostalgia.

Back then, Bob Marley protested against colonialism.  It’s ironic that these same words apply perfectly to the current ANC leadership. It is time to chase these crazy bald heads out of town.

Them crazy, them crazy
We gonna chase those crazy
Baldheads out of town
Chase those crazy baldheads
Out of town

I and I build a cabin
I and I plant the corn
Didn’t my people before me
Slave for this country
Now you look me with a scorn
Then you eat up all my corn

We gonna chase those crazy baldheads
Chase them crazy
Chase those crazy baldheads out of town

Build your penitentiary, we build your schools
Brainwash education to make us the fools
Hate is your reward for our love
Telling us of your God above
We gonna chase those crazy

Chase those crazy bunkheads
Chase those crazy baldheads out of town

Here comes the conman
Coming with his con plan
We won’t take no bribe, we got to stay alive
We gonna chase those crazy

Chase those crazy baldheads
Chase those crazy baldheads out of town

— Bob Marley, Crazy Baldheads

James the Shoe Shiner

A few weeks back I was humbled by about 100 graduates.  They attended an event called Level Up hosted by ThoughtWorks in Johannesburg.  I was speaking last and everything that I wanted to say was said by the time I stood up.  So, I went impromptu and spoke about why we change jobs, that a focus on design will see you through in the world of software, and that life is a cycle and we will do well to do the same as is being done to us right now to start our careers.

Then I rushed off to the airport, and the content of my improv talk hit me fully with stark reality.  When the airport is quiet, like it was that Saturday evening, I know that business is bad for the shoe shiners.  Here was James, the only shoe-shiner around.  It was 6.30pm.  He finishes work at 7pm.  I asked him how much short was he of his daily quota.  It was a slow day, he said.  I opted for an expensive shoe shine so that James met his quota.  I know that if James didn’t meet his quota, it is taken off his weekly wages.

James, it turns out, is from the Eastern Cape.  He graduated last year with a Diploma in Electrical Engineering.  He came to Johannesburg to get a job.  It is a tough job market, but he was finally offered a job.  But he couldn’t take it because the job required him to have a driver’s license.  Now he shines shoes to save money to take driving lessons, then he enters the job market again.  That job won’t wait for him.

My talk from half hour before of why we change jobs hit me hard.  In John’s mind, the only thing he needed was an education, and everything will work out.  Then he discovers these hidden constraints.  He changes jobs – temporarily, he hopes.

This moment has lingered with me for weeks and it clarified three things only today.

  • Indignity is imposed, humility is a choice.  To make a person kneel before another to polish shoes is an indignity.  That a person has chosen to polish shoes as the needs to an end is humility.  The work we do is not what defines us.  We define our work.  I have heard several agile coaches telling people that if they are unhappy, then they should change their company or change their company.  Most have changed jobs feeling an indignity that was imposed upon them.  I wonder, though, whether life would have been better when that same “horrible” job was embraced with humility.  I know sometimes we have to flee.  In my life, when I fled too quickly, the real issue was never resolved.  It just surfaced some time later.
  • Living in the promise of the future or the longing for the past is to live in limbo.  The past ends in this moment, and the future starts in this moment too.  We live in the present.  That is part of being truly agile.  Just changing jobs because of something in the past or the lure of the future somewhere else is to lose what we have in the moment.  It is not enough, though, to live in the moment.  To be in the moment demands an awareness.  Adjusting a course based on awareness beats reacting to stimulus.  Reacting to a stimulus puts us into a fight or flight choice.  Adjusting with awareness keeps us in a contemplative place.  Agility is about continuous transitions, each so tiny, that we can’t see the jagged edges over time.
  • Unexpected constraints take us into uncharted places.  James needing a  driver’s license is his unknown constraint.  Now he shines shoes.  We don’t know who he might meet shining shoes that may take his life on a different path.  Similarly, the truly agile team will embrace an unexpected constraint and look for opportunity in the uncharted beyond.  The process religious agile team will try to reject the unexpected constraint, fearful of the great beyond – like anti-bodies fighting a virus.  To be truly agile, we need to just embrace the unexpected as it pops up.  I see many teams attempting agile software development by just tweaking settings on known constraints.  That works for a short time only.  Our  software, after all, is only one requirement away from being hopelessly invalidated.

One shoe at a time, James’ life is changing.  James is agile.  Working with humility, embracing the unexpected constraint of needing a driver’s license, living in the moment, being aware of an empty airport and short of his daily quota.  That is how we should build software – living in the moment with awareness, letting it change our life daily.

I believe this is possible.

Many teams co-operate, not collaborate

At Agile Africa 2013, I presented a session about lessons learned living through apartheid to democracy and how this manifest itself in teams in which I found myself during those times.  Each period has many little stories, all relevant to the way we behave when we work together, especially in quite diverse teams.  I’m not going to focus on these stories, but rather on the main theme of that talk.  The slide deck for this session is available here.

The first time I spoke about this publicly was in 2010.  It’s taken me a while to articulate this particular aspect.  This is the first time I’ve written about it and I want to share that.  It is my take on a concept that I find is handed out too easily in agile software teams – collaboration.  I have chosen specific words to describe different kinds of relationships.  My usage might differ from their dictionary meanings.

Now let us start with the first period – the days of apartheid.

ContemptContempt

In the apartheid years of S.Africa we had a distinct “us and them” political and social structure.  It was whites holding a superior position over blacks.  It was only beneficial to one – those that held the superior position.

I call this kind of relationship contempt.  There is no trust between people and no shared values too.  The only value system that matters is that of superior.  This is a case of oppression.  It is the oppression of another’s values and identity and of freedom of choice.  Hence, we feel imprisoned.  The consequence of this is the under development of the oppressed.

When have contempt, there is no chance of working for mutual benefit in any kind of way.  It is just enslavement.  In software teams, I’ve experienced this overtly, without any facade or subtleties.  Some have been extreme cases, reflecting the bitterness of racism entrenched in our bigger society.  More common, though, are subtle situations of contempt.  For example, a developer considers another developer as less skilled – stupid to be precise – then we have relationship of contempt.  Architects and developers, managers and team – these are instances of “us and them”.  We can have such titles, but it is the attitude that arises when we build a culture where one holds themselves superior to others.

Co-operationCo-operation

In the early 1990’s South Africa entered a period of transition.  We were dismantling apartheid and building a new political and social system that ensured that we never go back to subhuman bondage.  This required enemies to find a way to work together for the survival of both.

I call this a co-operative relationship.  It is when both sides still retain their own values, but find a way to fulfil a common objective.  To achieve this common objective requires them to build trust.  I imagine trust as the payload carried in every action.

I sometimes explain it like this.  If we were renovating a kitchen, then the brick layer, the plumber and painter need to co-operate in order to complete the kitchen on time.  Each of them have their own work ethic, which is largely tucked away and insulated from the rest.  They need to agree on certain things like the bricklayer will complete the kitchen before the plumber can put in the taps, and the carpenter waits for the plumber before building the cupboards.  Should one not honour their commitment, then trust is destroyed because it affects the integrity of the others as seen through the eyes of the home owner.

This is the most common type of relationship that the majority of software teams practice.  We only need a co-operative relationship to build a reasonably efficient cross-functional team. Unlike contempt, people are no longer treated as inferiors.  Instead, everyone stands on their own pedestals of similar height.  The only thing that binds the team is the trust that comes from executing specific actions to achieve the common goal.

Most people call this collaboration, but I call it co-operation.  I can understand why we consider this collaboration.  It is the degree of trust that is necessary for this to emerge.  It is also reinforced by giving a team a name as a sense of common identity.  For me, that kind of naming leads to tribalism.  We gather to paint our faces in familiar markings, exhibiting the same brand.  Our behaviour becomes one of exclusivity than inclusivity.  It reminds me of the tribes that formed in the somewhat controversial book Lord of the Flies.  Pretty soon, we have intertribal tension.  This actually pops up quite quickly in multi-team Scrum implementations where all teams are working of a common backlog.  That backlog introduces real or invented dependencies between tribes, and so the tension starts emerging; just a shadow of first but easy enough to envelope all in darkness.

So, what then, do I consider collaboration?

collaborationCollaboration

After South Africa ushered in a new constitution, and successfully held its first democratic elections, the cracks started to appear.  We heard words such as “we need nation building”, “we are a rainbow nation”, and more recently “everyone can lead S.A.”.  All these words refer to one thing – we need to share a common identity.  That is what collaboration is for me – we have trust and now we focusing to share a set of values that exposes a powerful new identity.  It does not mean that we ignore or assume the presence of trust.  Our actions still carry a high payload of trust.  That never stops.

Building a shared value system is not easy, nor quick.  It is a challenge because each of us in this collaborative relationship have our own hierarchy of values.  Now we are being challenged to reconfigure personal hierarchies to overlap.  When we identify those intersections, then we are standing on the same pedestal.

When we work of a common platform – of even just one common value – then our basis for existence changes.  We stop becoming exclusive and starting behaving inclusively.  When a “stranger” walks in, seeking participation, then we look for that tiny tendril that allows them to plug into our shared values. Over time that tiny tendril grows stronger, and deep roots are planted.  Sometime the new person enlightens us and we expand our value system too.

What happens if that new person does not have that thin tendril that we can plug in?  This maybe a meta-value.  The one value that is necessary for collaboration to take root is that we respect all values; being prepared to explore the adoption of any value.  If we don’t have this single meta-value then we can’t bootstrap collaboration.  Without this meta-value we are unable to explore any value.  Collaboration comes to a halt.

The good news is that the each person does not have to have deep roots into the value system.  Tapping in lightly is enough.  The depth of the roots that are grown is tied to each person on that platform.  Those with shallow roots will move on to other teams that allows them to grow a deep root.  The period of time when they planted that root so shallow is still a wonderful time because we shared an identity.  That we cannot take away.  Inclusion is a way of growing.

When people anchor themselves deeply into the value system, we will find that those teams have a longer life. This is because we have a pathway to succession.  Having strong roots means that we create sufficient time for others to grow strong roots too.  In this collaboration, our sense of tribe is meaningless because the values will nourish and support generations of people.  Having a name becomes a label for the values that define us.

A closing thought agile adoption

I’ve tagged this with XP (extreme programming) as well as agile, and ubuntu – the zulu philosophy.  XP is what made me aware of the power of values in a team.  In my ignorance, I often attempted to instil values very early in a team.   I thought that I needed to build from a solid foundation.  It never worked.  I found that building a collaborative team is inverted.  We build the walls and the roof before we build the foundation.  The walls and the roof is trust that is needed, and the foundation is the values that are nurtured.

A collaborative team, as I describe it, is rare. It may take years to establish that foundation, with lots of corrections along the way.  That is ok.  That is the journey.  Enjoy it for exactly that.

Extreme Programming is as relevant now as before

Extreme ProgrammingLast year I met a few people that had never experienced a waterfall project.  Their first software project was with Scrum.   What stood out for me was that these developers had not read Kent Beck’s Extreme Programming.  I also discovered the same for several that were years into their agile development journey – never read the XP book.

Perhaps we are past the sceptical stage, and the notion of working in 2 to 3 week cycles is not considered so extreme any more.  XP may not be as popular as Scrum and Kanban at the moment, but it is still intrinsic to successful agile software development.

I confess that I find a lot of the XP practices are difficult and I have certainly not mastered any.  I think it might be simply that software development is just difficult.  Another reason may be that XP warrants that I change my approach, shifting my attitude to software development quite fundamentally.  That is the extreme part of XP for me, not the 3 week cadence.

Here are a few aspects of XP that I have grown to appreciate over time.

XP makes being a decent human explicit

XP has a lot to offer for the welfare of each person, and for our collective welfare too.  In a software world where we are learning to embrace our diversity, these insights are valuable for nurturing what is still a comparatively young sector.  Certainly, XP offers values.  However, just declaring a set of values does not mean that we live our values.  That’s where XP’s principles and practices kick in.  These exist to nurture those values.  Having a shared set of values and principles is essential in a team, because we use these to help us eliminate options and make decisions.  Value based decisions always seem like choosing the harder option, but the payback is often greater.

XP is about fine grained feedback

XP introduced me to the idea of feedback and made me aware of the theory of constraints – that we just shift problems around when we introduce or remove constraints.  Now we see the rise of lean which focuses on one piece flow, not short time cycles.  That is a different constraint and we have new feedback points like pivoting and waiting until  the last responsible moment.  All these require explicitly designed feedback paths.

Producing a piece of software with one piece flow means that we must bake the feedback into the software we produce.  For example, if we redesign the landing page for a web site, then we need to bake in measuring page views and time spent on the page as part of the release.  Without explicitly thinking about how we receive feedback, we just end up producing a chunk of software that has very little value.  Once we have feedback loops baked in, we are back to observing the shifting of constraints.

XP practices such as continuous integration, collective code ownership, simplifying vigorously, small and frequent releases all aid in building precise feedback loops.  These practices directly support one-piece flow, devops, and continuous delivery.  Yet, most people are unaware that some of these practices are there to increase collaboration.  For example, when there is a problem in the build pipeline, then people will swarm to fix the problem.  It is the high signal conversation after the failure that is valuable.  XP helps increase the signal to noise ratio in our feedback  loops.

XP is not just process centric

As a developer, I am yet to find a set of practices that appeal directly to my work at the most basic level – working with code.  I’ve noticed that many teams that embark on an agile journey tend to focus more on an agile process, than their code base. After a while their code base becomes their biggest bottleneck.  Not obvious is that the potential to regress to a waterfall-like development process is high.  When we have little control over the code base, then we may well reach for more grand planning upfront, less frequent and larger releases, and longer testing cycles.

The code base becoming a bottleneck is not an event that suddenly happens.  It is an accumulative effect.  It sneaks up on us.  Some symptoms are a drop in throughput; it becomes more painful to build, release and deploy; and introducing a new feature needs design contortions.  I’ve referred to this before as “trying to sprint in mud”.   We expend a lot of energy, without going far.

The consequences of a code base out of control is the real kicker.  The code we write affects everyone, not just developers and our customers.  It introduces stress to everyone associated with the software – analysts, managers, customers, family and friends too.  That’s why in XP there are practices that help us keep the code malleable, interwoven with practices that deal with our collective welfare.

With a code base out of control, we can forget  about one piece flow, lean style.  The same goes for continuous delivery and devops too.  Even with the best build pipeline tools, a badly managed code base will not make life easier.  It is like using really bad quality ingredients to make an exquisite meal.

XP keeps a focus on design all the time

XP introduced us to test driven development and continuous refactoring to keep our code bases under control.  Some people use test coverage as measure of that.  I don’t subscribe to coverage as a measure of control.  I prefer weighing up the cost of defects against cost of releasing late.  For example, if getting a feature out means securing a market position, then I would go for lower coverage because releasing late will hurt more than releasing early with defects.  This might be a controversial statement but it is still a viable feedback option.

By the way, it is worth noting that TDD does not yield good designs.  Instead, it raises warnings of problems with our design.  Good design comes from another XP practice of always aiming for simple designs.  Simplicity comes when we gain deep insights into the domain.  Acquiring knowledge takes time – it is a slower feedback loop.  Consequently, achieving a simpler design occurs slower than churning out software at the end of each cycle.

XP allows for code first agile adoption

I prefer to emphasise getting the code base under control as my main target in an agile adoption.  I lean on XP heavily for achieving this goal.  Along the way, the people and process aspects slot in quite naturally.  As the code base is brought under control, it creates breathing room to introduce other non-technical practices.  I’ve found that it is less painful to focus on the code base first, XP style, and slot in other practices along the way.

When I’ve tried it the other way – process before code – I’ve experienced a kind of agility that is only skin deep.  The reason is that the act of adopting the code centric practices requires significant change in developers.  Until developers embrace changes in themselves, in the way they approach coding, the agile adoption will remain superficial.  By “superficial” I mean that it does not penetrate down to the code itself.

Superficial agile is not a bad thing.  I know of some organisations that will benefit from this approach, but only for a while.  For some, that is sufficient.  If we really want to go beyond skin deep agility, then I think it is necessary to reach for XP’s practices that deal with code.  There really is nothing else that deals with writing code in short cycles or one piece.

Should you read Extreme Programming?

My answer is a resounding yes.  It is a wonderful book, written with a lightness and easiness.  Yet is dense with deep insights in every sentence.  The one last aspect of XP that I cherish is that I could ease into to it slowly and deliberately.  I did not have to apply the entire body of work instantaneously, in one huge swoop.  And I am still easing into it.

We are better than this

The people whose contributions to a public Twitter conversation that I mentioned in my last blog post were offended by my opinion on themes that we touched upon.  I’ve apologised over Twitter that my intention was not to be personal, negative nor offensive.  I’ve read my blog post again, taken in the feedback over Twitter and from Kevin’s blog.  I believe that I could have given better context by acknowledging the value of the code retreat and Kevin’s efforts in the community.  Also, I could have expressed my initial tweet differently or at a different time.  That it distracted Kevin from his code retreat was insensitive of me.

Events like Kevin’s code retreats, Black Girls Code, and other initiatives are important to improving our community and software development in South Africa, in general.  Yet, we are still facing glaring disparities in our sector and in our country that I find difficult to ignore.  My fear is if we all take defensive positions, we may never ever discuss such matters openly and respectfully.  Like I said in my twitter conversation to Kevin, this is not about him and not about me too.  It is about us and our future.

The fight for equality is so much tougher when we have freedom.

A question of demographics and then some

Update: Kevin Trethewey has extended this conversation on his blog.

I commented on a photograph that Kevin Trethewey shared of the code retreat that was hosted in Johannesburg, South Africa recently.  His response was very defensive and it clearly showed that he was taking my observation personally.  That was something that I didn’t anticipate, as I have not seen Kevin carry prejudices of other race groups.

What followed was quite an interesting conversation on the disproportionate representation of non-whites in the South African software community.  I’m going to comment on parts of this conversation.  For the full conversation, go here.

A point to note here is how quickly people focused on racial composition as opposed to gender composition of the group.  That is a problem in itself, even for the most gender-aware man in a software community.

This is a long, long, long post, mostly because it is a brain dump of the themes that were flowing in this conversation.


There  is adequate opportunity and access for everyone

Kevin believes that there is fair access and opportunity for all of us to attend an event such as the code retreat.  I disagree. For example, I know a few black software developers that don’t live in a conveniently accessible middle class suburb.  For them, getting to work is a challenge.  To get to a community event outside of the peak public transport service hours is a huge issue.  The same applies to people attending university.  A black kid living in a township with minimal to zero essential services loses at least 3 hours of study time a day when compared to her more privileged peers.  Accumulate that over a month and we find that she loses about 6 days of work a month.  Our lives are not equal in far more fundamental ways than we wish to acknowledge.

It is also a matter of survival, and not that people lack ambition to be better developers.  When we have less than enough of everything that is essential for living, then a meagre salary is a big deal.  Many black developers I know are supporting their entire family, if not an extended family.  When we’re in survival mode, ambition is lower priority.

I would love to see events hosted closer to the people that have greater logistical challenges.  That’s why Braamfontein was a good venue for Agile Africa.  Maybe in 2014, we will have a software conference in Soweto for the first time.  When I want to help someone out of a dark place, I prefer to go into the dark and walk out with them, taking the bumps together, than to just stand outside and shine a light in the distance to where they should go.


People are apathetic

Simon Stewart was of the opinion that it is a lack of caring – an apathy.  I don’t know which group of people Simon refers to here so I will steer clear of making assumptions.  What I have noticed is that the majority of software developers don’t care much for self-learning.  The code retreat that sparked this discussion is a case in point.  Twenty seven developers pitch up to practice their TDD skills.  I don’t know how many developers there are in Johannesburg, but I guess that this is less than 1% of the total.  Regardless, I don’t have a problem with this apathy.  Some people “give a shit” about different things, like earning a salary first.  Because they care less for self-learning, does not mean they don’t care at all.  It just means that their value system is stacked differently from others.

I don’t believe it is about software craftsmanship either.  I think software craftsmanship as a community appeals to people that share that value system and the way that community engages with others.  I’m not one for the craftsmanship movement for reasons that are irrelevant here.  However, I do respect that this community has a right to exist alongside all the others.  Not caring about software craftsmanship does not mean that a developer does not care about growing.  Some of the most accomplished, self-taught developers I know care far less for craftsmanship than I do.  We need to help people understand their potential, and provide diverse ways in which they can fulfil their potential.  Sometimes people will break through that ceiling, setting up a new potential to fulfil.


Lack of diversity is commonplace

Simon rightfully points out that the lack of diversity is common.  I concur.  I’ve spoken at a few conferences outside our continent and it is the same in most places, as much as it is within ours.  Again, that it is so does not mean that we should accept this.  Make up your own mind, but I cannot accept that.  I’ve been talking about diversity publicly for several years now.  I’ve been on projects where the direct result of us not working with our diversity resulted in a tragic cost of the human spirit.  I cannot accept that we are prepared to allow that to happen again and again and again.

I try to be a lot more aware, a lot more empathetic.  I’ve been fighting my own unsettling biases to appreciate our diversity, to celebrate our differences and embrace our uniqueness.  The scary part is this very process of appreciation demands that we drop all our prejudices.  To do that requires self-reflection and clarity to face the truths of our deeply internalised, and camouflaged stereotypes.  Be prepared to cry for ourselves.  Be prepared to feel lighter on the other side of that.


Quotas and Meritocracy

Steven asked what he thought was a silly question.  But it is not silly at all – “How should a code retreat represent the demographics of the country?”  For me it comes down to us building a significant software development sector in South Africa.  The white minority cannot satisfy that agenda alone because there aren’t enough of them.  We need black software developers too.  Parallel to this, we need fair and equal gender representation too.  South Africa has one of the most humane constitutions in the world.  Yet, I see many software developers adopt the non-inclusive behaviour of frat boys, country clubs and colonial style thinking of the west and north.  That is a learned behaviour that I refuse subscribe to.  Given our history of the inhumanity of apartheid, I find this behaviour equally abhorrent.  Incidentally, I was part of a very diverse group of developers that were reflecting on what we don’t like of the software development community, locally and globally.  What emerged were distinct pointers to this imported, inhumane subculture: elitism, arrogance, fear, bullying, sexism, racism, monopolies, nepotism and a lot more.

Steven also dangled the option of quotas.  South Africans are masters of quotas.  We’ve tried that in almost everything from sports to business.  It turns out that Steven is not offering quotas as an option.  What is relevant for me is the other side of the quota argument – inclusion by merit.  Meritocracy is a subtle form of exclusion.  In Africa, where we have significant disparity on multiple levels.  Meritocracy just widens the gap between those that have excelled and those that may excel but don’t have opportunity to do so.

I’ve been guilty of exercising meritocracy to the extent of building a company based on inclusion by merit.  Being elitist is one of the easiest ways of being exclusive, and oh so easy to cover up whilst staring in the face of everyone.  What changed for me was a change in my attitude.  I now carry an attitude where I respect the potential of each individual.  I realised that we need people to contribute in their own little way, acknowledge that contribution and then help that person raise their own ceiling.


We must challenge the bias of minorities

Here is Steven’s alternative to quotas.  I ask that we not flame him or attack his position.  This is his position and I want us to respect that.  If I want to change Steven’s mind, then I need to offer him alternatives, not antagonise him.

I agree that tech events are open to everyone.  I don’t accept that non-whites and women – the “minorities” – don’t attend because they hold prejudices against the pale males.  I think the problem is to do with accessibility, opportunity and a necessity for survival prioritised higher than self-learning, as discussed above.  By the way, Steven, as a white male, you are the minority.

Yet, there is a strong element of intimidation.  This, we cannot ignore.  I’ve been on the receiving side of intimidation.  It is subtle and, strangely, more powerful in its exclusionary effect.  It exhibits itself as quiet arrogance, and other times as open boorishness.  Tech conferences are testosterone events where geeks flex their muscles using absurd measurements of superiority like “my big data is bigger than your big data” or “I’ve got more pull requests accepted on blah-blah-blah repo than you”.  All it takes is one moment of acknowledgement, a moment where there is a little light shining on us and people smile and say that we are awesome.  It is that moment when we pack the soap box in our suitcase and take it everywhere.  From that moment, the superior position is taken and others struggle to fit in, white or not, male or not.  Tech events are open for those of us with soap boxes. Oh, and you must make sure your soap  box is bigger than other soap boxes.

This is why I promised myself to always step off the podium and be accessible to everyone in the room and beyond.  This topic of soapboxes, self appointed popes, cardinals and priests of industry, blogging, tweeting, etc deserves deeper discussion more than this already far too long blog post.  And, yes, I know that my irony is that I’m blogging and tweeting too.


Tell me when you’re done talking

I was uncertain by what Kevin meant when he asked me to let him know when I’m done talking.  It would have been easy for me to fire a flame attack along the lines of “WTF makes you think I’m all talk and no action?”.  That’s not useful because twitter is a horrible medium that draws assumptions out of people.  So, I just asked for clarity first.  This is important; a behaviour that I try to instil in myself all the time – pause, listen, think and ask for clarification if I have even the slightest doubt.  That double check saves me a lot of unnecessary flaming.  I also found that asking for clarification in the face of cynicism does defuse the situation. Then again, rushing around with a burning flamethrower, defending every single one my opinions is a horrible way to spend my life.  It doesn’t take me forward in the least bit.  Worse, still, is that it causes me to stagnate.

I do try my best to not draw public attention to the work that I do.  I respect the privacy of being in a group and have learnt a long time ago that it is self-serving to tweet about such work.  As a mentor to a few people for many years, I respect that unspoken code of confidentiality.  Being a mentor, for me, is a two way street.  I need to be equally open and honest as I expect of the other.  Sometimes, that means being quite personal and deeply private.  So, I don’t crow about my work.  But I blog and open conversations based on observations which I’ve extracted from my time with others.

Those that work closely with me know the extents to which I go to change the status quo.  Then there are those that are even closer that know the extent to which it pains me when I can’t change the status quo.  At a deeply philosophical level, is it possible to give unconditionally – without even the tiniest drop of self-serving satisfaction?


Celebrate the success

Mark Pearl, on a separate conversation stream, reminded me that that progress was being made.  That is true, there is progress being made.  I mentioned that at Agile Africa that the demographics are changing, slowly for the better.  I think we are moving forward and I am not so concerned about the pace either.  There is a part of me, though, that is cautious of grand celebrations for small steps.  That is just me.  I prefer constant acknowledgment of progress for each tiny step.  It keeps me humble, knowing that the big picture is still not fully painted.


Can I plug in, please?

Hearing that I am doing something, Kevin invited me to breakfast to hear more so that he can plug in.  I’m happy to share with most people, but this work isn’t about plugging in.  It is not a thing at all.  It is about my individual behaviour and attitude.  There are no recipes that I can offer to make this better.  It requires me to find a connection with myself.  It is about holding a heightened sense of self-awareness, of each thought, each action and each consequence.  It demands a level of honesty with myself that is, at times, extremely hard to face.  This is a journey that I’ve been trying to walk.  It is painful to look in the mirror and see beyond the reflection.  I am motivated by a belief that we have a primal encoding for decency and an inclusive humane culture.  Everything else is an invention that is designed to obscure that basic encoding.

For more than 50% of the time, I get it wrong – my behavior and attitude.  So, I cannot stop here and I must persevere with this.  This is why demographic representation is important.  It is a measure of our collective shedding of invented biases and a statement of us including each other without prejudice.

To Kevin and the others, thanks for opening a powerful, multi-faceted conversation.  And if you want to plug-in to this inclusive, humane culture, then you will need to plug-in to yourself first.  This is not something I can help others plug in to.  I can only behave in a way that I want our software community to behave.  I believe that others will change in step.