19 June 2008

Damager or Manager?

Since Open Letters to Management seem so flavour du jour, I thought I'd save the following fine rant from oblivion. Names and project details changed to protect the guilty, of course.

DumbTribe is a small startup in the mobile space. They have started seeing some good traction for their product, but are completely chaotic in their "management" of the company. The company is 100% reliant on IT, yet, whilst they're willing to spend an ordleplex of money on fancy new offices, they're astoundingly short of cash when it comes to things like buying another server to act as failover for their single server. Said server is the sole source of income for the business.
What [ManagerX] calls a "blogger tool" is really a form of Content Management system that ends up providing (among other things) Atom and RSS feeds...

[ManagerX] wrote:

> This has certainly taken longer than we initially thought it would. I
> think it was over a few of months back that we were expecting a
> finished blogger tool.

You seem to have forgotten that there were other tasks that YOU prioritised ahead of the "blog" tool -- development of the feed aggregator and the [BigClient] pilot, not to mention system administration tasks more numerous than I can recall, design and coding assistance to my colleague, installation and maintenance of essential technical infrastructure indispensable to organised development (some of which runs on my own servers, at no additional charge to [DumbTribe], simply because that was the fastest way to get the tools in place.) I regret that I am unable to work on more than one thing at a time, but these are rather complex systems dealing with some very erratic, "dirty" data coming-in, and, like most men, I don't multitask well.

> It is of zero use in it's present state(just like the blogger tool you
> created was in it’s 'unskinned' state(to me the level of things that
> fell under 'skinning' was surprising.

First: I warned right from the start that installation of the necessary software was the quick and easy part, but that changing templates -- "skinning" as you call it -- would take at least several days for someone expert in the templating system. I also made it clear that such templating was NOT in my sphere of competence. Evidently nobody was listening to the bits they didn't want to hear.

Second: I will not take responsibility for your inability to produce a coherent specification for the tool. Lack of any technical specification underlies the several misdirections and false starts. A powerpoint does not BEGIN to form a clear technical specification.

Example: I, at one stage, asked you how many "blogs" it is necessary for the system to support. At that time I had in mind to use a particular piece of software as the foundation infrastructure. Your answer to my question was "thousands!" which answer had a significant impact on my technical decision-making, since the tentatively-chosen solution is unsuited to such large volumes. I certainly made it clear that I was unfamiliar with the more suitable tool, and, indeed, the "skinning" -- the writing of custom templates -- turned out to be more problematic than I anticipated. I made a poor guess in the face of inadequate information, a misleading business requirement and insufficient time to evaluate alternative technical solutions.

In the long run it turned out that a "blog" system is just what [DumbTribe] does NOT need. What IS needed is an article/story management system for providing Atom/RSS feed output. This is in the final stages of development.

You seem to have forgotten that using a blog-system was initially mooted merely as a temporary stopgap solution to provide a mechanism for getting article content into feeds; it was never intended to be the "real" solution. What I have been developing is such solution. Assuming you don't sabotage delivery with yet another interruption.

I would have been finished 10 days ago had [my colleague] had enough spare hours to assist me on areas where I do not have the deep knowledge of data structures and code she already has in place, and if the "specification" had not been changed on a number of occasions. Unfortunately other more pressing issues have had to take precedence on her time, with resulting delays on the "blog" project. Furthermore the assistance I have (gladly) given [said colleague] on other projects has also had the effect of taking several days from "blog" development.

> I am sure you understand what this looks like from our end. It just
> feels like you can’t give us what we need.

Yes, I am pretty sure I DO understand. It seems to me that you think one of two things; either:

1. That I am incompetent to produce working software, or

2. That I am dishonest and lie to you about my activities.

I am neither, and find either accusation hurtful, denigrating, and completely unprofessional. Software development, unlike so many other jobs, does not allow one to delude oneself about the limits of ones knowledge or abilities, so the charge of incompetence is easier for me to dismiss when I consider its source; I know exactly how good I am.

Clearly you have absolutely no clue how software development works, nor what is a "normal" pace of production for software systems. The fact that your most-recent experience of software development is exemplified by [colleague], who is prepared, for reasons I cannot comprehend, to endanger her health and wellbeing by working outrageous hours in order to meet ridiculous, unrealistic and arbitrary deadlines does not alter the truth of what I am saying. Nor is it my place to attempt to teach you how software development works; for that sort of work I charge considerably more than you pay me.

The fact that you have badly under-resourced this area of the business is hardly my fault.

> I am so frustrated and feel if I have to explain what we need again I
> will go mad.

Unfortunately software is all about the detail. If you do not tell a developer all the detail that they need, they will guess, and likely guess wrong.

Therefore, where details are lacking I will ask again and again and again. I have on occasion asked users to describe their requirement from beginning to end as many as 8 times in a single day in order to be sure I understood the requirement. Then I asked them a couple more times the next day. I am deeply sorry if my need to know what you want in full detail drives you mad -- I certainly do not wish to cause such mental anguish.

> Please can you confirm that you understand and accept all the
> functionality that we need

No. I do not believe I understand what you need, particularly as you keep changing the requirements. I am not a mind-reader, and you have not produced a comprehensive technical specification.

Example: Your comment on Monday, "Make sure we can direct the 'see original story' link to a site of our choosing (e.g. [ClientA] sites or [ClientB] sites)" This directly CONTRADICTS the requirement laid out in your powerpoint that NO such link be present. What am I supposed to do with that? I can put such a link in (though linking to what, I have no idea, nor do you say -- another missing detail) or I can leave it out (as is done at present.) I am happy to do either, or to make any changes you require, since I understand that business requirements can and do change from time to time. However, changing what is already implemented does unfortunately take some time and cannot simply be done with a wave of a magic wand.

Example: The original requirement was for articles to have a single image attached. Then an image or a URL. Then multiple images or URLS. Then it became "unusable unless we can upload video". Then we didn't need video any longer. Then we were back to one image/URL. Currently I am informed that multiple images/URLS are a non-negotiable requirement. Every time a change such as this is introduced it costs me hours or days in the attempt to comply.

And you wonder why there have been delays.

My current aim is to get the system in place with the capability to upload ONE image or attach ONE URL, either of which shall appear at the tail-end of the feed content (another detail not specified.) It is my belief that it is better to get SOMETHING up and running, even though we all agree that it is NOT the end-product desired. Then we evolve it to the state we desire. After all, that is why it is called "soft"-ware.

> and let me know what *date* we can expect a working tool to start
> testing.

In the absence of a full, clear, comprehensive specification, no such estimate can be made by anybody. In effect you are waving your hands about, saying "build me a Tudor-style house over there" and then demanding that I tell you how long it is going to take without giving me the plans for the house, specifying the building materials, size of the house, number of bedrooms, etc. When you supply me with a proper User Requirement Specification -- for which I will gladly supply a Word template outlining all the necessary information it should contain -- I may consider beginning to make estimates.

> I don’t think it is unreasonable for me to ask you to commit to a
> deadline, the brief is surely clear now after all these emails back
> and forth and the very easily accessed example of blogger.com which I
> asked you to use as a starting point.

On the contrary I think it thoroughly unreasonable to make such demands. The "brief" (whatever THAT is) is non-existent. To point to blogger.com and claim that that is what you want is ridiculous, since blogger.com is totally unsuited to your needs. If it were suitable there would have been no need to build anything else and we would not be having this conversation.

I will remind you that I am contracted to deliver 40 to 80 hours per MONTH of work -- not per week. This was deliberately and clearly negotiated up front. Consequently I do not work full 8-hour days on DumbTribe activities, which, too has its effect on delivery schedules. The fact that I consistently seem to end up working more than the agreed number of hours per month seems to be taken for granted, or alternatively is regarded by you as an attempt to rip you off. On the contrary, it is a good-faith attempt to come some way further than I strictly, am contractually obliged, in an attempt to help DumbTribe meet its goals.

A lack of planning on your part does not constitute a crisis on my part.

In the extremely unlikely event that I elect to extend/renew my contract, should DumbTribe wish to do so, please be assured that I will require a considerable tightening-up of the conditions relating to all of these issues.

13 June 2008

Otherwise Occupied

Spending my time preparing for a course -- "Introduction to Java Programming" -- for a Major Corp in CT next week. It's a course I must have presented dozens of time. Perhaps hundreds. But its been about 5 or 6 years. Actually, I did teach it once a year or two ago... I'm sure the company I did the work for were satisfied. I'm pretty sure my students were mostly happy. I wasn't. The break from teaching had left me rusty. God knows how many tiny but important details I left out by mistake. Did I talk about volatile variables?

In a 5-day "intro" course, there's not much time to fuck about. There's a certain set of Java's features that you simply have to talk about, at least in a simplified and superficial way, simply because your students are guaranteed to trip across them almost immediately they walk out of the classroom.

On the other hand -- let's be honest -- there's a limit to what the human brain can absorb in 5 days!

This time around its a little different. I'm teaching for a single company. The Program Manager's focus seems to be quite squarely on getting her people up-to-speed, and not on any other corporate ass-covering or thinly-veiled-reward-holiday-for-week sort of bullshit. Believe me, I've seen that crap all too often!

I was able to stretch the course schedule to 6 days instead of 5 -- a huge relief and I can take things at a slightly less frantic pace. I frankly don't quite know how I would fit the basics into a 5-day course anymore, given that one has to cover (at least at a shallow level) enums, annotations and generics. I guess that maybe the modules on threading and concurrency would have to fall away, but how the hell do you justify doing that when you absolutely know that your students are -- willingly or un- -- going to confronted with those as soon as you can say "Boo"?

Well, I've been working hard at getting my teach up to scratch, updating the (Day 2) OO module to include enums, and the (Day 4) module on the Collections API to teach generics, etc., etc. Of course my (development) contract customer is a little miffed that I'm not putting in 18 hour days for them... Tough Tittie -- their time is worth about 1/4 what I'll earn from teaching, and they've hardly made any effort to improve the disgusting working conditions!

I think it will be a killer course. I'm certainly aiming to make it so!

Along the way, Jason and I have been having some very entertaining discussion about the deeper, occult details of Java concurrency and system design. Something clicked last night; we both reached the conclusion that it would be fun (and probably educational for all concerned) to get more people in on that conversation, so we've revived my idea-of-many-years-standing -- a workshop/seminar format gathering, aimed at top-level, ten-years-plus-experience OO designers, architects and senior developers: A general framework to guide and channel discussion, aiming for an honest sharing experiences and learning. No fluff!

I facilitated (I would hardly dare claim "taught") one such many years ago, and, in my own humble opinion, it was the best course I ever led! I think that quite a few of the participants would agree with me. Certainly a majority of them kept in touch with me for an otherwise inexcusable number of years afterwards! After all this time, there still exists a huge gap in the professional-education space... beyond a certain level, expert practitioners (and I think it is true of any field, not just software design) have nobody to talk to -- nobody around them who is at a level where they can meaningfully push back on ideas -- nobody to brainstorm with. Or at least, far too few. The Advanced Practitioner Workshops would aim to try and (partially) fill that vacuum.

So maybe its back to teaching again for a while! It could be fun to focus on actually delivering actual deep learning for a change instead of just aiming to be Brilliant Clown For A Week!

03 June 2008

UI Design Encourages Mistakes, Boosts Profits

Having just finished my banking and tax admin for the month, I fire up the bank's online system to fork the money over to the various landsharks, fatcats and leeches.

Bank payment UITo make the various tax payments I first have to select which of my accounts to use for paying.  Then I select who I want to pay.  (For some value of "want".) The system insists that you click the "Search" button to check that you have entered the corect payee account details, but, of course, it only tells you this after you navigate away to the next page, and, when you return to the payment details form, it throws away all the details you have already captured.

I also have to enter a 19-digit reference number and, guess what, if I get it wrong (as I am likely to do with such a long number) it only tells me on the next page, and, again, throws away all the work I have done to fill in the form, forcing me to redo it from scratch.  Including forcing me to redo the payee "Search", despite the fact that my browser has captured the field details perfectly.

The real kicker is that it also ditches the account-number from which I want to make payment, substituting the "default" account (which happens to be my personal account and not the business account)  Of course, the account number is metres away up at the top of the web-page, so I don't notice that I'm making payment from the wrong account.

As a result I pay business taxes out of my personal account.  When I eventually discover my mistake, I have to transfer the money over from my business account to fix things.  And I am going to get hit with the withdrawal fees again.  To add insult to injury, the payment pushes my personal account into overdraft.  Bam!  Overdraft fees!

This UI is so wrong, in so many way, you would think that the bank  would have "inspired" and "motivated" to fix this monstrosity years ago.  Do you wonder why Standard Bank can't be "bothered" to fix their broken user-interface?  Do you wonder why they make incredible profits?

Did you think the two facts are unrelated?

20 May 2008

Invisible Work

Here's a great quote from Jim Waldo, courtesy of Dan Creswell's Blitz blog:
…Even worse than not being visible to the customer, work done on designing the system is not visible to the management of the company that is developing the system. Even though managers will pay lip service to the teaching of The Mythical Man Month, there is still the worry that engineers who aren’t producing code are not doing anything useful. While there are few companies that explicitly measure productivity in lines-of-code per week, there is still pressure to produce something that can be seen. The notion that design can take weeks or months and that during that time little or no code will be written is hard to sell to managers. Harder still is selling the notion that any code that does get written will be thrown away, which often appears to be regression rather than progress.
Never a truer word!

08 May 2008

Reprise

My week for being haunted by old ghosts.

A bit more than a week, actually... It all started last Thursday with a call from a lady working with a lot of organisations that support HIV counselling, treatment and management. A lot of organisations. She had tripped (how?) across a product/project that a few of us put together some years ago that we called Projectory. Projectory is a collaboration and communication platform, specifically aimed at software-development organisations and teams. Think of CollabNet. But better, of course! ;-) Certainly quite different in some key ways! Except we never got the business off the ground, mostly through an unlucky turn of events that resulted in us losing key sales people at a most critical time.

My caller was wondering whether the Projectory platform could be adapted to help them to communicate, collaborate and coordinate better with a couple of hundred other organisations. Well, we've set up a meeting for next week, and we'll see... What a blast from the past, though! I had all-but-forgotten about Projectory... Thankfully I have the code archived away somewhere safe.

And then it happened again. A call from an ex-colleague a couple of days ago: Could we put together a rough estimate and proposal for a social-networking platform for World Cup 2010. In case you're living in a cave (or the USA where "football" means something completely weird) South Africa will be hosting the 2010 Soccer World Cup, and, for South Africans, it is a very big deal. This is, after all, the second biggest sport event in the world after the Olympics. (China get the hell out of Tibet!) The weird bit is that what's being asked for is very, very close to the project we called Flightwish -- a social-networking platform centred around group travel opportunities and concepts. We failed to get funding for Flightwish, though we tried hard. Normally I would favour a small-start, organic-growth, little-or-no-funding startup model, but that path is clearly a very poor fit for a Grow-Big-Fast webalicious venture. Since then, a few others have started playing in that space, but I have yet to see any of them put together the exact combination of ingredients we planned. Would we have done better? Who knows?

But now we may just get a chance to try it again.

Key takeaways:
  • I suck at Sales and driving Sales, therefore I am a poor fit for CEO of a startup.
  • Flightwish taught me a deep hostility to the idea of a single "window of opportunity"; it's rubbish.
  • Beware the Websites Of Yesteryear! You put up a website. It's out there. You forget about it. Google doesn't! Fix them up or shut them down.
  • The "social" potential -- the ways that the web opens-up for collaboration and group communication -- we've barely scratched the surface of what's possible.
  • VC people in SA are mostly bankers with fancier job titles. And we all know the collective noun for bankers, don't we...

30 April 2008

Software Estimation Considered Harmful

problem

"So how long do you think it's going to take?"

We've all been asked the question a thousand times and more. Project Managers, Client Liaison, Salespeople, Marketing Managers,... they all want to know. And we, like sheep, like the suckers we are, because we try to please (you try, too, please!) suck hard on our spacebar-calloused thumbs, and guess.

"I guess a couple of days."

"A week."

"Three months."

"A year; maybe 14 months."

Nobody really trusts those really long guesses, though, so that's where the project management experts get involved, break the task down into itty, tiny bitty little bits, parallelise them, ALAP, ASAP, lead and lag. And then we all sit down around a table, and for each and every one of the itty tiny bitty bits the project manager asks the dreaded question.

"So how long do you think that one's going to take?"

The Real Answer, the Truest Truth, is, "I don't know." Perhaps a tiny voice deep inside our soul cries out, "I don't care! It is going to take as long as it takes." But for mysterious reasons all tangled up in our wetware, all tied up in the social hierarchy dynamics of the human ape and those twisty strand of ribonucleic acid in our hardware, "I don't know" marks me as less-than-competent. And "I don't care" is career limiting; "Not a team player. Fails to identify with the organisations goals and ethos."

So we guess, and we guess, and we guess again. And we're (almost) always wrong!

For 50 years we, as a craft, as an industry, have been guessing wrong. For 50 years our projects have mostly finished "late", and we keep wondering "Why?" Books have been written, Methodologies developed, PhDs awarded, Management Disciplines imposed and entire Consulting Industries built on the premise that it is ''possible'' to estimate software effort better than we presently do. Or, if we can't estimate better, then we can manage the estimation risk better. Or the process. Or those pesky damn programmers.

All in vain. The projects keep coming in late.

where it all comes from

How did this abysmal state come about?

I think it is rooted back in the 1950's, when the first Big Software projects were undertaken, most notably by the US military. Quite naturally they applied the project management strategies that have worked for them since time beyond time. Strategies that have successfully built forts, dug moats, laid siege to cities and moved large numbers of soldiers across continents and seas. The trouble is that those are extremely well understood problems that have been solved thousands -- millions -- of times, ever since Akkad invaded Ur. For such classes of problems, estimates of time and effort are pretty reliable. If you're a logistics planner, an army engineer, you know, from loads of practice and handbooks distilling five thousand years of experience, just how long it takes for a boatload of soldiers to move a given distance, and just how much food, water and fuel they need along the way.

"Where there are in the field a thousand swift chariots, ten thousand heavy chariots, and a hundred thousand mail-clad soldiers, with provisions enough to carry them a thousand ''li'', the expenditure at home, including entertainment of guests, small items such as glue and paint, and sums spent on chariots and armour, will reach the total of a thousand ounces of silver per day."
-- Sun Tzu, ''The Art of War''
It is an extremely well understood problem domain!

Software development is a different sort of beast. It's all new to us. Amost every problem has never been tackled in its entirety before. I don't claim that software development is unique in this; I am pretty sure that developers of new aircraft, ship engines, new forms of bridges, all face the same problems as software development. What -- perhaps -- distinguishes software development is that it's all new every time! We never get to repeat our past developments. If we were repeating -- exactly -- repeat-developing a requirement -- in every detail -- we would already have the code and we'd have no need to write anything! And if there's one thing I've learned about developers, its that we -- most of us -- hate doing the same thing twice! It's probably a function of our predisposition towards ADD/ADHD.

all change

But, no! In truth there are always differences between the Systems That Have Gone Before, and the Systems We're Developing Now. Even if you've developed interface to a dozen payment gateways, 100 gets you 1 that the next payment gateway has some unique characteristics all its own. Or perhaps some key technology has undergone a significant change in the last few years. Or some new tech has crept into the picture. "Our service is provided through a RESTful API"...

Then, too, how do you account for the Buggerance Factor. Murphy's Law. The simple fact that even the simplest piece of software depends on a ''huge'' number of other bits'n'pieces being in exactly the right places, configured exactly right, at exactly the right time?

I should not have had to spend an hour this morning tracking down the fact that a key library -- a jarfile that should always exist in a standard, accessible place -- had mysteriously been Taken Up. Vaporised. Gone. Perhaps my disk is going flaky? It shouldn't have happened. I should not have wasted an hour figuring out why the application wouldn't run. But I did.

I should not have spent a morning last week discovering that, despite the Vendor's persuasive assurances to the contrary, the version 8.2 driver does emphatically not work properly with the version 8.1 server. Let us not even ask the question, "Who installed the 8.2 driver?" A fruitless waste of time, energy and stress hormones.

How the hell do you predict that particular morning and figure it into your time and effort estimate? Or the half-day spent figuring out that there's a bug in a key data-access library you're using (and it's not your choice that mandated its use, but some arbitrary "policy".) And then another hour figuring out a way to work around the bug. Just how exactly, when you're Project Planning some 4 months ahead of a frustrating and unproductive morning, do you predict those?

You just blew your estimate.

managing estimation risk

I am well aware of various approaches to software development that try to futz around the problem -- some of them with some marginal success -- by giving up the idea of a project being "finished". But nobody seems willing to confront the central problem head-on.

Estimating "How Long It Will Take" is a Broken Idea.

Like the drunk looking for his spectacles under the streetlight "Because that's where I can see to look for them" we keep searching for ways to make estimation more accurate, more reliable, more amenable to conventional management thinking. What we really need to do is screw our courage to the sticking point, and accept that there is, really, honestly, truly, no alternative:

We must completely abandon the whole concept that software effort is amenable to estimation at all.

call to arms

Give up the crutch!

The next time -- and every time after that -- that someone asks you "So how long do you figure that's going to take?" -- "So when do you think we can go live?" -- Just Say No!

Just say, "It will take as long as it takes."

I guarantee you some excitement in the short minutes immediately following, but let go of your fear! Immediately you will find yourself skulling in the calm pond of assurance and truth that lies beyond the fear. Live and enjoy this Truth, for it will set you free. If other people want and need to make deadline commitments, let them be the ones to suck their thumbs, making up fantasies and lies. Do not allow them to push that responsibility onto you. Don't allow them to turn you into the liar. Just tell them,

It takes as long as it takes. It always does, anyway!

25 April 2008

Quotable

"The Ark was built by one man.
The Titanic was built by a team of professionals."
Related Posts Plugin for WordPress, Blogger...