…Even worse than not being visible to the customer, work done on designing the system is not visible to the management of the company that is developing the system. Even though managers will pay lip service to the teaching of The Mythical Man Month, there is still the worry that engineers who aren’t producing code are not doing anything useful. While there are few companies that explicitly measure productivity in lines-of-code per week, there is still pressure to produce something that can be seen. The notion that design can take weeks or months and that during that time little or no code will be written is hard to sell to managers. Harder still is selling the notion that any code that does get written will be thrown away, which often appears to be regression rather than progress.Never a truer word!
20 May 2008
Invisible Work
Here's a great quote from Jim Waldo, courtesy of Dan Creswell's Blitz blog:
08 May 2008
Reprise
My week for being haunted by old ghosts.
A bit more than a week, actually... It all started last Thursday with a call from a lady working with a lot of organisations that support HIV counselling, treatment and management. A lot of organisations. She had tripped (how?) across a product/project that a few of us put together some years ago that we called Projectory. Projectory is a collaboration and communication platform, specifically aimed at software-development organisations and teams. Think of CollabNet. But better, of course! ;-) Certainly quite different in some key ways! Except we never got the business off the ground, mostly through an unlucky turn of events that resulted in us losing key sales people at a most critical time.
My caller was wondering whether the Projectory platform could be adapted to help them to communicate, collaborate and coordinate better with a couple of hundred other organisations. Well, we've set up a meeting for next week, and we'll see... What a blast from the past, though! I had all-but-forgotten about Projectory... Thankfully I have the code archived away somewhere safe.
And then it happened again. A call from an ex-colleague a couple of days ago: Could we put together a rough estimate and proposal for a social-networking platform for World Cup 2010. In case you're living in a cave (or the USA where "football" means something completely weird) South Africa will be hosting the 2010 Soccer World Cup, and, for South Africans, it is a very big deal. This is, after all, the second biggest sport event in the world after the Olympics. (China get the hell out of Tibet!) The weird bit is that what's being asked for is very, very close to the project we called Flightwish -- a social-networking platform centred around group travel opportunities and concepts. We failed to get funding for Flightwish, though we tried hard. Normally I would favour a small-start, organic-growth, little-or-no-funding startup model, but that path is clearly a very poor fit for a Grow-Big-Fast webalicious venture. Since then, a few others have started playing in that space, but I have yet to see any of them put together the exact combination of ingredients we planned. Would we have done better? Who knows?
But now we may just get a chance to try it again.
Key takeaways:
A bit more than a week, actually... It all started last Thursday with a call from a lady working with a lot of organisations that support HIV counselling, treatment and management. A lot of organisations. She had tripped (how?) across a product/project that a few of us put together some years ago that we called Projectory. Projectory is a collaboration and communication platform, specifically aimed at software-development organisations and teams. Think of CollabNet. But better, of course! ;-) Certainly quite different in some key ways! Except we never got the business off the ground, mostly through an unlucky turn of events that resulted in us losing key sales people at a most critical time.
My caller was wondering whether the Projectory platform could be adapted to help them to communicate, collaborate and coordinate better with a couple of hundred other organisations. Well, we've set up a meeting for next week, and we'll see... What a blast from the past, though! I had all-but-forgotten about Projectory... Thankfully I have the code archived away somewhere safe.
And then it happened again. A call from an ex-colleague a couple of days ago: Could we put together a rough estimate and proposal for a social-networking platform for World Cup 2010. In case you're living in a cave (or the USA where "football" means something completely weird) South Africa will be hosting the 2010 Soccer World Cup, and, for South Africans, it is a very big deal. This is, after all, the second biggest sport event in the world after the Olympics. (China get the hell out of Tibet!) The weird bit is that what's being asked for is very, very close to the project we called Flightwish -- a social-networking platform centred around group travel opportunities and concepts. We failed to get funding for Flightwish, though we tried hard. Normally I would favour a small-start, organic-growth, little-or-no-funding startup model, but that path is clearly a very poor fit for a Grow-Big-Fast webalicious venture. Since then, a few others have started playing in that space, but I have yet to see any of them put together the exact combination of ingredients we planned. Would we have done better? Who knows?
But now we may just get a chance to try it again.
Key takeaways:
- I suck at Sales and driving Sales, therefore I am a poor fit for CEO of a startup.
- Flightwish taught me a deep hostility to the idea of a single "window of opportunity"; it's rubbish.
- Beware the Websites Of Yesteryear! You put up a website. It's out there. You forget about it. Google doesn't! Fix them up or shut them down.
- The "social" potential -- the ways that the web opens-up for collaboration and group communication -- we've barely scratched the surface of what's possible.
- VC people in SA are mostly bankers with fancier job titles. And we all know the collective noun for bankers, don't we...
30 April 2008
Software Estimation Considered Harmful
problem
"So how long do you think it's going to take?"
We've all been asked the question a thousand times and more. Project Managers, Client Liaison, Salespeople, Marketing Managers,... they all want to know. And we, like sheep, like the suckers we are, because we try to please (you try, too, please!) suck hard on our spacebar-calloused thumbs, and guess.
"I guess a couple of days."
"A week."
"Three months."
"A year; maybe 14 months."
Nobody really trusts those really long guesses, though, so that's where the project management experts get involved, break the task down into itty, tiny bitty little bits, parallelise them, ALAP, ASAP, lead and lag. And then we all sit down around a table, and for each and every one of the itty tiny bitty bits the project manager asks the dreaded question.
"So how long do you think that one's going to take?"
The Real Answer, the Truest Truth, is, "I don't know." Perhaps a tiny voice deep inside our soul cries out, "I don't care! It is going to take as long as it takes." But for mysterious reasons all tangled up in our wetware, all tied up in the social hierarchy dynamics of the human ape and those twisty strand of ribonucleic acid in our hardware, "I don't know" marks me as less-than-competent. And "I don't care" is career limiting; "Not a team player. Fails to identify with the organisations goals and ethos."
So we guess, and we guess, and we guess again. And we're (almost) always wrong!
For 50 years we, as a craft, as an industry, have been guessing wrong. For 50 years our projects have mostly finished "late", and we keep wondering "Why?" Books have been written, Methodologies developed, PhDs awarded, Management Disciplines imposed and entire Consulting Industries built on the premise that it is ''possible'' to estimate software effort better than we presently do. Or, if we can't estimate better, then we can manage the estimation risk better. Or the process. Or those pesky damn programmers.
All in vain. The projects keep coming in late.
where it all comes from
How did this abysmal state come about?
I think it is rooted back in the 1950's, when the first Big Software projects were undertaken, most notably by the US military. Quite naturally they applied the project management strategies that have worked for them since time beyond time. Strategies that have successfully built forts, dug moats, laid siege to cities and moved large numbers of soldiers across continents and seas. The trouble is that those are extremely well understood problems that have been solved thousands -- millions -- of times, ever since Akkad invaded Ur. For such classes of problems, estimates of time and effort are pretty reliable. If you're a logistics planner, an army engineer, you know, from loads of practice and handbooks distilling five thousand years of experience, just how long it takes for a boatload of soldiers to move a given distance, and just how much food, water and fuel they need along the way.
Software development is a different sort of beast. It's all new to us. Amost every problem has never been tackled in its entirety before. I don't claim that software development is unique in this; I am pretty sure that developers of new aircraft, ship engines, new forms of bridges, all face the same problems as software development. What -- perhaps -- distinguishes software development is that it's all new every time! We never get to repeat our past developments. If we were repeating -- exactly -- repeat-developing a requirement -- in every detail -- we would already have the code and we'd have no need to write anything! And if there's one thing I've learned about developers, its that we -- most of us -- hate doing the same thing twice! It's probably a function of our predisposition towards ADD/ADHD.
all change
But, no! In truth there are always differences between the Systems That Have Gone Before, and the Systems We're Developing Now. Even if you've developed interface to a dozen payment gateways, 100 gets you 1 that the next payment gateway has some unique characteristics all its own. Or perhaps some key technology has undergone a significant change in the last few years. Or some new tech has crept into the picture. "Our service is provided through a RESTful API"...
Then, too, how do you account for the Buggerance Factor. Murphy's Law. The simple fact that even the simplest piece of software depends on a ''huge'' number of other bits'n'pieces being in exactly the right places, configured exactly right, at exactly the right time?
I should not have had to spend an hour this morning tracking down the fact that a key library -- a jarfile that should always exist in a standard, accessible place -- had mysteriously been Taken Up. Vaporised. Gone. Perhaps my disk is going flaky? It shouldn't have happened. I should not have wasted an hour figuring out why the application wouldn't run. But I did.
I should not have spent a morning last week discovering that, despite the Vendor's persuasive assurances to the contrary, the version 8.2 driver does emphatically not work properly with the version 8.1 server. Let us not even ask the question, "Who installed the 8.2 driver?" A fruitless waste of time, energy and stress hormones.
How the hell do you predict that particular morning and figure it into your time and effort estimate? Or the half-day spent figuring out that there's a bug in a key data-access library you're using (and it's not your choice that mandated its use, but some arbitrary "policy".) And then another hour figuring out a way to work around the bug. Just how exactly, when you're Project Planning some 4 months ahead of a frustrating and unproductive morning, do you predict those?
You just blew your estimate.
managing estimation risk
I am well aware of various approaches to software development that try to futz around the problem -- some of them with some marginal success -- by giving up the idea of a project being "finished". But nobody seems willing to confront the central problem head-on.
Estimating "How Long It Will Take" is a Broken Idea.
Like the drunk looking for his spectacles under the streetlight "Because that's where I can see to look for them" we keep searching for ways to make estimation more accurate, more reliable, more amenable to conventional management thinking. What we really need to do is screw our courage to the sticking point, and accept that there is, really, honestly, truly, no alternative:
We must completely abandon the whole concept that software effort is amenable to estimation at all.
call to arms
Give up the crutch!
The next time -- and every time after that -- that someone asks you "So how long do you figure that's going to take?" -- "So when do you think we can go live?" -- Just Say No!
Just say, "It will take as long as it takes."
I guarantee you some excitement in the short minutes immediately following, but let go of your fear! Immediately you will find yourself skulling in the calm pond of assurance and truth that lies beyond the fear. Live and enjoy this Truth, for it will set you free. If other people want and need to make deadline commitments, let them be the ones to suck their thumbs, making up fantasies and lies. Do not allow them to push that responsibility onto you. Don't allow them to turn you into the liar. Just tell them,
It takes as long as it takes. It always does, anyway!
"So how long do you think it's going to take?"
We've all been asked the question a thousand times and more. Project Managers, Client Liaison, Salespeople, Marketing Managers,... they all want to know. And we, like sheep, like the suckers we are, because we try to please (you try, too, please!) suck hard on our spacebar-calloused thumbs, and guess.
"I guess a couple of days."
"A week."
"Three months."
"A year; maybe 14 months."
Nobody really trusts those really long guesses, though, so that's where the project management experts get involved, break the task down into itty, tiny bitty little bits, parallelise them, ALAP, ASAP, lead and lag. And then we all sit down around a table, and for each and every one of the itty tiny bitty bits the project manager asks the dreaded question.
"So how long do you think that one's going to take?"
The Real Answer, the Truest Truth, is, "I don't know." Perhaps a tiny voice deep inside our soul cries out, "I don't care! It is going to take as long as it takes." But for mysterious reasons all tangled up in our wetware, all tied up in the social hierarchy dynamics of the human ape and those twisty strand of ribonucleic acid in our hardware, "I don't know" marks me as less-than-competent. And "I don't care" is career limiting; "Not a team player. Fails to identify with the organisations goals and ethos."
So we guess, and we guess, and we guess again. And we're (almost) always wrong!
For 50 years we, as a craft, as an industry, have been guessing wrong. For 50 years our projects have mostly finished "late", and we keep wondering "Why?" Books have been written, Methodologies developed, PhDs awarded, Management Disciplines imposed and entire Consulting Industries built on the premise that it is ''possible'' to estimate software effort better than we presently do. Or, if we can't estimate better, then we can manage the estimation risk better. Or the process. Or those pesky damn programmers.
All in vain. The projects keep coming in late.
where it all comes from
How did this abysmal state come about?
I think it is rooted back in the 1950's, when the first Big Software projects were undertaken, most notably by the US military. Quite naturally they applied the project management strategies that have worked for them since time beyond time. Strategies that have successfully built forts, dug moats, laid siege to cities and moved large numbers of soldiers across continents and seas. The trouble is that those are extremely well understood problems that have been solved thousands -- millions -- of times, ever since Akkad invaded Ur. For such classes of problems, estimates of time and effort are pretty reliable. If you're a logistics planner, an army engineer, you know, from loads of practice and handbooks distilling five thousand years of experience, just how long it takes for a boatload of soldiers to move a given distance, and just how much food, water and fuel they need along the way.
"Where there are in the field a thousand swift chariots, ten thousand heavy chariots, and a hundred thousand mail-clad soldiers, with provisions enough to carry them a thousand ''li'', the expenditure at home, including entertainment of guests, small items such as glue and paint, and sums spent on chariots and armour, will reach the total of a thousand ounces of silver per day."It is an extremely well understood problem domain!
-- Sun Tzu, ''The Art of War''
Software development is a different sort of beast. It's all new to us. Amost every problem has never been tackled in its entirety before. I don't claim that software development is unique in this; I am pretty sure that developers of new aircraft, ship engines, new forms of bridges, all face the same problems as software development. What -- perhaps -- distinguishes software development is that it's all new every time! We never get to repeat our past developments. If we were repeating -- exactly -- repeat-developing a requirement -- in every detail -- we would already have the code and we'd have no need to write anything! And if there's one thing I've learned about developers, its that we -- most of us -- hate doing the same thing twice! It's probably a function of our predisposition towards ADD/ADHD.
all change
But, no! In truth there are always differences between the Systems That Have Gone Before, and the Systems We're Developing Now. Even if you've developed interface to a dozen payment gateways, 100 gets you 1 that the next payment gateway has some unique characteristics all its own. Or perhaps some key technology has undergone a significant change in the last few years. Or some new tech has crept into the picture. "Our service is provided through a RESTful API"...
Then, too, how do you account for the Buggerance Factor. Murphy's Law. The simple fact that even the simplest piece of software depends on a ''huge'' number of other bits'n'pieces being in exactly the right places, configured exactly right, at exactly the right time?
I should not have had to spend an hour this morning tracking down the fact that a key library -- a jarfile that should always exist in a standard, accessible place -- had mysteriously been Taken Up. Vaporised. Gone. Perhaps my disk is going flaky? It shouldn't have happened. I should not have wasted an hour figuring out why the application wouldn't run. But I did.
I should not have spent a morning last week discovering that, despite the Vendor's persuasive assurances to the contrary, the version 8.2 driver does emphatically not work properly with the version 8.1 server. Let us not even ask the question, "Who installed the 8.2 driver?" A fruitless waste of time, energy and stress hormones.
How the hell do you predict that particular morning and figure it into your time and effort estimate? Or the half-day spent figuring out that there's a bug in a key data-access library you're using (and it's not your choice that mandated its use, but some arbitrary "policy".) And then another hour figuring out a way to work around the bug. Just how exactly, when you're Project Planning some 4 months ahead of a frustrating and unproductive morning, do you predict those?
You just blew your estimate.
managing estimation risk
I am well aware of various approaches to software development that try to futz around the problem -- some of them with some marginal success -- by giving up the idea of a project being "finished". But nobody seems willing to confront the central problem head-on.
Estimating "How Long It Will Take" is a Broken Idea.
Like the drunk looking for his spectacles under the streetlight "Because that's where I can see to look for them" we keep searching for ways to make estimation more accurate, more reliable, more amenable to conventional management thinking. What we really need to do is screw our courage to the sticking point, and accept that there is, really, honestly, truly, no alternative:
We must completely abandon the whole concept that software effort is amenable to estimation at all.
call to arms
Give up the crutch!
The next time -- and every time after that -- that someone asks you "So how long do you figure that's going to take?" -- "So when do you think we can go live?" -- Just Say No!
Just say, "It will take as long as it takes."
I guarantee you some excitement in the short minutes immediately following, but let go of your fear! Immediately you will find yourself skulling in the calm pond of assurance and truth that lies beyond the fear. Live and enjoy this Truth, for it will set you free. If other people want and need to make deadline commitments, let them be the ones to suck their thumbs, making up fantasies and lies. Do not allow them to push that responsibility onto you. Don't allow them to turn you into the liar. Just tell them,
It takes as long as it takes. It always does, anyway!
25 April 2008
20 April 2008
Wizzards
Programming software is Magic.
I don't mean that there's something mystical about it, nor that it is intrinsically inaccessible to ordinary people. Nor (I emphatically add) do mean that it is like Magic. In every aspect I can think of, the act of Programming software meets all the criteria for performing Magic. Magic in the Swords and Sorcerers or Unseen University sense. "Alakazam!" and the Prince turns into a Frog. "Shazam!" and you're whisked away to a far, far place at high speed.
Just look at the facts: We (programmers) write programmes -- spells -- in arcane and cryptic symbol languages unknown to the common mob. Get the slightest part of the spell wrong, and, at best, it fails utterly to do anything. At worst it runs amok and fearful consequences ensue -- fires, floods, loss of money and even life! Get it just right, in every teensy, tiny, ball-aching, nit-picking detail2, and Lo! out of nothing, stuff happens in the real world. Gold changes hands. Trains run on schedule. Music plays and Feyries dance1.
Where nothing was before the spell was cast, something comes about solely because of the spell. That's Magic.
And, just like in the fantasies where Wizards keep pet Dragons and dribbly candles set atop skulls are the acme of interior decoration, programmers frequently work at odd hours, with intense, monomanic concentration bordering on the inhuman. And, like the traditional Wizardly Schools, programmers are admitted to different schools of various arts and degrees. So we have Clerics -- programmers content to churn out the boilerplate code needed to keep the wheels of commerce (and most web applications) running, but lacking any true proficiency with martial weapons of higher degree; Monks -- who eschew the use of particular weaponry but, ninja-style, willingly embrace whatever comes to hand as combat fodder; Wizards -- capable of serious Magic, but forget their spells once cast, capable of wonderful stuff, but doomed to repeat it -- with minor variations -- time after time; and then there are the Sourcerers -- Masters of The Source3 whose code is so elegant and expressive, so parsimonious and pretty as to make brave Programmers weep with envy and admiration.
No, you can not deny. Programming really is the realisation of the ancient idea of Magic. Say the Magic Spell and Change Reality.
[1] Well, anybody who wants to dance, I suppose, really.
[2] ...and My God, there's an inordinate amount of crappy detail that all has to be Just So!
[3] Waves to Ken, Doug, Dave, Jason, Paul, Johan, Bob, Brian, John and several dozen others...
I don't mean that there's something mystical about it, nor that it is intrinsically inaccessible to ordinary people. Nor (I emphatically add) do mean that it is like Magic. In every aspect I can think of, the act of Programming software meets all the criteria for performing Magic. Magic in the Swords and Sorcerers or Unseen University sense. "Alakazam!" and the Prince turns into a Frog. "Shazam!" and you're whisked away to a far, far place at high speed.
Just look at the facts: We (programmers) write programmes -- spells -- in arcane and cryptic symbol languages unknown to the common mob. Get the slightest part of the spell wrong, and, at best, it fails utterly to do anything. At worst it runs amok and fearful consequences ensue -- fires, floods, loss of money and even life! Get it just right, in every teensy, tiny, ball-aching, nit-picking detail2, and Lo! out of nothing, stuff happens in the real world. Gold changes hands. Trains run on schedule. Music plays and Feyries dance1.
Where nothing was before the spell was cast, something comes about solely because of the spell. That's Magic.
And, just like in the fantasies where Wizards keep pet Dragons and dribbly candles set atop skulls are the acme of interior decoration, programmers frequently work at odd hours, with intense, monomanic concentration bordering on the inhuman. And, like the traditional Wizardly Schools, programmers are admitted to different schools of various arts and degrees. So we have Clerics -- programmers content to churn out the boilerplate code needed to keep the wheels of commerce (and most web applications) running, but lacking any true proficiency with martial weapons of higher degree; Monks -- who eschew the use of particular weaponry but, ninja-style, willingly embrace whatever comes to hand as combat fodder; Wizards -- capable of serious Magic, but forget their spells once cast, capable of wonderful stuff, but doomed to repeat it -- with minor variations -- time after time; and then there are the Sourcerers -- Masters of The Source3 whose code is so elegant and expressive, so parsimonious and pretty as to make brave Programmers weep with envy and admiration.
No, you can not deny. Programming really is the realisation of the ancient idea of Magic. Say the Magic Spell and Change Reality.
[1] Well, anybody who wants to dance, I suppose, really.
[2] ...and My God, there's an inordinate amount of crappy detail that all has to be Just So!
[3] Waves to Ken, Doug, Dave, Jason, Paul, Johan, Bob, Brian, John and several dozen others...
15 April 2008
Bad. Punctuation,
Tripped across this little quote on /. this morning:
A Linux machine! Because a 486 is a terrible thing to waste!
-- Joe Sloan, jjs@wintermute.ucr.eduAlas, I can clearly see that somebody screwed-up the punctuation. Surely that should read
A Linux machine! Because a 486 is a terrible thing. To waste!
26 March 2008
SABC's website sucks
Email to info@SABC.co.za:
Not that I'm suggesting anything, mind...
"Why does SABC's website suck so badly when viewed in Firefox?BTW: if you leave off the "www." prefix, you get to see exaclty what software they're using to drive their (very b0rked) portal. Now I'm not suggesting that this might render them susceptible to getting the portal cracked, but anybody who has set up a portal server that incompetently has quite possibly left some default logins/passwords in place. Maybe?
"Not to mention that it is completely unusable with Javascript disabled, which renders it inaccessible to people using Braille readers or text-based browsers of any kind; this violates the constitutions provisions against discrimination."
Not that I'm suggesting anything, mind...
01 March 2008
User-interface Reboot
This article by Mr Mirchandani gets it exactly right: UI again ...don't pretty up, destroy!
I have never forgotten the experience of early last year. Our car had been stolen, and we were jumping through the licensing department's hoops to get the old car de-registered, and our new car registered. Well, 10-year-old, 2nd-hand car, since that's all we could afford with what the insurance company deigned to pay out -- another saga for another day.
First we could not de-register the old car, because it was flagged on the licensing system as "stolen", so no changes to its details are permitted. WTF? We could not unflag it, since that would require the police to mark the car as recovered, complete with verification of engine, chassis, VIN and registration numbers. Eventually we left the matter in the hands of one supervisor who took pity on us as I crumpled in the face of this actively-hostile "information" system. She solved the impasse by going outside the system: phone calls to a special contact in Pretoria -- "high friends in low places."
Then we had to register the new car. The details had to get captured no less than 5 times! Twice, manually by myself, the remainder by the clerk punching a terminal. And two of those instance involved recapturing the vehicle details from a form still-hot from their system's laser printer. The system already had the details, yet they still had to be manually recaptured. This is insane! Weren't computers supposed to save us work?
I have never forgotten the experience of early last year. Our car had been stolen, and we were jumping through the licensing department's hoops to get the old car de-registered, and our new car registered. Well, 10-year-old, 2nd-hand car, since that's all we could afford with what the insurance company deigned to pay out -- another saga for another day.
First we could not de-register the old car, because it was flagged on the licensing system as "stolen", so no changes to its details are permitted. WTF? We could not unflag it, since that would require the police to mark the car as recovered, complete with verification of engine, chassis, VIN and registration numbers. Eventually we left the matter in the hands of one supervisor who took pity on us as I crumpled in the face of this actively-hostile "information" system. She solved the impasse by going outside the system: phone calls to a special contact in Pretoria -- "high friends in low places."
Then we had to register the new car. The details had to get captured no less than 5 times! Twice, manually by myself, the remainder by the clerk punching a terminal. And two of those instance involved recapturing the vehicle details from a form still-hot from their system's laser printer. The system already had the details, yet they still had to be manually recaptured. This is insane! Weren't computers supposed to save us work?
23 February 2008
As BAD as Some can be, Others can be GREAT
We interrupt the on-going diatribe between my self and Datapro/Vox Telecom[1] to bring you Good News for Modern Persons.
In the supermortal words of Hubert Farnsworth, "Good News, Everyone!".
Last eve some mishap caused my DSL model/router to disconnect. For some while it failed to reconnect: AUTH_FAIL, it said.
Now, my ISP, WebAfrica, whom I hold in very high regard, has been having an occasional little trouble in recent times with their authentication servers. So: patience is the order of the day. It was quite late in the day, so my bed called, nothing in my little local network really needed Internet access overnight, so I left matters until the morning, in the hopes that the problems would be resolved without any input on my part.
They were not.
So... Onto the phone this morning. Less than two rings! (Contrast this with giving up after an hour on hold last week with Telkom!) Spoke to a chap who was remarkably candid: "Yes, we have had a problem, and a few accounts seem (for reasons we don't fully understand, yet) to have been stuck in an "inactive" queue. We're terribly sorry. I am sorting it out right now [clickety clickety clickety click]; would you like to hold?"
I declined to hold. The pain of being on hold to Telkom being too fresh in my psyche, I suppose. After suitable pleasantries I hung up.
A couple minutes later the phone rang. Same chap from WebAfrica. " I see that your modem seems to be having some trouble connecting. Could we please confirm the password it is using to connect...?"
Well, Bugger Me Sideways With A Spoon! Not only did WebAfrica's support guy sort the problem out instantly, with an ordinary, human-to-human acknowledgment that something had, indeed, gone wrong, but, after I had explicitly said "Ticket closed; I'll call you if there is any further problem." had monitored the situation to make sure that I -- The Lowly Customer -- had been properly sorted out, and called me back to make sure of it!
What am I saying, here?
Here's a significant point: None of us (modulo the Absolutely Bloody Minded) is so stupid as to believe that everything Works Flawlessly All the Time. Shit Happens. We know this. When it does, please don't lie to us and use phrasing designed to imply that we, the Customer, are Stupid, Insane and/or Lying! Please don't pretend that it is Somebody Else's Fault or an Act Of (somebody's) God. (Hello, Telkom!) If you've fucked up, admit it, apologise, and move on. Nobody will hold it against you. In fact, given the current climate of Assumed Corporate Infalibilty, we'll sympathise and likely offer to help you fix it!
Just say "Yes. We Had a problem. We've fixed it. (OR: Here's what we're busy doing to Fix It.) We're sorry."
If it is a significant proportion of the working day, offer a credit for the lost service time. Not difficult, is it? Not Rocket Science!
I cannot think of a way to praise this enough!
This most recent incident is the perfect exemplar of the sort of brilliant, attentive, honest service I have unfailingly received from WebAfrica! I have had a friend[2] phone me up especially to say, "Thank you for putting me on to WebAfrica as a service provider! I've since recommended them to at least 15 other people!".
I kid you not!
If anyone in South Africa wants or needs ADSL service, Internet access or web-hosting, do yourself a favour: www.webafrica.co.za.
Their rates are amongst the lowest around. Their service is out of all proportion to what you pay! (i.e. It's brilliant!) If they ever get bought out by Vox Telecom I shall probably have to leave the country -- and even then I won't find an ISP as good!
In the supermortal words of Hubert Farnsworth, "Good News, Everyone!".
Last eve some mishap caused my DSL model/router to disconnect. For some while it failed to reconnect: AUTH_FAIL, it said.
Now, my ISP, WebAfrica, whom I hold in very high regard, has been having an occasional little trouble in recent times with their authentication servers. So: patience is the order of the day. It was quite late in the day, so my bed called, nothing in my little local network really needed Internet access overnight, so I left matters until the morning, in the hopes that the problems would be resolved without any input on my part.
They were not.
So... Onto the phone this morning. Less than two rings! (Contrast this with giving up after an hour on hold last week with Telkom!) Spoke to a chap who was remarkably candid: "Yes, we have had a problem, and a few accounts seem (for reasons we don't fully understand, yet) to have been stuck in an "inactive" queue. We're terribly sorry. I am sorting it out right now [clickety clickety clickety click]; would you like to hold?"
I declined to hold. The pain of being on hold to Telkom being too fresh in my psyche, I suppose. After suitable pleasantries I hung up.
A couple minutes later the phone rang. Same chap from WebAfrica. " I see that your modem seems to be having some trouble connecting. Could we please confirm the password it is using to connect...?"
Well, Bugger Me Sideways With A Spoon! Not only did WebAfrica's support guy sort the problem out instantly, with an ordinary, human-to-human acknowledgment that something had, indeed, gone wrong, but, after I had explicitly said "Ticket closed; I'll call you if there is any further problem." had monitored the situation to make sure that I -- The Lowly Customer -- had been properly sorted out, and called me back to make sure of it!
What am I saying, here?
- I could have raved about not being kept on hold in some support-queue for an hour or longer.
- I could have raved about the great service I received during the handling of my support call.
- I could have raved all night in San Fransisco with the hot chick on her way to Hawaii (but that's another story!)
Here's a significant point: None of us (modulo the Absolutely Bloody Minded) is so stupid as to believe that everything Works Flawlessly All the Time. Shit Happens. We know this. When it does, please don't lie to us and use phrasing designed to imply that we, the Customer, are Stupid, Insane and/or Lying! Please don't pretend that it is Somebody Else's Fault or an Act Of (somebody's) God. (Hello, Telkom!) If you've fucked up, admit it, apologise, and move on. Nobody will hold it against you. In fact, given the current climate of Assumed Corporate Infalibilty, we'll sympathise and likely offer to help you fix it!
Just say "Yes. We Had a problem. We've fixed it. (OR: Here's what we're busy doing to Fix It.) We're sorry."
If it is a significant proportion of the working day, offer a credit for the lost service time. Not difficult, is it? Not Rocket Science!
I cannot think of a way to praise this enough!
This most recent incident is the perfect exemplar of the sort of brilliant, attentive, honest service I have unfailingly received from WebAfrica! I have had a friend[2] phone me up especially to say, "Thank you for putting me on to WebAfrica as a service provider! I've since recommended them to at least 15 other people!".
I kid you not!
If anyone in South Africa wants or needs ADSL service, Internet access or web-hosting, do yourself a favour: www.webafrica.co.za.
Their rates are amongst the lowest around. Their service is out of all proportion to what you pay! (i.e. It's brilliant!) If they ever get bought out by Vox Telecom I shall probably have to leave the country -- and even then I won't find an ISP as good!
[1] A "keyboard/finger" interaction nearly made that "Pox Telecom", whIch would have been appropriate...
[2] We've known each other over 35 years, now... I think that qualifies as friendship, no?
[2] We've known each other over 35 years, now... I think that qualifies as friendship, no?
22 February 2008
Taking on the Spammers: Datapro/Vox Telecom - Part 4
Mr Douglas Reed, CEO of Vox Spamacom Telecom, parent company to Datapro, replies:
Is it just me, or does this sound just a tad arrogant? What I am hearing: "We're big; that means we can spam with impunity, since we're too big to get blocked." and "Shut up and eat your spam!"
My response:
If anybody out there thinks I am being irresponsible or unreasonable (obviously with the exception of any Datapro or Vox Telecom employees or agents!) please, please say so by leaving a comment on this blog. I promise not to delete any relevant comments...
Let's just note for the record that he failed, completely, to address any single point of substance or question in my response...
We run an ISP with over 18,000 corporate customers and 180,000 SME's and
we have customers who utilise various services. These include list
servers where customers use their own databases and we don't have full
control. The DataPro and Vox databases are within our control and
consist of individuals and organisations who have provided their details
to us. The reason we have you on our Company database is because you are
obviously listed as a technical contact for some of our customers. We
cannot offer opt in opt out facilities for our communication to our base
because the news letters communicate important information that the
technical contacts need to be aware of. However if you want to be
excluded please give us the details and provide us with new technical
contact details.
The other choice is do what the rest of us do and add the user to your
junk mail list.
Interestingly enough this mail ended up in my junk mail folder which
basically means that I received unsolicited mail from this in the past
or you cc'd thousands of people.
Is it just me, or does this sound just a tad arrogant? What I am hearing: "We're big; that means we can spam with impunity, since we're too big to get blocked." and "Shut up and eat your spam!"
My response:
Dear Mr Reed, On 18/02/2008, Douglas Reed <douglasr@datapro.co.za> wrote:
> The DataPro and Vox databases are within our control and
> consist of individuals and organisations who have provided their details
> to us. The reason we have you on our Company database is because you are
> obviously listed as a technical contact for some of our customers.
The (many) spam emails that form the basis of my complaint to ISPA are directly from Datapro and Vox Telecom; this is not about spam from your customers. One spam message bears your name as "signatory".
You will note from my earlier correspondence with Maggie Cubitt that I have tried repeatedly, using numerous channels, to "opt out" of these mailing lists, without any success.
Why don't your opt-out procedures work? (As required by the ECT Act.)
Although some of your technical staff are certainly in possession of my email address as "technical contact" for some of our mutual customers, this does NOT extend a license to your companies to send me unsolicited bulk email on ANY subject.
Further, I will note that I have never -- not even once -- receive a bulk message on any technical subject. The emails forming the basis of my complaint have ALL been of a nature that can only be characterised as "marketing crap". I did not, ever, at any stage, give any person or system representing your companies, permission to send me marketing crap. The fact the your companies have done so is known in the email management industry as "address repurposing" and is considered a sure sign of "spam spoor".
> The other choice is do what the rest of us do and add the user to your
> junk mail list. I will repeat what I wrote to Ms Cubbit:
<quote>
having my own email address removed from your mailing lists is of only limited interest to me in this matter. The larger issue, which it is my main purpose to tackle, is that of Datapro and Vox Telecom blithely spamming, over an extended period of time, continuing in the face of numerous good-faith attempts to unsubscribe, and in direct violation of
1) their own Terms of Service,
2) the email provisions of the ECT Act, and
3) the ISPA's Code of Conduct.
</quote>
The point this: "adding Datapro/Vox Telecom" to my "junk mail list," as you suggest, fails to eliminate or mitigate the primary complaint against spam: the receiver has to pay for it. Putting Datapro/Vox Telecom into my "junk mail list" does not mean that Datapro/Vox Telecom cease being spammers.
To (attempt to) be completely clear on this: since you seem to have overlooked the point:
* This is not about Datapro and Vox Telecom spamming ME.
* This IS about Datapro/Vox Telecom spamming AT ALL.
> Interestingly enough this mail ended up in my junk mail folder which
> basically means that I received unsolicited mail from this in the past
> or you cc'd thousands of people.
Nonsense.
No such conclusion can be inferred.
Having personally administered email and spam-filtering systems, I can tell you that you cannot draw any such conclusion; thogh it /may/ call into question the competence of the people managing your spam-filtering systems.
> We run an ISP with over 18,000 corporate customers and 180,000 SME's and
> we have customers who utilise various services. These include list
> servers where customers use their own databases and we don't have full
> control.
What I read into this is that you believe that your organisations are "too large for the rules to apply". I have some bad news... There are other organisations far, FAR larger that manage to adequately, and to the full satisfaction of the anti-spam community, police their customers' mailing lists and email activities. I am pretty sure that both Outblaze and AOL are larger than your operations; both manage to maintain an impeccable reputation for managing the spam problem and speedily terminating spammy customers.
Of course, neither one spams their customers directly, as your organisations have done.
Part of your (companies') responsibility to the Internet community is to police your customers and their mailing lists. Ways to do this include monitoring their behaviour, and maintaining and ENFORCING uncompromising Terms of Service. Your response suggest an unwillingness to do so. This is a slippery slope. Next your sales-staff will be writing "pink contracts". (Google for it!) Should you require access to better expertise than your organisations evidently possess, I shall be glad to forward my consulting rates.
All of this remains (largely) irrelevant. The numerous spam messages I have received are from your organisations; not from your customers. Your unwillingness to eliminate spam from /within/ is, perhaps, indicative of your willingness to tolerate/profit-from spammy customers from without.
Here is the response I expect: As I see it (prove me wrong?), you have two choices:
1. Throw away all mailing lists under your control, and start from scratch to build new mailing lists. Of course you WILL follow established Internet procedures for building permission-base email lists. (Somehow, I doubt this one...)
OR
2. Send a ONE TIME email to all addresses on your mailing lists, explaining (in full) the situation, expressing your companies' regret that such an unacceptable and untenable situation has come about through the action of a few misguided individuals, and asking the recipients to confirm that they WISH to be subscribed to the relevant mailing list. Should recipients so confirm their desire to participate, your staff should proceed in the full confidence that those persons have positively opted-IN. Any email address that fails to reply, or that expresses a desire to opt-OUT must be removed from your databases.
This (second) option should be followed-up with a comprehensive on-going (so that new-hi[r]es get the message, too) educational message from the organisation: "We don't tolerate spam in any shape, manner or form." (together with a detailed explanation of just what that means.) Your marketing and sales staff may require particularly persistent education.
Forgive my lack of optimism.
Since you (read: your organisations) do not know the email address(es) being spammed, you may be sure that I am in a position to monitor your organisations' actions on this, and will report accordingly.
PS: You, Mr Reed, might wish to consider that a small one-man consultancy such as myself, may frequently be in a position to make recommendations to customers concerning their choice of service providers in the Internet Services industry. Either to recommend providers, or, alternatively, to discourage use of any particular provider. Your call...
If anybody out there thinks I am being irresponsible or unreasonable (obviously with the exception of any Datapro or Vox Telecom employees or agents!) please, please say so by leaving a comment on this blog. I promise not to delete any relevant comments...
Let's just note for the record that he failed, completely, to address any single point of substance or question in my response...
16 February 2008
Taking on the Spammers: Datapro/Vox Telecom - Part 3 - email Ping Pong
Obviously the spammers thought they could just listwash me and be done. Here's Datapro's latest response:
My response to them:Subject: RE: Response to ISPA complaint Date: Fri, 15 Feb 2008 12:55:45 +0200 From: "Maggie Cubitt" <maggiec@voxtelecom.co.za> To: "Mike Morris" <me> Hi Mike Apologies, as I can fully understand your frustration, which is why I am attempting to resolve it comprehensively and finally. I am unable to find the e-mail address mikro2nd@gmail.com on the Contacts database from the DataPro CRM.. and you have extracted the delivery address from your notepad doc. Can I please just confirm that the Newsletter was delivered to the e-mail address mikro2nd@gmail.com?
Forgive me my skepticism... ;-)Maggie Cubitt wrote: > Apologies, as I can fully understand your frustration, which is why I > am attempting to resolve it comprehensively and finally. Please understand that having my own email address removed from your mailing lists is of only limited interest to me in this matter. The larger issue, which it is my main purpose to tackle, is that of Datapro and Vox Telecom blithely spamming, over an extended period of time, continuing in the face of numerous good-faith attempts to unsubscribe, and in direct violation of 1) their own Terms of Service, 2) the email provisions of the ECT Act, and 3) the ISPA's Code of Conduct. > I am unable to find the e-mail address mikro2nd@gmail.com on the > Contacts database from the DataPro CRM.. and you have extracted the > delivery address from your notepad doc. Can I please just confirm that > the Newsletter was delivered to the e-mail address mikro2nd@gmail.com? The spam was not delivered to that email address, but another one. I am not willing to assist you in listwashing -- the much-loathed practise whereby spammers remove the addresses of the whiners, but continue to blast their unwanted spew out to the Silent Majority Who Just Hit Delete. I never opted-in to any mailing list belonging to Datapro or Vox Telecom, but was placed on it without my knowledge or consent via person(s) with whom I had contact for purely technical purposes on behalf of my own clients. This, in turn, means that my email address was repurposed for marketing spam. In turn Datapro's mailing list was repurposed by Vox Telecom, a company with which I have certainly never had any business relationship. (Yes, I do understand the relationship between the companies. No explanation needed.) Please take note that this is NOT the only list from which I get spammed by Datapro, so your problems are deeper and wider than listwashing a single whiny anti-spam "activist" from a single ill-constructed mailing list or database. If your lists are NOT fully confirmed-opt-in (and clearly they are not,otherwise I wouldn't be bothering you), then they're spammy lists until you can verify, with a full audit trail, that each and every recipient has positively confirmed their wish to opt in. Any addresses that cannot be so confirmed must be removed from your databases. All databases. The procedure for confirming mailing-list opt-in has been well-established, well-understood, standard practise in legitimate email management for at least the last 30 years, and is correctly implemented by every respectable mailing-list management system. I would expect an ISP as large as Datapro to be conversant with such established, accepted, and widely-implemented industry-standard, and to have the resources to ensure compliance. I realise that these practices are somewhat more stringent than required by SA law, but will point out that the ISPA Code of Conduct (para 28) mandates that "ISPA members must operate with due regard for established Internet best practices, as set out in the various request for comment (RFC) documents and as mandated from time to time by established and respected Internet governance structures." That reads: "established Internet best practices", not "ineffective South African law". I believe that mailing list operation is covered by RFC-3098 among other resources. Furthermore, you will, no doubt, have noted that the sample email sent to you is in violation of even the very modest requirements of the ECT Act. Not to mention the long-term on-going failure to heed good-faith removal instructions as required by the Act. I trust that Datapro's forthcoming response to this will measure up to the full scope of the organisation's evident ignorance of, or unwillingness to implement, Internet standards and best practise.
Taking on the Spammers: Datapro/Vox Telecom - Part 2
Yesterday, 14 Feb, the following response to my complaint to the Internet Service Providers' Association about one of their member's spamming activities:
FYI: Mr Reed is the CEO of Vox Telecom (the parent company), so hopefully we've got the attention of a Big Shot.
I have forwarded the most-recent offending email -- "signed" at the bottom by a Mr Gary Sweidan, Datapro's Managing Director, I am sure he is blissfully unaware of the content, or that it is being blasted to a who-knows-how-large list of unwilling , unconfirmed, not-opted-in recipients.
Sadly for them, I redacted out all the recipient email address details and message UUIDS that might server to identify the address it was sent to ;-)
One of the few spammer activities more loathsome than "address repurposing" is listwashing -- removing the whiners from your list whilst blithely continuing to spam the quite ones who Just Hit Delete.
FYI: Mr Reed is the CEO of Vox Telecom (the parent company), so hopefully we've got the attention of a Big Shot.
From: "Maggie Cubitt" <maggiec@voxtelecom.co.za>
To: <me>
Cc: "Douglas Reed" <douglasr@datapro.co.za>
Hi Mike As a listed Telecommunications Company we do take any reports of this
nature extremely seriously. We were very concerned to receive the
notification of your complaint to ISPA, and are obviously anxious to get
this resolved as a matter of urgency.
As there are many companies in the Vox Telecom Group and as DataPro, as
an ISP, does provide a bulk mailing service to customers as well, there
is a possibility that you are on one of our customer's databases.
In order to investigate this properly I would really appreciate if you
could forward me the "February newletter" to which you refer so that I
can investigate this thoroughly for you.
I look forward to your response.
Regards,
Maggie
I have forwarded the most-recent offending email -- "signed" at the bottom by a Mr Gary Sweidan, Datapro's Managing Director, I am sure he is blissfully unaware of the content, or that it is being blasted to a who-knows-how-large list of unwilling , unconfirmed, not-opted-in recipients.
Sadly for them, I redacted out all the recipient email address details and message UUIDS that might server to identify the address it was sent to ;-)
One of the few spammer activities more loathsome than "address repurposing" is listwashing -- removing the whiners from your list whilst blithely continuing to spam the quite ones who Just Hit Delete.
15 February 2008
Taking on the Spammers: Datapro/Vox Telecom - Part 1
For well over a year now I've been getting spammed by Datapro (a Vox Telecom subsidiary) with sundry Friendly Newsletters, Product Offers and Special Crap We're Sure Will Interest You. Now we're in a fight argument complaint-resolution discussion.
Background
Datapro is a fairly large supplier in SA of web and email hosting, ISP services, and all the myriad little bitty services around that. They're also one of only 15 "Large" members of the Internet Service Providers' Association -- the industry's self-regulation watchdog in SA -- and hence a signatory to ISPA's Code of Conduct, which includes a clause saying, in effect, "members won't support spam or spamming."
I have never been one of Datapro's customers because I think their technical standards are... dodgy... to say the least. But, I have had contact with some of their technical staff in the course making changes to email, web-hosting and DNS on behalf of some of my clients who do use Datapro as their service provider. For whatever misguided reasons. Evidently, some of Datapro's tech staff have had their email address-books "harvested" by The Marketroid Department. Or Something. How ever it happened, my email address got repurposed without my knowledge or prior consent. A major point, here, is that I have never been in a business relationship with this company.
In the anti-spam world "repurposing" is considered a Very Bad Thing, and will result in instant and permanent blacklisting on some aggressively well-run mail servers.
I've lost count of the number of times I have emailed the sender asking, demanding, pleading or threatening legal action, in the interests of getting off their mailing lists. Countless times I've clicked on the (rarely present) "unsubscribe" links and jumped through web-page hoops to get unsubscribed. Nary a confirmation have I received. Nor has any of this actually diminished the volume of crap I get from them.
To add insult to injury, Vox Telecom, the parent company, have in turn taken to spamming their subsidiary's lists.
A Lightbulb Moment
A short while ago, a contact on one of the local Internet-industry mailing lists I haunt, suggested that I lodge a complaint with ISPA. I must confess that I had never seriously thought about it, but maybe worth a try...
I waited. Made sure I gathered and archived the evidence. Then, last Tuesday, I struck: lodged a complaint via the ISPA's webform:
Action The First: The Complaint
Background
Datapro is a fairly large supplier in SA of web and email hosting, ISP services, and all the myriad little bitty services around that. They're also one of only 15 "Large" members of the Internet Service Providers' Association -- the industry's self-regulation watchdog in SA -- and hence a signatory to ISPA's Code of Conduct, which includes a clause saying, in effect, "members won't support spam or spamming."
I have never been one of Datapro's customers because I think their technical standards are... dodgy... to say the least. But, I have had contact with some of their technical staff in the course making changes to email, web-hosting and DNS on behalf of some of my clients who do use Datapro as their service provider. For whatever misguided reasons. Evidently, some of Datapro's tech staff have had their email address-books "harvested" by The Marketroid Department. Or Something. How ever it happened, my email address got repurposed without my knowledge or prior consent. A major point, here, is that I have never been in a business relationship with this company.
In the anti-spam world "repurposing" is considered a Very Bad Thing, and will result in instant and permanent blacklisting on some aggressively well-run mail servers.
I've lost count of the number of times I have emailed the sender asking, demanding, pleading or threatening legal action, in the interests of getting off their mailing lists. Countless times I've clicked on the (rarely present) "unsubscribe" links and jumped through web-page hoops to get unsubscribed. Nary a confirmation have I received. Nor has any of this actually diminished the volume of crap I get from them.
To add insult to injury, Vox Telecom, the parent company, have in turn taken to spamming their subsidiary's lists.
A Lightbulb Moment
A short while ago, a contact on one of the local Internet-industry mailing lists I haunt, suggested that I lodge a complaint with ISPA. I must confess that I had never seriously thought about it, but maybe worth a try...
I waited. Made sure I gathered and archived the evidence. Then, last Tuesday, I struck: lodged a complaint via the ISPA's webform:
Action The First: The Complaint
Let's see what results...NameISP: Datapro/Vox Telecom name: <redacted> email: <redacted> Address: <redacted> Telephone: <redacted> Cellphone: <redacted> SectionCoC: E. Unsolicited bulk mail (spam) Details: I have never been a customer of Datapro. My only interactions with them have been on behalf of my clients, in the course of managing clients' DNS, email, hosting, etc. technical requirements where those services have been provided (at the clients' choice) by Datapro. As such my interactions have been with technical service personnel only. During the course of such interactions Datapro staff have, without my consent or prior knowledge, added my email address to various mailing lists that they use to send marketing "newsletters" and advertisements (a.k.a. address repurposing.) I have on numerous occasions requested that my details be removed from all mailing lists and databases under Datapro's control to no avail. I have made such requests telphonically, by email, and by clicking through the (rare) unsubscribe links that some of this spam contains. Finally I have records good enough to prove my point. Their latest "February newletter", sent in duplicate today, 9 February 2008, is in clear violation of 1) my past instruction to them of 2 August 2007 (and subsequent, evidence-free removal-link-clicking) 2) the ECT Act itself, in failing to meet the information provision and opt-out requirements of the Act, and 3) the ISPA Code of ConductCopies of all relevant emails are available from myself.
Subscribe to:
Posts (Atom)
