Turning URLs into URIs: shrtn - A URL redirector/shortener.

The shutting down by Google of their project-hosting has forced me to migrate my URL-shortener project to BitBucket (because Mercurial is so much nicer than GIT), and along the way caused me to take it upon myself to resuscitate the project. For one thing, I've passed out URLS that rely on it residing at 1.mikro2nd.net, and, as things currently stand, those are dead links. For another, I've figured out a second, more important, use case for URL redirectors than mere shortening. 

While shortening URLS may have some (highly debatable) utility, I consider them to be, on balance, a harmful thing. They make the link destination opaque, and the poor sucker clicking on them has absolutely no idea where they might end up. Then, too, the indirection allows the redirector host to potential introduce any sorts of mischief or stupidity into the user's browse path. These are not insurmountable problems, just small stumbling blocks in the path of the concept, but they do argue for considering URL-shortening to be harmful. Or at least seldom accruing benefit to the clicker.

What they do buy you is the ability to gather ego-gratifying statistics on who clicked your links, which can give you a measure of how well your voice is heard in the social Internet. An oft misleading measure, to be sure, but sometimes some measure is better than none, as long as the user of said statistics remains aware of their limitations and biases.

There is, though, a very legitimate and compelling use for the idea of redirectors: They provide a bridge between the world of URIs and the world of URLs. (For the longest time I was a bit hazy on the distinction between the two, but I'm better, now, thank you.) A shortener (or redirector) service allows us to publish identifiers of stuff (I'm trying to avoid the "content" word having developed a nasty allergy) to the 'net without worrying about the location or address of that stuff. For example, I might have published a document (or an application or a collection of photographs or whatever) using some file-hosting service. Let's call it odearieme.com. Now it turns out that odearieme.com was also hosting a bunch of stuff that the American copyright fascists decide they want disappeared. So they have the FBI kick down the doors of ohdearieme's hosting centre with the aid of a compliant country's spook services, and they steal remove the servers holding your perfectly legitimate and legal "stuff". Too bad. Anybody hanging on to the URL you gave them referring to ohdearieme.com is now stock out of luck. Had you used a redirector, though, it would be no problem. All you'd have to do is upload a copy of your "stuff" to a new file hosting service ad change the destination URL in you redirector. This is, indeed, as useful thing, and, after all, exactly what they Domain Name System is all about.

So there's my compelling use-case for a URL shortener/redirector, and I still have a couple of hundred words to write to reach my day's target. Let me describe, then, some of my other thinking around this little project.

I've already implemented two or three different storage schemes for the "database" that the server needs in order to work, and exactly which one gets used for a given deployment is merely a configuration issue. So what's another one? The trouble is that it is becoming cumbersome to include all the storage implementations, along with their dependencies, in the final deployed product. I know that many, if not most, developers would just chuck everything and the bathtub into the deployment, but it offends my sense of neatness. I want to build a number of separate artefacts: one that contains the actual redirector server, and then one for each storage scheme. That way a deployer (or, heavens forfend, an actual System Administrator) can deploy only the exact artefacts they need. This also means that updates to one module don't necessitate a refresh of the entire system. More immediately, it means that I want to build a number of separate artefacts for this project, rather than a simple WAR file, making Maven a much better fit than the straightforward Ant build generated by Netbeans, so I'm having to (at last) learn something about using Maven properly. I guess I'll learn to live with the tediously slow builds, though it does feel like the 1990's called and want their Makefiles back.

Then, too, I'm of a mind to host this thing (at least my own personal redirector) on Google's massive cloudy infrastructure. Given the meagre volumes of data I'll be shifting (which, presumably, speaks volumes for the paucity of my Social Internet Fu) I'm pretty-well certain I can keep hosting it there free for all eternity—or until Google decide to shut down their cloudy hosting engines or make them for-pay only. If or when that happens I guess I'll have to move the redirector back onto my own infrastructure, so I don't want to lose the ability to deploy to my own (Tomcat) application server. That means teasing out the deployment configuration and infrastructure-specific stuff from the core of the redirector code itself. All sounds reasonably doable for me, and I'm using the exercise to polish up my Mercurial, Maven and AppEngine skills, not to mention brushing up on changes in the Java language and APIs. I might even use it to improve my Javascript skills or get properly to grips with one of the JS front-end frameworks.

Darker Corners of Android: BroadcastReceivers and Intents

Not all in Android-land is lightness and brightness. A few dark corners and cul de sacs await the unwary, ready to trap us in sticky tarbaby snares of mystery debugging. One such is a case we recently encountered in relation to BroadcastReceivers and the way Intents get delivered to them.

I've written previously about implicit Intents versus explicit Intents in the context of describing a useful code-pattern for preparing Intents. To briefly recap, explicit Intents name the class of the component they are aimed at whilst implicit Intents simply describe the desired action they intend to trigger and allow the Android intent-delivery system to find the best-matching component to handle the Intent. Indeed, a better naming scheme might have been "broadcast Intent" for implicit Intents and "unicast Intent" for explicit Intents.

BroadcastReceivers can be registered for receiving broadcast Intents in one of two ways, either through a declaration in the Android manifest file, or via a dynamic registration by another component.

If you register a BroadcastReceiver via a manifest declaration, then you must provide the class-name of the receiver implementation. To communicate with the receiver, you broadcast an Intent – either one that matches the receiver's intent-filter (if it was declared to have one) or an Intent that specifies the class-name of the receiver. In this latter case it's hardly a broadcast, really, since the Intent is directed at a specific target receiver. Nevertheless, this is a perfectly valid way to send events and information to a BroadcastReceiver.

If you dynamically register a BroadcastReceiver – by calling Context.registerReceiver() – you supply a BroadcastReceiver instance and an IntentFilter describing the sorts of Intents that should be passed to the receiver. And herein lies a small gotcha. You register an instance.

If you try to send an Intent to the receiver by naming the class of the BroadcastReceiver, it will never get delivered. Sending the Intent using something like

Intent i = new Intent( this, MessageReceiver.class );
sendBroadcast( i );

won't work. The Android system will not match the Intent you declared to the class of the BroadcastReceiver instance you registered. The only way to send Intents to dynamically registered BroadcastReceivers is by having them match the IntentFilter attached to the receiver when it gets registered.

Explicit Intent matching just doesn't work in this case.

I've put some example code on github that demonstrates all this. There's a simple BroadcastReceiver (the MessageReceiver class) that gets registered dynamically by the MainActivity. The MainActivity instance then tries to send Intents to this receiver instance using two different methods. First it sends an Intent that matches the Action declared in the receiver's intent-filter. That Intent gets through successfully. Then it tries to send an Intent that explicitly names the MessageReceiver class. It never arrives.

As much as I love the eventful nature of Android software architecture I view this as a bug (Come on guys... How hard would it be to do a getClass() on the instance that gets registered?) but it is what it is. It took us a day or so to figure out why Intents were mysteriously failing to be delivered to an otherwise perfectly ordinary BroadcastReceiver, so I thought I'd save you, Dear Reader, the trouble. It's pretty straightforward to cope with (rely on Intent actions rather than explicit Intents) once you know of this issue, but I have not seen it documented anywhere in the Android reference materials^ε.

Share and Enjoy.

---

^ε If anyone does spot a description of this in the official docs, please do let me know!

Android Patterns: Manufacturing Intents

Activities, Services and BroadcastListeners are the basic components of application structure in Android. They are all started by firing an Intent that causes the Android system to activate them. There are two very different kinds of Intent that might activate an Activity: implicit Intents, and explicit Intents.

We present here a useful pattern for creating those Intents in a way that makes code more robust and easier to maintain.

For the purpose of simpler description we'll talk about starting Activities using Intents, though the same pattern makes most sense when used (with due care and discretion) for all components.

Implicit Intents

Implicit Intents do not name the specific Activity they trigger. Instead Android looks for the best match between an Intent's content and any Intent Filters declared by the Activity. This is what happens when your application's main Activity is started by an Intent that has its action set to ACTION_MAIN and its category set to CATEGORY_LAUNCHER.

Implicit Intents and their matching by IntentFilters is the key way that applications are able to interact with the entire ecosystem of applications and services in an Android system. It allows us to run an application that (for example) makes use of a web-browser or email client without having to embed all the functionality of a browser or email client in the application itself; we can merely assume that a suitable application exists in the ecosystem, and all our application has to do to leverage that capability is to fire the right Intent. This system of implicit Intents is one of the most powerful aspect of Android, and one which many applications fail to use well. How many applications have we seen that try to provide their own web-browsing capability, ending up providing some half-baked, crippled web-page fetcher-and-renderer, while we know perfectly well that the system almost certainly hosts one or more full-fledged, powerful web-browsers, if only they would just fire an appropriate Intent instead of trying to go it alone.

Explicit Intents

Explicit Intents are Intents that name a specific component – an Activity, Service or BroadcastReceiver – as the thing to be activated using the components class name. As soon as an Intent names a specific component, all other forms of Intent matching against filters is abandoned, and Android activates only the specific component, without any regard for the other bits of data the Intent may carry.

In terms of Activities, explicit Intents are used to implement in-application navigation between Activities, and many of the target Activities will not have any associated IntentFilter at all, so there is no other way to activate them other than with explicit Intents fired form other parts of the application.

Pattern

Each Activity, Service or BroadcastReceiver should provide a factory that returns template Intents for starting that Activity, Service or BroadcastReceiver. In the simplest case – say an Activity started by an explicit Intent – this might be a simple factory method:


public class ListManagerActivity extends Activity {

    public static Intent getStartIntent( Context context ){
        return new Intent( context, ListManagerActivity.class );
    }
    ...
}

In this most elementary form such a factory method is not, by itself, very useful. It becomes ''much'' more compelling when the Activity in question expects or requires certain additional data to be present in its starter Intent:


public class ListManagerActivity extends Activity {
  public static Intent getStartIntent( Context context, UserInfo userData ){
    checkNotNull( userData );
    final Intent startIntent = new Intent( context, ListManagerActivity.class );
    startIntent.putExtra( EXTRA_USERINFO, userData );
    return startIntent;
  }
  ...
}

In this case we achieve two worthy objectives with this simple factory method:

1. We make sure that the Intent that starts the ListManagerActivity always contains the userInfo extra that (presumably) the activity absolutely requires in order to behave correctly, provided, of course, that we only ever manufacture start Intents for ListManagerActivity using this method, and never by calling new Intent(...) – something that can quite easily be checked using various code-quality audit tools.
2. We keep the code that builds start Intents (with their extra data) close to the code that consumes those Intents (and that extra data) so that, if we change the notion of what data an Activity requires, we're already in the right place in the code to make the corresponding fixes to the manufacture of those Intents, rather than having to hunt around all over our codebase.

There might be more complex cases where components get started using a variety of different Intents. Perhaps Intents might carry different extra objects or flags affecting the component's behaviour. Perhaps the component might be activates using implicit intents with varying options for their data URI. Indeed, a common mistake when passing a data URI in an implicit Intent is forgetting to correctly set the mime-type for the data URI, in which case matching the Intent against an IntentFilter mysteriously fails.

In such complex cases it may be appropriate to supply several Intent-factory methods in the class that gets activated. It may even be desirable to provide an Intent Builder class:


public class ListManagerActivity extends Activity {
    public static class Builder {
        private Intent startIntent;
        public Builder( Context context, Foo mandatoryData ){
            startIntent = new Intent( ListManagerActivity.class );
            startIntent.addExtra( EXTRA_FOO, mandatoryData );
            startIntent.addFlags( FLAG_ALLOW_NEW_ITEMS );
        }
        public Builder addVitamins( final Vitamin nutrient ){
            startIntent.addExtra( EXTRA_VITAMIN_ID, nutrient.getId() );
            return this;
        }
        ...
        public Intent build(){
            validateIntent();
            final Intent constructedIntent = startIntent;
            startIntent = null;
            return constructedIntent;
    }
}

TL;DR:

Don't create start Intents for Activities, Services and BroadcastListeners willy-nilly in myriad and random places all over your codebase. Rather have each such component class provide factory methods or builders to manufacture those start Intents.

This ensures that

• the Intents carry all the necessary data that the component expects, 
• that the data is added in close proximity to the places where it is consumed, so that changes in the requirement are easily mirrored in the corresponding changes in the creation and structuring of that data, and
• mandatory data required by the component can be reflected by mandatory arguments in the signature of factory methods, ensuring that required data cannot be easily forgotten.

Netbeans EE Dependency Madness

Dredging up an ancient persona-time web-project and dusting it off caused me to reinstall the Base-EE module for Netbeans - long my IDE of choice. To my dismay, Netbeans pops up the following list of dependent modules:

This is madness!

I will never be using JSF, IceFaces, PrimeFaces or Struts. There will be frost on the ground in Hades before I ever use any part of Spring Framework without being severely and repeatedly bludgeoned with a blunt instrument. And if I ever do choose to host a webapp "in the cloud", Oracle is probably going to be my very last choice, though I can understand them being desperate to punt their wares in an IDE that they do, after all, fund in some measure.

So why are these listed as required dependencies? It might be that the EE Base module uses some features from those modules, in which case, why have those not been broken out as separate modules, this being one of the huge strengths of the Netbeans Platform?

Something is very smelly in the nation of Netbeans!

Web-design HowNotTo

Seen recently on a mailing-list I receive:

> These guys stock 29mm Crown Caps http://www.africancork.co.za

Went off to take a look at their website...

"You have to be logged in to view our products"

They have to be joking! I can only assume that they actively hate it when people want to buy their stuff. Needless to say, I bounced.

Do people really still misunderstand the nature of the web that badly when we're already more than 10% of the way through the 21st Century?

Google Groups: The Orphan Child

It's been quite a while, now, since Google rearranged the menus providing links to their various sites and tools. And right from the start Google Groups has been Missing In Action. At first I thought it might just be a small oversight, but evidently it's a deliberate design decision.

Google Groups: Nowhere to be found.

I wonder why... I find Google Groups to be one of the most useful tools they offer, especially when I am working with people who are not very net-savvy -- those who are not as comfortable with web-based tools as I am. The only rationale I can come up with is that Google would like us all to migrate over to g+ and abandon the somewhat old-school "forum" style that is Groups. Ain't gonna happen!

Unless they force the issue by killing-off Groups, in which case I will hazard a guess that existing fora using Google Groups as their carrier will simply hike on over to elsewhere that does provide a forum-style interface.

I'm not a huge fan of forum-style UX, and I avoid it as far as possible, but it is something simple and familiar to many people, and so sweetly occupies a place where it is the best thing to use because it is the simplest thing for the largest number of users. I do hope it's not going to vanish completely in a stupid attempt to force everyone over to g+... (Like what Microsoft seem to be trying to do to their user-base.) That would be,... well, not quite Evil, I guess,... but certainly quite an unfriendly move.

Unit-testing Android Apps: Netbeans and Robolectric

Testing for Android Apps is a bit of a mess. The Android Test Framework dismally fails to make a clear distinction between unit tests, functional/integration tests and UI tests, and assumes that all tests will be run as an Android application on a device or emulator (AVD) as Dalvik code. For unit testing this is extremely clunky. It means that unit tests must be written as a standalone Android project, built as an APK and deployed to the device running a build of the application under test, where it takes over the running of the application to instrument it for testing. This is quite a slow process, and slowness is the very opposite of what we want for unit tests.

All I wanted, though, was to run (quickly!) some simple unit tests against domain classes. There might be some Android mocks needed where my application classes rub up against the SDK, but that's just accidental.

Then, too, there is (currently) a bug in the Ant build script for the Android Test Framework that means that, out of the box, tests cannot be built or run using Ant. I neither know, nor do I care, whether the test might be runnable if I were using Eclipse; I'm a diehard Netbeans user, and Netbeans uses Ant (or Maven, and possibly soon, Gradle) rather than relying on a parallel build system of its own. I think this is one of Netbeans's key strengths. Along with the still-evolving NBAndroid Module and Netbeans's superb out-of-the-box handling of XML files, I've been finding it an absolute pleasure to use for Android development. But unit-testing remained an itch that I had to scratch.

The upshot of all this is that I could not find a satisfactory -- or, indeed, a working -- way to write and run unit tests for the Android project I'm working on.

Down into a Rabbit Hole I went!

Emerging a couple of days later, here's the solution I came up with... It's probably not the best solution ever. It's certainly not definitive. But it Works For Me.

I'm using Robolectric -- a framework that modifies Android platform methods on the fly and provides shadow classes for Android platform classes so that unit test can be run in a standard JVM on my development machine.

Predictably, it was not all Plain Sailing!

The Robolectric documentation is pretty sparse, and very strongly oriented towards IntelliJ and/or Maven-and-Eclipse usage (didn't investigate too closely, don't care!) If I'm going to be shafted into using Maven, then my unit testing is so slow that I might just as well stick with using the standard Android Way of Testing, so I wanted to avoid that. Additionally, this brings the advantage that I can use JUnit-4 style tests (with a caveat - see below.)

The first step is to create a standard Java Project in Netbeans. We already have a directory structure like

...project-root
       |
       +---android-main-project
       |
       +---other-project-related-dirs

so I created my unit-test project on the same level as the main project:


...project-root
       |
       +---android-main-project
       |
       +---other-project-related-dirs

       |

       +---unit-test-project

In Netbeans add at least the following libraries as dependencies for your test project:

• Your main project's bin/classes directory - otherwise the test-project cannot find the classes you want to test.
• JUnit
• Any mocking library you might choose to use - PowerMock looks pretty good, though I haven't used it beyond tyre-kicking yet.
• android.jar for the SDK version you have as your targetSdkVersion in the AndroidManifest of your main project. This is important, since the Robolectric framework uses a custom test-runner (which we'll slightly modify as described below) that examines your AndroidManifest to figure out where to find resources.

The order you specify the libraries is critical: android.jar must come after junit in the classpath for your unit-tests.

Now only one more things remains: telling Robolectric where to find your main project's manifest file and resources...

In your test-project, create a custom test-runner like

net.mikro2nd.android.junit;

import java.io.File;

import org.junit.runners.model.*;
import com.xtremelabs.robolectric.RobolectricConfig;
import com.xtremelabs.robolectric.RobolectricTestRunner;

public class MyTestRunner extends RobolectricTestRunner {
    public MyTestRunner( final Class<?> testClass )
        throws InitializationError
    {
        super( testClass, new RobolectricConfig(
            new File( "../android-main-project" )
        ));
    }
}


Of course you should now annotate your test classes using

    @RunWith( MyTestRunner.class )
    public class DeviceIdTest{
        ...
    }


Now you can "Test" your test project to run all unit tests, select just a single unit-test class to run, or even debug unit-tests without leaving Netbeans at all. And you can run them in a CI server.

Slightly less convenient than unit-testing a non-Android project, but still way better than the alternatives (no testing, Eclipse or Maven, or the bloody slow Android Test Framework.)

My thanks to Justin Schultz who's Eclipse Robolectric Example put me on the right track. All I've done is adapt his hard work for Netbeans use.

Moving Furniture: The Scrum Way

As a housewife, my wife
Wants the bed in one room swapped with the couch/futon/bed in another
So that  she can more comfortably sit and watch the Olympics on TV in the sunnier, warmer room.

With 3 programmers in the room, it was inevitable that we would choose an Agile approach to this requirement. The User Story above was our first step, and was duly agreed to by said wife as Accurately Describing What She Wanted and its Business Value.

Given the constraint that both rooms involved are too small to contain both pieces of furniture simultaneously, we realised that we would need a temporary variable as a swap space. I'm pretty sure that, barring some major advance in our understanding of Quantum Mechanics, pieces of furniture can't be swapped by being XOR'd through each other.

We broke the Story down into some Tasks:

1. Remove the bed from Room 1 into the Swap Space.
2. Remove the Futon from Room 2 into the Swap Space.
3. Move the Futon Base (a finger-eating piece of folding woodworkery) from Room 2 into Room 1 and position it in its final position.
4. Move the Futon from the Swap Space into Room 1 and position it on its Wooden Base.
5. Move the bed base from the Swap Space into Room 2 and position it in its final position.
6. Move the mattress from the Swap Space into Room 2 and place it on its base.

Seemed like a reasonable breakdown at the time. We did some estimations and figured we could fit it all into a single Sprint.

Here's how it played...

Task-1 went quite smoothly. Mattress and bed base moved to the Swap Space.

Task-2 also went well. Please bear in mind that the futon and mattress are large, floppy and heavy items, so not all that easy to move about.

Martin and I -- Pair Moving -- make an attempt at Task-3. Space is very constrained, so moving the Futon-base into Room 1 involves a brief detour into Room 3. Halfway there we suddenly realise that -- because the base is asymmetric -- we have to turn it around in the Swap Space before it can be Deployed into Production.

We complete Task-4, only to realise that we were entirely wrong about which way round the base has to go. We've moved it into the room the wrong way round. And the room is too small for us to turn it around short of Interesting Topological Manoeuvres.

So we back out again, via Room 3, back into the Swap Space where there is sufficient room to turn the base around.

Some discussion ensues. We realise that we should have had better Unit Tests in place for this Task. Finally we manage to complete the Task.

It's a tight fit, but the Futon/couch will serve our User perfectly as she wished.

Sadly there's an unforeseen bug. A small drawer unit now blocks the doorway, making for extremely poor UX. Not our problem, though. We've met our spec and implemented the Story. People will simply have to leap over the drawer-unit for the time being. Changes will have to be the subject of a subsequent Sprint.

Happily I can report that the rest of the Sprint went quite as anticipated. We were even able to implement a small Improvement Task by orienting the bed in Room 2 in a much more User Friendly fashion.

I do wish Furniture Topology was amenable to Version Control, though. It would have made our Task-4 issues a bit simpler to resolve. But then, perhaps we should have abandoned the Sprint at that point.

Still not quite sure how all the Scrum ceremony helped, though.

JSP/JSTL: Versions and Standards

It baffles me. So many developers who use  infrastructure standards like servlets and JSP, but don't have any idea what versions of these they're coding to. They are blissfully unaware of whether they're targetting HTML4.01, XHTML-1 or HTML-5, CSS 2.1 or CSS 3, Servlet 3, 2.5 or 2.4...

Maybe I'm just funny that way - I think that managing dependencies in a tight, controlled way really matters! Otherwise things quickly spiral out of control, and you end up with that classic Design Pattern, "Big Ball of Mud".

On the other hand, I freely confess that I keep forgetting which pieces belong together. It's such a confusopoly.

So, mainly as a Note To Self, here's a quick lookup:

JSP/Servlet/JDK Versions

Servlets	JSP	JSTL	Java EE	Tomcat	Min. JDK
3.0	2.2	1.2	5	7.x	1.6
2.5	2.1	1.2	5	6.x	1.5
2.4	2.0	1.1	1.4	5.5.x	1.4

The list goes on, but if you still have systems in production running those archaic versions, you're probably in deeper trouble that mere version-confusion anyway!

Work Wanted

This is a small call for help. I am looking for work, and need your help.

Contract or permanent. Preferably (but not exclusively or even at all) telecommute. I can code, design, architect software, consult in any of these (and dev process/team issues) and teach a variety of Java, OO Design and web development topics, and would be happy to do any/all of these. I additionally have some experience doing system administration work. I work in Linux/Unix environments and know next-to-nothing about Windows.

I've done web stuff, and loads of backend "heavy lifting" coding where reliability, scalability, etc. are important. I am not good at quick'n'dirty. Skeptical of the value of buzzwords and big-arse frameworks.

Check out my "business" website for more details about me and what I've done in the past.

R (as they say) "highly neg".

If you have anything suitable, or know of anything that fits, please drop me a line.

I will be in Cape Town and environs next week, so an ideal time to get together and chat about possibilities and opportunities.

Android Nails Sandboxing

So I'm learning to programme the Android platform. Despite constantly typing it as "Androind" finding programming fun again after many years of regarding it as somewhere between tiresome drudgery and only mildly interesting in sporadic parts.

It's early days, yet, but I do think that Android's architects had one flash of brilliant insight: Using Unix user and group permissions to sandbox applications. Brilliant! We've had this mechanism since forever, and let's be honest, it's never been all that useful except in the very early years of Unix when we actually did have to put multiple users on a single computer. And even then, most users didn't understand it. Questions about umask and file permissions are among the commonest of Unix confusions I've run across for the past 25-odd years.

Warping the idea to mean that every application is a unique user is a flash of inspiration.

QOTD

'the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."'

Donald Knuth 25 April 2008

(Okay, so I'm late to the party. As always.)

Design using Other Peoples' APIs

Where you are dependant upon somebody else's API, decouple from that API at the earliest possible opportunity so that the remainder of your system works in terms of your own abstractions rather than that somebody else's. This shields you from the random, spurious, and often unwarned changes they may make. It also enables you to place guards against the various stupidities they may likely perpetrate in the name of fashion or unthinkingness, and ensures that you are - as much as possible - forced to deal only with your own stupidities and unthinkingess.

This injunction includes decoupling from your own APIs where those are non-core to the subsystem under design.

Housekeeping Note - Server Change

Explaining why I've been so quiet lately: Migrating data and upgrading the software that runs the blogs and farm site (plus a bunch of other stuff) to a new server.  Yay upgrade!  Boo problems!  Just in time, too, it would seem, since the old server started mysteriously and frequently rebooting for no good reason, so I'm pretty sure that its been down more than its been up for about ten days now. :-(

Sorry if its all been very dodgy.

If anybody notices anything noticably untoward, please let me know -- I think I've moved everything over successfully, but not yet 100% sure, but, with the old server dying, I just want everything off it as soon as possible, so haven't had time to test all my new configuration properly.

Web Site Passwords

"Signing up" for yet another something-social-facebook-wannabe website, I was struck by a random Thought Particle.


Why do all these websites ask me to enter a password twice?

No, seriously! I know the stock answers. Hell, I've written such web-signup forms myself, more times than I care to think about.

Am I that likely to misspell a password? And who would care, when all I have to do is click a link that says something like, "I forgot my password!" to get a new password sent to me. Or a reminder. Or my original password. Or some other way of recovering from my "spelling error".

So, tell me again, why are we typing these things twice inthe first place?

The Magic Key to Hiring Software Developers

Over the past several months and a half I seem to have run across a lot of Development Managers¹ who repeatedly and quite consistently make poor-ranging-to-terrible hiring decisions about prospective developers.

One company I know about has had such terrible luck in hiring developers that they're looking at outsourcing all their development needs. And they're a software house! But you can hardly blame them for feeling demoralised and dispirited after numerous bad hires. Or can you...?

I recently met a programmer - let's call him Arthur² - who was looking for some Java training. He has experience developing in C, and was looking for a basic Java foundation course, but he needs it to be spread out over evening and weekends. I found this to be admirable! A programmer investing in broadening his skills in his own time and at his own cost!

I don't normally take on that sort of part-time training, but decided to try and assist a fellow Seeker In Pursuit Of Excellence, and engaged Arthur further on ways we might be able to work together. Especially challenging, since I am not even within 500km of the same city as Arthur.

As the emails and telephone conversations flowed back and forth, I soon developed a sense the something was amiss. It took a while, but Arthur eventually told me that he needed the training "urgently and quickly". The penny dropped...

It took me back to a day some 8 or 10 months ago when, whilst helping a client with some recruitment interviews, I came across a candidate-programmer who clearly was unable to program at all. A fellow who allegedly held advanced degrees, and could certainly talk a good project, but who evaded and avoided any and all questions about actual programming. Asked anything about how he would design a programme for a variety of small and typical programming problems, he ducked and he dived, twisted, turned and blathered. Clearly he was unable to write code at all.

It is my working hypothesis that Arthus has talked his way into a Java development position, but is unable to code in Java. By his own admission he knows nothing of object-oriented design. I further conjecture that he may be unable to programme at all, and is not so much seeking a Teacher, but rather wants someone to whom he can effectively sub-contract his daytime work.

There are two mysteries wrapped up in these incidences.


How do such "developers" get hired in the first place?

Clearly the managers hiring them never, ever ask them to write code at any point in the recruitment process.

I have seen a good number of articles of late urging companies who are hiring software developers to make sure that candidates write code live, sometime during the interview process. Isn't this obvious? If you want to hire a bus driver, isn't it logical to put them behind the wheel of a bus and see how they handle the job? Why do we apply a different standard for testing the competence of software developers?

The actual form of code, and the depth of testing can vary, and need not be done all at once. You might simply ask for a whiteboard sketch of a solution in initial interviews. It may not always be necessary to sit candidates down with an IDE and some of your senior developers. Even the most superficial competence checking will quickly revela any bullshit artistry, and I firmly maintain that all people are possessed of exquisitely sensitive bullshit recognition skills.

In a sense, though - for me, anyway - there's a weirder question...


How does someone completely lacking competence in a skill have the chutzpah to talk their way into the job at all?

I can't imagine applying for a job as a Bus Driver. Sooner or later someone's going to expect me to actually drive an actual bus. And I can't. What makes people think they can get away with it just because the job ad says "Java Developer"?

Moral of the story? Just repeating what so many others have already said, in the hope that we can get the word out better. And we do have to get more hiring managers to pay attention, because evidently a whole lot of them are not paying attention yet. If you're hiring a developer, have them write code in front of you. If you really don't have the skills to judge their comptence, and you're starting with a completely new technology so that you lack any already-hired developers who do have the necessary technical skills, then get an outside consultant in to help. Or something.

But please stop hiring bullshit artists who claim to be software developers.


They're giving the rest of us a bad name.


Update: Also beware of technical Trainers who are unable to show you any of their code. Another smell of bullshit artistry!

---

[1] People acting in the role of Dev Manager, at any rate.
[2] I don't know anyone named Arthur, so I'm reasonably sure I'm not subconsciously picking on anyone.

shrtn: A URL-shortener

Everyone should have their own personal URL shortener, shouldn't they?
I figured that this wouldn't take more than a couple of hours to write. And, indeed, the core functionality didn't take much more time than that. But then we start designing our way round the shortcuts and quick-and-dirty hacks we've used to "get things going quickly", writing unit-tests and comments explaining our thinking, adding some JSP pages so that we can exercise the whole mess, brewing a couple of batches of beer in between times... let's just leave it at a little bit longer!


Why?

Indeed! Why would anybody want Their Very Own Personal URL Shortener?


First: I don't really trust all the "cloudy" hype going around right now. For a start, I have no good reason to trust bit.ly, is.gd, goo.gl or any of the other several-dozen public shorteners. Not that I have much reason to distrust them, but really, I don't know them or the people behind them from a bar of soap. And why should I, like a sheep, participate in generating value^[1] for someone who gives me little or nothing in return aside from a shorter, opaque URL that requires an extra network round-trip? And let's not forget that these entities have a nasty tendency to vanish, sometimes rather abruptly. Companies get bought and the acquiring company borgs the product, or sees no value in it, or any of a thousand other corpthink accidents may happen.

Then, too, what sort of assurance do I have that I'll ever be able to get my data (and if I shorten a reference, it's my reference) out of their service ever again? Granted that Google does make some effort in that direction (or at least nods benignly while their engineers do it), but, like the actions of the kakistocracy throughout history, things are only good until a single bad apple rots the barrel.


Second: I don't have PHP deployed on my servers and have no wish to add to the system-administration burden I already have to deal with, so I distinctly want something written in Java...


Third: A whole lot of the URL-shortening services out there don't give any analytics. At least not of those that I can self-host. I think that the analytics angle is compelling. Conventional web analytics - like Google Analytics - are only accessible to the people who created and host the content under analysis. They know where their audience came from, when and how, but nobody else does. If I refer people to some web-stuff^[2] I'd like to get an idea of how many people I influenced - how many people followed my recommendation. It is a measure of my own reputation and influence, so highly personal^[3]. URL-shorteners give us a way to measure, with a reasonable degree of accuracy and assurance, the influence we have in persuading others to follow our webby blatherings.

I will confess that Google's shortening service is pretty good, and has some nice-ish analytics, but I still think it's in our own best interest to keep at least some stuff out of Google's (or anybody's) mitts. Just on general principles.


An Unexpected Bonus

As it turns out, this is a really, really nice project to use for teaching a JSP/Servlet course, so I'll be reversing it into my Java web-dev course. It covers all the principles I like to get across, from container-managed security through session management and clever use of error-pages, to exploiting the underlying infrastructure properly (instead of hoping that some crapulatious web framework will substitute for your own lack of knowledge or understanding.)


Where?

I'll be releasing the code under the GNU Affero General Public License (probably via Google Code). Just have some tidying-up to do first (like getting license notices in place.) The first deployment is very feature incomplete - there's quite a bit I'd still like to add to the app - and some downright dodgy implementation details that need replacing in time, but for now its working for me.


Drop me a line if you're desperate to have it work for you and can't wait...

---

[1] At least I assume they get some value out of hosting their shorteners, otherwise why would they do it?
[2] I despise the word "content", despite using it quite frequently.
[3] And, YES, ego-gratifying^[4].
[4] Or ego-destroying, as the case may be.

SEO Fail

So. I've been working quite hard at getting course material together... mainly for OO Software Analysis Design courses and a Design Patterns course. The motivation comes from years and years of teaching Other Peoples Courses and experience a deep and abiding dissatisfaction with the material I am handed to work with. I strongly suspect that my students sense this...

Not Good!

So - after a very, very long time umming and aahing over it - I've put my own course materials together. Blatantly stealing tricks and tips from the HeadFirst books, plans include incorporating video and even music. Dare I say I've done a whole lot better than the usual run-of-the-mill courseware? Order of magnitude? But that would be presumptuous, wouldn't it?

Along the way I realised that I needed to pay a whole lot of attention to my (business) website. The old one sucked. Bad! Not at all descriptive of what I'm now doing or trying to achieve in the OO design and architecture space. So I spent a little time reading up on the website of that famous search engine about how best to structure my site and its content for searching. How to better market what it is I do. I read all about "not using Black Hat SEO" techniques. I paid strict attention to their advice! And I went forth and restructured... but only in a good way! What you see is what the googlebot gets. No tricks!

The result?

Falling from a page-rank of nearly fuck all, I now have a page rank of... zero. Thanks, Google! I guess I'll take my advertising business over to Microsoft, now.

I can only speculate what might have brought this on. Perhaps I got listed on some directory site that Google's software considers Bad News. That's the most likely thing I can think of... Or did I just use words like "software design and architecture" a time or two too many? Perhaps it was the word "Java"...

Unfortunately - like The Borg - there's no Human Person one can contact at Google to say, "Hey! I'm really a Person. What went wrong here and how do I fix it?" Pretty much I'm screwed. Where my site was ranking very well for searches on stuff like "software design training", "Java training"  and "software architecture course" last week, this week I'm nowhere to be seen. Instead you'll see listing from schmucks offering the same-old same-old. The same tired, half-hearted training crap that led me to start developing my own course material in the first place!

So what's a Real Human to do? I guess I'll just have to live with it. After all, my reputation amongst intelligent human beings is top-notch... not something I have to worry about. So do I need to get upset over how some software see me?

Probably not.

Test-First for Mental Health

I've been thinking recently about how a Test-First approach to development is good for our emotional well-being and psychological balance, and I believe this may be the most important reason to adopt test-driven  development.

Consider how we would traditionally develop software using a conventional Test-Last approach. We read our requirement spec (whatever form that takes), perform some design work -- as much as we feel we need to go forward with some confidence, and we start cutting code. Time goes by. Our codebase grows. We're doing some informal testing as we go along to feel confident that the code under development does what we think it should be doing...

And here's Important Point Number One: "the code does what we (the developers!) think it should be doing". Not what the business thinks it ought to be doing. See? We're inherently unsure -- even if only subconsciously -- whether we're doing work that actually serves the business. And unsureness is insidious and destructive to our well-being. We feel much better about ourselves as developers when we know that we're producing useful stuff that really has value. So here's the first way that having tests defined up-front helps us preserve our mental balance: With a reasonable set of tests1, defined together with our business partners, there's no bullshit involved; no trying to convince ourselves of the (probably shaky) value of our ever-growing codebase. We have an external, automated, objective, easily understood yardstick of value.

But there is, I think, a deeper psychological value to the Test-First approach.

So much of the time -- I will thumbsuck something like 99% of the time -- our code is broken. The system we're working on doesn't work. This is inherent in the fact that it takes us time to design and write software, and the continual not-workingness can be powerfully demoralising. Especially when we (finally, right towards the end of the project lifecycle) start formal testing, and our tester (often users) start coming back with bug after bug after bug after bug, and all that beautiful code we're so proud of is labelled "Less Than Satisfactory" -- usually in much stronger language than that.

Herein lies the secret destructiveness of Test-Last. Our code is always broken, and we can never do more than run after the brokenness trying to patch it up, until, at last, demoralised and exhausted, we move on to the next job or project.

Contrast this with a Test-First pattern.

We first write a bunch of tests. Sure it takes some time, can often be pretty tedious, and is frequently little more than a first-pass best-guess at what the code is supposed to do. Almost certainly we will add more tests as our code grows and evolves and begins to tell us the shape it wants and needs to be. But at least we start with a battery of (automated!) tests. And we start with the expectation of them all failing. This is by definition! We haven't written the necessary code, yet. A short time later we may have enough of the boilerplate in place that we can at least compile everything and actually run our tests, but we still expect them all to fail because our methods all read

throw new UnsupportedOperationExcetion( "TODO" );

So we run our tests, and they all fail. And we're happy about that! Everything went as we predicted. We feel in control.

As we go along replacing those stubbed-out methods, some of our tests begin to go green. We're making visible progress towards a well-defined end goal. We gain an ever-increasing feeling of confidence and accomplishment. We have tangible, objective evidence that we're making progress. Even when we make some of the "test bar" revert somewhat to red -- because we add new tests, or because we do some large-scale refactoring that breaks stuff -- we still feel good about it, because we know that our self-built safety-net is working! We've actually decreased the likelihood of things going wrong, so we know we're enhancing the quality of our product. (And every good developer I've ever worked with has an inner Proud Craftsman lurking in their soul.)

So it seems to me that Test-First builds our confidence and self-esteem as we go along, whereas Test-Last contains an inherently destructive, soul-sapping subversiveness to it.

I think this value outweighs the sum of all the other benefits (many though they are) of Test-First.

---

[1] Note that I make no distinction, here, between Unit Tests, Functional Tests, or System Tests (which would probably involve real databases, external systems, and so forth, albeit serving test data to our system-under-test.) I don't really care what form of Test-First is most useful to your team and situation. It's the principle that counts!

Fusion Cuisine: Slicing and Dicing Courses to Suit Different Palates

Ever been faced with setting up training for some of your developers, but the courses offered by the usual suspects fail to cover the technologies your team uses? Or cover technologies that you don't use? Or lack the depth you're looking for in particular areas? Or too much depth...?

I was recently faced with a request from a client who want to put some of their junior developers through an Intro to Java course. The developers in question come from various other programming backgrounds, and already understand OO concepts and normal development disciplines. The "obvious" choice is the Sun/Oracle SL-275 mainstay course which I have been teaching (on and off) for something like 15 years, now.


But! A good part of SL-275 – and of my own Intro to Java course – delves into developing applications using the Swing GUI toolkit, used to construct desktop applications. This particular client is a company that only does web-applications, so learning a bunch of Swing concepts does them little good. I was happily able to propose that, instead of using Swing for that portion of the course, we could substitute some Servlet and JSP concepts and development. This arrangement serves them much better, as it also equips their developers with knowledge they will be able to immediately put to productive use. Fortunately I have lately put in a lot of work on a Servlet/JSP course, so I'm able to cycle some of that material into the Java Intro course.


It's not as simple as it sounds, though!

The Swing portion of "Intro to Java" also serves some other pedagogic purposes. It reinforces some of the OO groundwork covered earlier in the course by driving home the differences between inheritance, containment and usage. (At least it does the way I teach it!) Swing also provides a good platform for introducing listeners (Observer Pattern in the GoF book), Java's anonymous-inner class mechanism, and the exception-handling framework, along with a bunch of other bits-and-pieces that are useful for new-to-Java developers to learn, like converting between Strings and numbers.

So the challenge, now, is to make sure that I can adequately cover the same territory, and serve the same purposes, using servlets and JSPs. Not too hard, really, but it should make for a fun course where we can do a lot more hands-on, practical development work than the stock SL-275 allows.

The other tricky bit to watch is making sure that the level of detail doesn't overwhelm new-to-Java developers with too many web-application specifics. It is important to keep the focus on "Introduction to Java", and not lose sight of the real objective in a welter of webapp configuration and deployment.

Just a single, simple example of where and when customised courses can much better match the client's needs than any standardised, templatised training provided by a 4-week-per-month manual-pusher.
Contact me for a course customised to your organisation's needs!

2011 Course Planning

Whew! Looks like March is going to be a busy month. I've planned no less than 3 courses back to back, so hard work ahead...

I'll be kicking off with the 5-day "Intro to Java" course on 7-11 March. It seems like Cape Town's demand for Java developers is insatiable. I've had requests from a couple of clients for this course simply because they just can't find good Java devs for their expanding teams, and they're going to be hiring developers with good OO experience from other (non-Java) environments and training them up for Java development as an alternative. Now, I know that any good developers loves learning new stuff, so I think these clients are doing a Good Thing by offering skills-development as part of their hiring process. My job is to give them the best possible Java Platform course, and I have great plans for delivering just that!

Next up will be the newly developed 5-day "Web Application Development with Servlets and JSP". If you've last looked at JSP technology a few years ago, possibly run away in terror at the sight of all the horrible in-page logic that made scriptlet-based JSPs such a nightmare for maintenance, then you really owe it to yourself to get up to speed with scriptlet-free, tag-based JSP development. The course includes topics on the new, new Servlet-3.0/JSP-2.2 enhancements that use annotation-based configuration.

And last, but not least, will be my much-acclaimed 4-day "OO Analysis and Design" course – thus fitting neatly into the 4 working days that remain after the Human Rights Day public-holiday.

I'm planning to run all the courses in Cape Town. (still looking for a great venue, so please drop me a line of you know of something extraordinary.)

Drop me a line if you're interested in any of these. Miss this opportunity at your own risk – I will probably not be scheduling courses in Cape Town again before August!

I'm offering an Early Bird discount of 20% to anybody who confirms a reservation before the end of January!