SOFTWARE CONSULTANCIES that are running on RUP have it all figured out: analysts analyze, programmers program, testers test, and project managers… well, who knows what they are doing anyway.

I am starting to perceive this division of labor as inappropriate.

Obviously, it has its benefits. The craft of software development requires so much ongoing learning that nobody could keep up with new languages, frameworks, and methodologies while working on projects and be a specialist in all disciplines of software. Hence we say, let analysts work with requirements, architects to turn them into design, coders to translate that in the appropriate language/platform, and testers make sure it all works reasonably well. Seems like a logical choice, doesn’t it?

Here is the drawback: by drawing thick lines between the disciplines, we emphasize their differences and make them separate while the opposite is desirable. The customer doesn’t need a requirements matrix, UML diagram, nor test scripts; and sure as hell he doesn’t care about source code. He wants to see his processes automated, his customers’ data kept clean and safe, etc. That won’t happen unless all roles involved in the delivery process know about each other.

True, most testers know that what they test is a program code, and likewise, analysts tend to have an idea about the program architecture they are implying when drawing complex class diagrams. I am not saying they don’t know anything; I submit that they know far less than they should.

A couple of years spent charting UML abstractions can make one think writing code is easy: just lay down the bricks as prescribed in the analysis document, and everything will be fine. Or, looking at it from a different angle, writing J2EE code for enterprise applications, year after year, can make one conclude the business proprietors are a bunch of clueless, desperate egomaniacs willing to throw millions of dollars at the latest buzzword (think SOA, think CRM, think BPM) hoping it will automagically generate satisfied customers, increase in sales, etc. When people don’t know the whole story, they can come up with some real strange ideas.

Writing good software is hard. Making it serve business needs is even harder. It takes a well-defined and reality-checked process, fearless execution, and lots and lots of guts to deliver. Here, I submit that all the people involved in software delivery need to be intimately familiar with the whole process in order to produce something useful. Sounds trivial, sure, but isn’t as obviously accepted as it should.

To anyone wanting to start a career in software, I’d recommend this: try to get your hands dirty. Try coding, analyzing, testing, and if they let you, managing, too, though I am a big believer in servant leadership when I think of software projects, so you may want facilitating (instead of managing) projects. Be a generalist: whatever you may want to specialize in will one day be obsolete anyway.

And when you find what suits you best, stick with it; if it’s Java or Rails, great, if it’s making sure the app does what it should, that’s cool, too. Just remember what comes before you and what follows when you think you are done.

I DELIEVERED A UML WORKSHOP to a group of my colleagues on Wednesday. They were all eager to participate and we went through many exercises. Given a rather vague assignment, they grouped into small teams, thought aloud, communicated beautifully, and came up with reasonable diagrams.

Throughout the workshop, we tried to design a web portal for a travel agency. I was as direct and clear about what I wanted as a typical stakeholder would be (read: not much). They were supposed to figure out what it was, and use their freshly acquired knowledge to create a model that a development team could use for the actual work.

Their Use Cases were more or less similar. Further into the maze of UML 2.0, similarities ended. The more tools a diagram offered, the more creative my students could afford to be. And they did: each activity and sequence diagram offered a unique view on the system being modeled.

UML is a language. It’s an abstraction layer that’s supposed to cover the underlying reality. And as every abstraction, it’s leaky. Some would argue that code is leaky, too, and I agree. The problem is, if you add another layer, are you moving towards or away from the reality?

Methodologies that are using visual modeling are assuming that visual representation of business requirements expresses the stakeholder’s intent better than his or her words. They might (not everyone has the writing abilities of a seasoned analyst) but they ultimately don’t tell the whole truth. They are useful abstractions but they are leaky. They cover just as much as they reveal.

It has been my experience that visualization is most beneficial at the beginning of the project when people – both stakeholders and the development team – need to find common vocabulary, agree on goals, and unleash their imaginative powers so that unclear needs can be materialized into working software requiremens that a programmer can solve. It’s the code that matters, and the more quickly it can be delivered, the better.  However, relying on UML (and other visualization methods) to provide the complete truth is just as time-wasting as writing tons of specs.

THERE ARE 2 TYPES OF PEOPLE WHO PROVIDE YOU WITH REQUIREMENTS:

The first group has a vague feeling about their need. You need to find the spot that itches and scratch it. They’ll expect you to know what they are thinking and won’t talk to you unless you are finishing their sentences and materializing their ideas on paper. They are easy to deal with when you’ve done your share of similar projects and can be their partner.

The other bunch is tricky. They may or may not know what they need but they sure as hell will tell you how you are supposed to do it. Example:

You walk into a meeting determined to (finally) get to the matter; if it takes interrogation of the 3rd degree, so be it. You’ve recently had a training of negotiation and other soft skills, so you are pretty confident you will, at the end of the day, know what they really want.

After a short chit-chat, they cover the table with Visio printouts and tell you where the buttons should be, what color, how will Systems X and Y integrate, how exceptions should be handled (the form will turn red and the user will click away a series of error messages…), et cetera.

Now, what’s a BA worth his name to do?

Talk about requirements, and you annoy them because they obviously know what they want and you are wasting their time. Concede the point and discuss their design, and you might end up with a solution that – surprise! – doesn’t serve their needs in the end. Why? Because their needs were so obvious they haven’t been tested.

Which brings me to my point. Business analyst is a natural partner and advocate of business needs and should fight for their fulfillment with both PM’s eager to slash the scope and developers who just want to do their thing. And as a good advocate, he has the right to extract and validate any information from his client that he needs to build his case. And damn well test it is since the opposing counsel will use whatever weakness he can find.

This isn’t about “translating” business requirements into technical specs, as the popular misconception about business analysis goes. Translating would mean taking whatever the client says and telling developers to build it. But if you do just that, you are wasting the oxygen in the room since developers could take such orders from business directly – who said they don’t speak plain English?

So – burn those Visio printouts. They stand in the way of understanding and obscure, not convey, what the needs are.

I’VE COME HOME from a six-months long assignment in Bratislava, Slovakia, and while I am not at liberty to discuss project details, I’d like to share a few bits that I’ve learned there. It might be relevant to you out there who are also engaging in requirements engineering and/or software delivery in general.

And no, I haven’t accidentally re-discovered America: it’s been yet another experience in a series of many, and this just happens to be the first time I talk about it.

I’ll try to put out a short series on the software delivery process as I see it from the Business Analyst perspective; this installment will target some process-related issues and likely be updated as I think stuff through.

PART ONE: Process

A well-defined development process that doesn’t stand in the way of innovation and at the same time puts clearly defined limits on it is … a Holy Grail, and as such rarely achieved in a production environment. Nor is it often attempted.

In my capacity as BA, I mostly work in a documentation-heavy, top-down process context, and that’s when I am lucky: it’s not uncommon to work in a completely ad-hoc environment.

Worst of all, many clients will have rigid rules as to how they want things done without thinking through what they should want in the first place. The developer is then put in a position of very limited power without having been given clear clues as to what he’s supposed to accomplish.

As a result, the business analyst has no choice but to define the process himself and fight for its implementation with both the suits who are used to being able to push requirements into development at any stage, and the IT that is either defiant and unwilling to lift a finger or lethargic after losing way too many battles.

You could object and say it’s the PM’s role to worry about the process, to which I say: it can be, but look at the big picture: PM isn’t as concerned with requirements as he (usually) is with putting checkmarks next to tasks and walking importantly through the hallways. (some offense intended).

It’s my prelimiary conclusion, then, that the BA is most competent to define the process (from requirements to implementation) where there is none, since he has intimate understanding of what the suits want and what the developers are capable of delivering. Unlike the PM, he is also concerned with the value being created and delivered as he is personally invested in seeing it materialized.

Next: requirements versus design (tomorrow).

Technorati Tags: software development, business analysis, software delivery process, sdlc

HOW TO PREDICT THE OUTCOME of a software development project? Raganwald suggests, among other things:

Sitting here typing this, I think the company who can do the best job of predicting the outcome of software development projects is Inkling Markets. That’s because their entire business is about finding a way for people to communicate what they really think of something, not just what they think other people want them to say about something.

to which witten adds:

However, I think there’s an even easier solution. Simply ask people what they think! No need for fancy buying and selling of virtual shares. If the project manager just went around periodically asking people working on a release about their opinion on the state of the release, a lot of the bottled-up warning signs and valuable risk information could actually be communicated appropriately, well before the post-mortem.

Out of many ways to estimate whether a software project will be delivered, I think polling people involved might be the least reliable one.

Which doesn’t say much about, say, developers’ ability to predict their outcomes as it does about the collective psyche of a development team.

Not to suggest that good communication isn’t essential for every venture, including software project. But you know that people don’t just communicate to exchange information; there’s a lot of buying, selling, and power gaming involved.

So asking a developer involved in the project what he thinks is the success % triggers a method that ultimately produces a value designed to cover said developer’s ass while not really saying much about the project’s chances.

Holding a poll among kibitzers and bystanders might yield less biased results. If your firm has 100+ people, betting on projects would bring new energy to the watercooler chat, though I’m not sure how well would you sleep if you knew 70 of your coworkers say you’ll fail. In the end, such a game would bring more harm than good.

A possible solution would be to let the customer bet on your success. Instead of your PM giving a boring Powerpoint on a weekly status, the customer’s rep would give you an up-to-date score of your project based on the customer’s perception. And how could that be boring, ever! The customer isn’t a disinterested observer, but politics notwithstanding, he’s in it to win. And given the chance, he’ll tell you how he rates your work, no problem.

The question then is, do we as consultants, contractors, vendors ever dare to ask?

ONE THING I DON’T LIKE about working for large organizations is the moment when something goes amiss. This one requirement that wasn’t analyzed enough blows out in your face, or that bug you thought was fixed turns out to be alive and well, and all of a sudden, there are meetings, mass emails, other meetings, and yes, yet more meetings to come.

All orchestrated so that the person responsible is found. Why? If the problem is serious enough, there may be consequences, if not, the goal isn’t really finding the guilty party but making sure that everybody else is innocent. Then everybody laughs and the meetings are over, without any follow-up as to how to prevent future mistakes of the same kind.

Either you are serious about finding the culprit and teaching him a lesson, or you realize that software development is part craft and part magic and that mistakes just happen; either way, there is a point to be made. But this?

Small teams do both: they recognize individual contribution and the inevitable shortcomings of every member. Larger groups tend to focus more on member safety; there are business goals, sure, but the primary goal is keeping the status quo. After all, the company is big, so we might just as well do our 9to5 and go home.

It pays to know what your client’s culture is. When it’s an agile, lean operation, quality just might be your job #1.  Other times, you as a software developer, project manager or analyst aren’t really an IT worker but a therapist. Just make everybody feel good about themselves: that’s often what our work is about.

I’VE ALSO HAD A PROBLEM classifying the role of Business Analyst as a bridge between business and IT, but if the Agile movement says IT should understand what business needs, I am not hopeful that just being able to come up with “better solutions than the business people can develop on their own” is good enough. Either business people will never understand technology well enough or IT folks won’t get the business needs well enough; but then, there would still have to bea bridge to cover the gap.

Indeed, when BA is staffed in IT, he becomes a documentor, an interpreter, and above all a tool of IT interests.

Therefore, I am convinced a business analyst on IT projects should be a member of the business team and as such play under their colors – as their advocate. If it means going head-to-head against IT, so be it. So often the programmers know full well what the business *really* wants, and don’t do it because it’s too difficult, costly, etc.

That’s not to say BA should necessarily push all ideas regardless of their merit; as Kevin says, he should be able to propose better solutions – based on his in-depth understanding of his colleagues’ needs. He should work with requirements as every good advocate should – with his critical faculties turned on.

Technorati Tags: business analysis, software development

Johanna Rothman reminds us that testing is not a service but an integral part of development. I would go even further and say that testing is – or should be – excercised regularly at every stage of a software project, and not only there. Testing provides the much-needed touch of reality for our goal-driven activitites. That’s to say, as soon as we define goals, we need to test that they are both desirable and attainable, then devise ways to reach them and again, test them, and so forth.

When testing is left to testers, the following will happen:

• the analyst won’t bother reality-checking the requirements
• the programmer will be casual about the quality and robustness of his code
• the business folks won’t start reviewing the product until it’s been developed

It’s much more costly to change things at the end of the project that at the beginning, and it’s been the agile methodologies that have tried to answer that by giving everybody involved an early look at the product being developed. But that’s not enough. Before a project is conceived, there is a business need that’s supposed to be satisfied, and quite often its importance is sadly overrated.

The biggest tragedy of failing enterprise projects isn’t that they aren’t delivered on time or on budget. It’s that they’ve been started at all.

Sometimes, there’s too much money and too little time to think it over.

THERE’S ONE ROLE missing in all software methodologies known to man.

The therapist.

There are analysts, programmers, project managers, and of course the “stakeholders” – the people footing the bill. And they all want the thing developed, delivered, and used. Yet, come the migration time, the Big Bang roll-out, blades are drawn out and mouths get dirty and there’s yelling and ghashing of teeth.

Because no one really knows if the thing is going to work.

The uncertainty is driving people nuts. Even if all known bugs are resolved and closed, there’s still plenty of unknowns. Maybe the testing wasn’t as thorough or honest. And maybe we haven’t thought of all the use cases.

And with the KPIs and quarterly bonuses on the line, no wonder that putting a system into production is such a drama.

It’s as if you built a bridge not knowing the first car to cross it would make it to the other side.

But you know what? Rarely do information systems have such a life-and-death import. Dollars might be at stake, lives – not as often. Which is why, at the end of a project, all team members should have a sit-down with a licensed psychologist who would tell them: good job, and if you screwed up and that printing routine isn’t going to work, too bad but life goes on. And you’re gonna fix that on Monday.