Software development projects can transfigure businesses and transform their prospects. They can enable a business to address whole new markets or shave thousands of worker hours off tasks. They can also fail; in 2020, up to 66% of software development projects were partial or total failures, with larger projects more severely affected. We’ll talk about why that might be in this post, along with the main reasons why software projects fail and what developers and businesses can do to increase the chances of success.
Scope creep
Scope creep doesn’t just kill software development projects. It’s a clear and present danger to any type of project. Creep refers to an engineering problem: under pressure, materials don’t just bend or stretch. They deform fundamentally, moving away from the pressure, like this:
In effect, by applying more pressure than the material can cope with, engineers are accidentally shaping it into something else.
Scope creep is when that same thing happens to the scope of your project. Maybe you have some definite plans established. Let’s say you’ll have a clearly-established, understood scope for the project: a couple of core features, a dashboard, and an API. You can budget time and effort for these fairly accurately. There are always changes and circumstantial pressures, but you can factor a lot of those in.
Then you hear from the client. After a chat with someone else in their company, they’re wondering if one of their features can become two features. Can the dashboard be permissioned, with administrator privileges? Can it draw data from outside the application? That would be great too.
Scope creep means more work in the same time, unless it’s addressed clearly. Developer teams have to be able to ascertain what a client’s ultimate goals are, consult with them about the best way to achieve those goals, and manage expectations going in. When clients start creeping, developer teams need to do all that again; otherwise, dev teams work overtime to deliver unrealistic goals and everyone is disappointed. When this process goes wrong it can lead to budgets and timetables creeping along with project scope. Time to market goes up. Pressure rises, expectations aren’t met, and projects fail to deliver.
Unrealistic timelines
Who’s setting timelines for the software development project? Ideally, these need to be worked out taking into account scope (see above!), capabilities, business needs and availability. Product owners and team leaders need the opportunity to check out and assess the requirements, and they need to be able to clearly communicate their thoughts — including any changes to the timeline they may see as necessary.
Without this opportunity, timelines will be prone to become increasingly unrealistic. Unclear or incomplete requirements, or requirements that keep changing can contribute to this problem — especially if the changes require the attention of specialists who may not be available. If the original scope called for JavaScript, CSS/HTML and SQL, but newly-added or poorly-specified features can’t be added without developers fluent in less common languages, the project will be slowed.
Setting impossible timelines doesn’t make things happen faster, and in general it actually tends to slow down development.
Testing and measurement
If a software project consists of a single long iteration cycle with a round of testing at the end, it’s being set up to fail. In the majority of cases, that’s an unwieldy approach that makes it likely the development team won’t have time to fix any issues they do find. However, as agile becomes the norm, this is much less likely to happen.
What is still common is poor, ill-defined acceptance testing. If you don’t have proper test cases laid out, you can’t be sure that testing has been sufficiently thorough and accurate — or even if you’ve been testing for the right things.
The solution to this problem is to invert the traditional approach to test design and build tests first. That means not just before the software is ‘code complete,’ but before coding proper has even begun. Then, you write the software to meet the requirements of the test, then test it. Testing ceases to be a way to identify unforeseen problems and becomes a way to verify planned functionality (though it should also screen for the unexpected!). The most effective way to do this is through automated testing integrated with agile software development, with manual testing functioning as a backstop.
Wrong features, wrong criteria
Building software involves input from multiple stakeholders. There’s nearly always enough ‘nice to have’ trying to make its way onto the spec to keep a team of thousands busy for a century.
Development teams might even add their own as their leads reach for familiar solutions to apparently-familiar problems. And clients can confuse things further by asking for the features they think they want instead of the outcomes they actually need to achieve.
The result can be criteria that select for many features that many people want, but don’t identify the features that are truly needed by the stakeholders whose requirements called the whole project into being.
The solution here is communication in the early stages of arranging the project. Clients need to feel sufficiently safe in their relationships with their developers so that they can have open-ended conversations about the business goals they’d like to achieve, and developers need to take the time to ask probing, concrete questions to build a clear picture of what really does need to get done. After that, it’s a question of avoiding scope creep…
Poor team management and dynamics
If it takes two people a hundred hours to build a wall, how long does it take fifty people? This isn’t a math test. It’s an organizational problem, raised and analyzed by ex-IBM Systems project manager Fred Brooks in his The Mythical Man-Month. Basic logic tells us that each person gets a certain amount of work done in a certain time. Add more people and the same work gets done in less time. Experience, carefully enumerated and explained by Brooks and backed by millions of others since, says it doesn’t work like that.
We can think of productive work as profit. Other efforts go to internal communication, planning, normalization and other activities that aren’t directly productive. We can think of these as overhead. As team sizes rise, overhead bloats until it consumes profit. The point comes where adding more hands slows things down.
Just how severe this effect is can be seriously counterintuitive. The most productive software development teams have on average… 1.5–3 people.
Without understanding this, project leaders can find themselves putting out fires with gasoline, adding more developers to time-crunched teams that are already fatally complex.
Unaddressed technical debt
No one develops software in a vacuum. Every project relies on tried-and-tested libraries, commonly-used languages and operating systems. And nearly every project has to slot in somewhere with a client’s existing tech solutions. Very seldom do you build from scratch on a green field.
However, when you take on a client’s existing IT infrastructure you can also end up taking on their unaddressed technical debt.
Technical debt is the sum of the IT tasks put off ‘til a later date: the aging server racks, rickety wiring and clunky hardware, but also the software that someone else decided was good enough to ship today and fix tomorrow. If they never fixed it, you get to work with it. Developers can wind up jumping through hoops to make sure their clean, human-readable, annotated code plays nice with… whatever that is.
This problem isn’t confined to client environments — Adobe made the whole world suffer this until it retired the notoriously-insecure Flash Player in 2020. But it can slow, or even derail, software development projects if they don’t discover it early enough to either cope with it, or persuade clients to fix it.
How AndPlus approaches software development projects
We’ve worked on hundreds of software development projects, from complex niche tools to large, multi-site implementations that involved input from multiple stakeholders. At the same time, we’ve always focused on outcomes, working with clients to identify real business needs and building software solutions to meet them.
In our eyes, everything else flows from that. Communicating with clients effectively doesn’t automatically solve the problems we’ve laid out in this post, but it does make it possible to address them before they become serious enough to put the project at risk. Careful roadmapping and product mapping, allied to agile development practices and a varied, up-to-date skill base, means we launch up to spec, on budget, and on time.
Takeaways
- Manage scope dynamically, don’t let it creep
- Set and maintain realistic timelines that take account of changes in circumstances, based on input from developers
- Test and measure at every stage, and develop to the test
- Establish correct criteria and design the development process to meet them
- Build small, highly-capable, effective teams, don’t try to solve problems by throwing more devs at them
- Identify technical debt and plan to mitigate or resolve it
