About that study on the failure rate of projects using Agile…
So recently, there was a study that said software projects adopting Agile are 268% more likely to fail than those that do not.
That’s a bold statement to make.
But before we jump into it in more detail, it’s important to note that the company behind the research, Engprax, does plug their book, Impact Engineering (which currently has 2.8 stars on Amazon at this time of writing) early on. So this could be viewed as a way to get people to buy this book.
The study was conducted between 3rd to the 7th May 2024, with 600 software developers (250 in the UK, 350 in the USA). To me, I find it odd that this study only lasted four days. That doesn’t seem that would be enough time to complete a study and claim that Agile is a cause for projects failing (funnily enough, most sprints last longer than 4 days).
Also, that’s a very small number of software developers to interview. In comparison, the 2023 stackoverflow survey had over 90,000 respondents. To have a better understanding of what people think of Agile versus other methodologies such as Waterfall, this study should’ve interviewed thousands of developers.
This was a part of the report that stood out to me the most:
Indeed, the Horizon IT system was one of the earliest large-scale projects to use an Agile methodology, namely Rapid Application Development, which has been condemned by Fujitsu engineering witnesses in the public inquiry (Terence Austin and the whistleblower David McDonnell) as a cause of the technical problems due to the absence of a robust requirements engineering process. Charles Cipione, the technical expert witness to the inquiry, summarised simply, saying that “if you don’t have a good design, it’s not going to work properly.” The failure to address these issues and cover-up attempts led to the Post Office scandal which has been described as the largest miscarriage of justice in British history, linked to multiple suicides with those wrongly imprisoned including a pregnant woman.
The research also found, disturbingly, that software engineers in the UK were 13% less likely to feel they were able to discuss and address problems than those in the US; the largest difference of all engineering practices between the two countries. The finding comes after November 2023 research by Engprax found that 75% of software engineers in the UK faced retaliation after reporting wrongdoing.
I find blaming Agile methodology for the Horizon IT scandal and retaliation for reporting something to be a huge stretch. There can be so many other factors that should be taken into consideration, such as blame culture, unrealistic deadlines and budgets. One of the key pillars of Agile is responding to change, over following a plan.
I also saw this:
Projects in which the software engineer reported feeling psychologically safe to discuss and address problems quickly when they emerged were 87% more likely to succeed those which didn’t.
This again doesn’t feel like an issue with Agile, more about an issue with the workplace environment itself.
Now don’t get me wrong, Agile isn’t perfect (and neither is waterfall). I’ve seen a few projects fail that have used Agile, but was it to blame? No. It was down to hugely unrealistic goals and budgets.
One of the key points of the Agile Manifesto is to have working software over comprehensive documentation. As software development becomes more complex, and more demanding, the importance for living documentation grows. Being able to onboard new joiners with up to date and accurate documentation is something that I view is as equally important as working software.
The Agile Manifesto was created in 2001, and things have changed hugely since then in software development. Maybe it’s time that there was a version 2.0 of Agile?
