A short journey through some Web Development Project methodologies
First off, this is an opinion piece. It’s not a definitive critique of all development processes, it is merely representative of my experience to-date with the different theories surrounding software/web development.
For the remainder of the article, I'll just refer to software/web methodologies as web ones. I've really only worked on web related projects in my career, but there's pretty much no difference (open to correction here though).
Starting off in my career, I worked in small companies where a defined process didn't exactly exist. It was more doing what was necessary to get a project over the line, and dealing with anything that crops up. You basically hacked at it until it worked. That was what the nature of the business tended to be. Delivery of small, standalone sites, usually with tight deadlines. On top of that, once a piece of work was delivered, there was typically never a chance to make improvements to it.
I've freelanced as well, which often meant extremely tight deadlines and a few late nights, and sometimes unrealistic expectations for a project. There was little room for maneuver from a development perspective. Requirements would change, and generally, deadlines would not. It's not great having no structure to work in. At least within certain methodologies the understanding that change impacts delivery is typically understood. Either the scope is reduced, sprints are extended, or additional sprints added etc.
Apparently, I've never worked in a company who has implemented the full "Agile" methodology properly. I say this as more of a shared opinion, as it is my experience working in teams where everyone has a different interpretation of what the Agile process is. Seemingly, everyone has used a combination of different aspects of it, but in terms of the Agile manifesto, there doesn't seem to be a place I have worked that understands and has implemented the principles of it completely, and that goes for most people I meet (not that I even understand the manifesto completely myself).
I love hearing people's criticisms and comparisons with past projects they've worked on. Agile, it's a nice idea. In almost every project I have been on, the approach has been slightly different. Borne typically out of past experiences. Anything I've read about Agile, it sounds completely logical. The 12 principles and values. The are all very reasoned in my opinion. Unfortunately, the reality is that they are not all followed. So when a company says they "are an Agile" team, it is hardly ever the case where this is strictly true. They cherrypick the desired principles that work for them, and ignore the rest. Mostly it tends to be the business is incompatible with Agile. It's not to say that it could never work, but not without a significant overhaul of most legacy businesses, and how each Department functions. It is especially true when working in a company where software is not your primary business, but rather a way you conduct the vast bulk of your business.
So without fail, issues typically crop up. I'm well used to this now. In a personality test I've completed, the one area I scored lowly on is essentially not liking change. To me, as a Developer, that shouldn't come as quite the shock. It's not that I entirely fear changes in a project, I would happily roll with those. It is just sometimes a major change can lead to the unfortunate conversation that it will have an impact on delivery. People will look to you as the Developer, and want you to say it can be absorbed with negligible impact to the schedule. Usually slurping up some contingency time you've banked for such an event. I wouldn't mind only that a change usually means losing some previously planned scope from the project, and in some cases, a major loss of scope from the Product. As a Developer, we do, much like the Product Owner, have a vested interest in delivering the best product we can, unfortunately, sometimes we can't do that. That is why I dislike change. However, that might be considered un-Agile according to one of the main tenets.
Waterfall / Iterfall
When studying for my degree, we learned about the basics of Development Methodologies. A focus of which was Waterfall. The idea that you have absolutely everything you need up front and a series of stage gates, or milestones which need to be hit to continue to the next phase. If a change happens at a later phase, you essentially have to go backwards to the first phase, and revisit and pass all gates again. This is incompatible with Agile, obviously. It relies on the fact you have all the requirements up front, and absolutely none of these change. In an idealistic world, absolutely no further requirements would crop up after the development phase has begun. The business world is littered with failed software projects, and by failure, there are varying degrees of failure. From complete budget and deadline overrun, to outright non-completion of the project.
Iterfall is an Iterative Waterfall model. Similar to above, but you expect there to be change, and not every stage gate needs to be revisited when a change happens. It tries to be an improvement on the Waterfall model. It kind of ends up being the go-to project methodologies when projects start to flounder, in order to exact some type of milestone orientated approach.
What drives the choice of Development methodology?
Generally, its size and complexity of a project. If you're doing a pet project, typically you're the one man band, there's no strict creation of user stories, designs etc. Everything tends to be in your head, so you fly by the seat of your pants.
If you work for a larger organisation, projects, no matter how small may require a lot of analysis and design up front to assess the impact to the system, in order to give estimates and cost for a given project. What happens quite often is, projects are not kicked off with enough lead in time to properly assess these impacts.
What I find, it tends to become Deadline Driven Development. Someone has a requirement. But crucially they also have a target date for release of said requirement. And often it's not a realistic timeframe in which to deliver all the asks. And what happens is, we shoehorn as much as can be achieved in the timeframe given, but at a detriment to the deliverable. Usually, the Business Owner doesn't get what they want, but they get a part of what they asked, when they wanted it delivered.
Newer companies tend to favour the latest trends, everything is about Agile these days, running a project in Sprints etc. But seldom are companies capable of releasing to production at the end of every sprint, as the work finished forms part of a larger project. However, contrary to that, there are exceptions. There are companies where Agile does work, and they steadfastly adopted the practice. They tend to be ones with a Start-Up mentality. When project milestones begin to be missed, and serious overruns ensue we tend to fall back to the tried and tested Waterfall/Iterfall method, in order to protect the delivery and prevent further scope creep. The latter which often tend to cause the project overrun and stagnation in achieving the delivery goals.
You may disagree with any of the points I've made above, this isn't a complete look at all the nuances of development. And there's always room for disagreement, so no issue whatsoever if you as the reader have a differing opinion, as this is merely my experience looking from a single viewpoint of development, and there's a good chance you've worked in better companies and projects than I have. But at the end of the day, web development project delivery is difficult. People find what works for them. There is no one size fits all solution. Smarter people than I have had projects fail for hundreds, thousands, if not millions of different reasons.
Projects generally consist of four things. The requirements which make up what the aim of the project delivery is, the time it'll take to deliver those requirements, the budget that time will cost and the deadline in which you aim to have the product ready. If there is pressure to reduce time or money, there's a natural knock-on effect on what can be delivered. Similarly, an enforced deadline forces what can be delivered in the timeframe given. It typically does not matter what the nature of the business is, how big or small. The same factors affect project delivery.
My main learning in all of this is to not listen to the snake oil salesperson selling their wares, trying to convince you of a better way to do things. If you've gotten a tried and tested process, and it works for you. As entrenched as that position may seem, generally there is a rationale for it. You're better sticking with what you know, as opposed to reinventing the wheel.