How to handle software requirements change after implementation has begun.
There is only one constant in software development, things are going to change. This is an immutable Law. The first step to properly handle “change” is to accept this fact. It is so important, that even in the Agile Manifiesto principals you can read: “Welcome changing requirements, even in late development”. So, they are not going away, but that doesn’t mean that you can do nothing about it.
Let’s address why it happens first because the people that don’t learn from history are condemned to repeat it, then how to handle them in a way that doesn’t create headaches.
To the human mind, it comes very naturally that when it is presented with a problem it will come up with a solution. The inexperienced managers implement control over their departments by this approach, as soon something comes up, they will tell what to do, they figure out everything on the fly, this is called reactive management or as I like to call it Cowboy management since everything is improvised. Is not possible for any human to foreseen what situations we are going to be presented to handle, so this management style comes naturally. People that work in hotels will tell you strange stories about how they are always surprised by the kind of things that happen on the hotels no matter how many years you been working on that industry.
So, it is not possible to see the future, but the mind is also the greatest pattern detector machine in the universe, it can find patterns even in places where there isn’t. Over time you will start noticing things that are repeating, let’s say we can see a cashier being paid by a customer with cash or credit card (it seems obvious and dumb but hold your horses). We cannot see the future, but we can see the pattern so if a new cashier is hire, we can let him find how to handle this by himself when the situation occurs (reactive management) or we can create a little document and a training to let him know in advance what to do (proactive management). This is how processes are born.
Processes are the foundation stone of all enterprise software projects. Software developers create tools to let users do their job easier and quicker based on these processes since no computer on earth has the capability to come up with a solution of unpredictable situations (at least by the time this is written). But the responsibility for those processes to be done properly is the person paying for the development of the software, not the developers, project managers, team leads and even the product owners which have the responsibility to document and manage the client’s processes for the project.
This is that way because, for example, if we are creating a software for accountants, nobody on the development team will be an expert in the subject to be able to tell how should the software handle the accounting, so is nonsense to make responsible to any of the development team for the processes, they are responsible only for the technical part and to properly implement the processes (also called business rules). There are some cases where the developers have made software for the industry during a lot of years and gain a lot of experience in that industry, but that is out the case for the scope of this.
When someone writes a new process, we say it is immature, because we only address the most common scenarios. Then the cashier is presented with new situations: a client wants to pay one part of the bill with cash and the other part with the credit card, very obvious but our software can’t properly handle that, so we are in trouble and we need to change the process, so it works properly. Then the store wants to accept coupons, then we need to accept mixed payments with coupons, cash, credit card, then the customer wants to use more than one credit card and then a new Law is passed, and we need to implement what the law dictates (the last one happens a lot in Mexico with the invoices).
So, things can get complicated easily from an apparent easy process, it cannot be predicted, foreseen or be prepared. The more situations a process can handle the mature it became and usually, I have seen that that takes at least a year, any software project is at least a year project even if we deploy in 3 months. Still, even with a high maturity level something unpredictable will come up that the process won’t be able to handle, and sections of the process also can become obsolete and had to be removed.
So, I hope by now is clear that not having changes during the development of the software and after is completely is out of the table. But that doesn’t mean that we can do nothing about it, it means that we need to integrate how to handle change to our work.
Rookie Projects Managers tent to make project plans like nothing is going to be wrong during the project and the plan will be executed flawlessly. It is known that ships are out of course most of the time during a trip, but still, they manage to get to their destination. How they do that? They make corrections during the trip. That is the first and biggest secret to handle changes. You must have a process to properly handle them.
There are two ways to handle changes your own work processes, aka the methodologies and the software architecture, the technical part. For the technical part, I already write something about here: How do you write code that is extensible and maintainable when you don’t know what new features and changes will be demanded in the future?
On formal methodologies, the first phase of software development is the Analysis, and after it is done, it will produce all the documents needed for the project. If something changes after the analysis phase there was process call it “change request” to handle those situations. A lot of literature has been writing about this but the problem with that approach is that is very bureaucratic and time-consuming. Also, a lot of people are not properly trained and poorly execute this methodology which leads them to believe it doesn’t work, but it does. For big projects, it is the way to work but for small or medium projects is overkill, I will not recommend it in those projects.
In Agile methodologies, it is simplified everything a lot. A product owner will create stories out of the requirements which are immutable. If you are coding a module and during the User Acceptance Test is found that the story is incomplete or wrong, as long the developer does everything that was specified on the story, the story is closed, and mark as done, then a new story is created to handle the changes. For small and medium projects this works like a charm, but in large projects, I have seen that is not the best approach since the number of changes increases exponentially with the size of the project, which leads to having a software refactor for each new story. The bigger the project gets, the harder it is to make refactors.
The only thing I don’t like about Agile methodologies is that they don’t put a lot of emphasis on the processes analysis and technical design (aka software architecture), and that is fine if the client has mature processes documented and the projects are small to medium, but for clients that their business is based on reactive management the project will be a sure fail and big projects will suffer a lot with the code refactors.
There are things that lead to a lot of changes out of the scope of the methodologies. One is clients trying to implement improvements in their working processes because now we are creating software for them. Usually, these projects fail. Creating a new process is not an easy task, as we discuss. The process tends to get complicated very easily. Implementing working processes on the software es hard and expensive, trying to make improvements on the fly will lead you to mature those processes when trying to implement on production the software. Not having this vision will lead the client to have a false impression that the developers are doing a bad job. I usually don’t let a client try to improve the processes, I always say: Let’s first make the software work with the current processes you have, after we implement it then we improve all you want.
Clients who use reactive management over time start to have a lot of problems to control their departments. It is very common that wrongly think that buying software magically will solve the problem instead of implementing processes. You end up, not as a developer, but as a processes consultor which is not your job and will suffer a lot trying to get things right. It is like shooting on the blind to a target. For those cases, I recommend this technique When to start a project by the Front end. — Alfredo Pinto Molina — Medium
Also, as you can see, you can have hundreds or thousands of different scenarios for a given process. You and your client need to be very clever on this, not all the scenarios necessarily need to be implemented or are worth it. Once I buy an item on Amazon, and when I fulfill the address, I make a mistake and put a wrong street number. I communicate to the vendor to correct the mistake before the item was ship so it won’t be delivered to the wrong place. In this case, the vendor instructs me that the order will be canceled, the credit card charge will be refunded and I will need to start over. For me it was like: It is just a simple correction, why cancel the whole order instead of just correct the address since the item hasn’t been sent. Well, let’s analyze it for a moment. Remember that every scenario must be coded, and coding costs a lot of money. If that scenario only happens very rarely it may not worth the money and the time invested on the coding to handle that situation, it will be cheaper to just cancel the order and hope that the client buys again the item (which I did and didn’t mind). This shows us that the client and the developer's team needs to be very clever in which scenarios worth implementing to have a return of investment (pick your battles). It is better and cheaper to simplify the process than trying to make it robust.
So even if both methodologies have their pros and cons you must see that having a lot of changes is not about methodologies, it is about three things: Poor quality requirements, poor project planning, and poor plan execution. The changes are going to happen no matter what, but you can impact the amount of them during the project by gathering quality requirements. So, it is a lot better to prevent changes during the analysis of the project than being very good at solving changes during software development.
I always recommend these two books that address the requirements more broadly: Software Requirements (Developer Best Practices) 3, Karl Wiegers, Joy Beatty, eBook — Amazon.com and the book Exploring Requirements: Quality Before Design: Donald C. Gause, Gerald M. Weinberg: 9780932633132: Amazon.com: Books
