At work I’m are engaged in the maintenance and development of a pretty big legacy monolithic application. Some time ago team size basically doubled. To keep things simple and activate more people we decided that we need to split the huge team into smaller more focused parties. Here is how we are working on many features of the same code base in single monolith with multiple teams.

First of all couple of assumptions that you should be aware of for which we’ve planned process:

First thing we noticed after split was the last pinpoint from the list above. One of the teams is working on big refactor of the application and as a result we produce some risky code that tends to get back from testing (different story). Because of it we’ve stopped deployment of other team’s work once or twice. To address this issue we had to rethink how we work with git.

So up until recently, we’ve been using simplest flow possible with gerrit. We had master branch and when getting closer to planned release we’ve been creating release candidate branch (rc) on which we’ve been stabilizing application (we are still lacking automated tests to do CD). In the meantime, ongoing development has been done on master branch. After the release rc branch was merged back to master and cycle repeated.

It was working ok with one team, but with two teams it was getting a bit out of control and has been causing some problems. As a result, we came out with a new flow aimed to solve this issue.

It is based on git flow and has been created with the independence of teams in mind.

We still have master branch but now it is meant for tested and verified commits. We still have release candidate branch to stabilize application before release (it is complicated…​). Each team has their own development branch on which they do day-to-day development.

From a developer perspective it looks pretty simple. I’m working on task ABC-123 I do some changes and push changes for review on branch development_a. When CR is done, change is accepted and lands on branch development_a it is my responsibility to deploy this version on environment_a. Now QA team checks the change and in case of any problems, it’s my responsibility to fix any issues. To this point, it looks pretty standard we’ve just changed branch name from master to development_a.

Things are getting a bit more complicated with release. A person marked as release manager aligns with teams and QA team on what is ready for the release (at this point it is ok for the team to say we don’t want to release now). Release manager merges ready to release development_X branches into master branch, resolves any conflicts (or finds people who will know how to resolve them). If everything looks good she creates release candidate branch and deploys it on the pre-prod environment. Next QA team to run all necessary tests on this version. At this point, all changes prepared for the release are already tested and everything should work just fine, so this step should more like a sanity check. In case of any problems on release candidate branch release manager fixes those changes (or finds people best qualified to do the fix). If possible fixes are applied on rc branch and testing process is executed again. If everything works then we do production deployment from rc branch. Release manager then merges rc branch back to master, and master branch to development_X branches. With next release cycle repeats.

We decided to keep master branch in the middle of the process because we want to encourage teams to merge their ready to deploy things to master branch as soon as they are tested and ready. this part of the process is not yet working for us but I believe we will get there. If not, we’ll just cut out master branch from release process and merge changes from development_X branches to rc branch and update master branch after the release.

Support of the application is also team responsibility and for any urgent fixes we have rc branch on which we can apply the single fix and deploy it on production without going through the whole release process.

As you can see it is not as simple as with one team but in reality, it is not as complicated as it sounds because as long teamwork is planned just right we do not have serious conflicts when merging changes. After running it manually a few times we will probably introduce some automation to the process. For now, we decided to do it by hand and give it some time to settle in and find out what needs to be improved.

The hidden agenda behind this process is that:

This is a living process and we are aligning details on how we work as we go. We tweak this workflow to our needs but this something we’ve started with and for now it works fine for us.