Marcell LippNowadays programmers are hardly working alone, most of the cases they are developing as a part of a team. Sometimes this team has just 3-4 members, but there are several huge projects with over 1000 developers. To organize their work is not easy. Not only from management point of view, but also technically.
First of all it is needed to track the different versions of the software. For that purpose there are several different version control systems (like Git, SVN, Mercurial etc.), but most of the cases it is not enough to have a version control system, you need to also make sure that the different versions are correct.
What is software integration?
Software integrator is a dedicated role at several companies. What does it exactly mean? Different components of the code and different part of the components are done by different team and different developers. They are all doing their part and sometimes doesn’t have an overview on the whole software. So in the end they are all done and they realize that even if their part is doing what is expected their code can not work together. Sometimes even can not build. It is because of different interfaces, different expectations from interfaces etc. Or simply after integration the software is becoming too slow or it is using to much memory. There are also different standards (coding guidelines etc.) which are not followed correctly in all code. These tasks are all the responsibility of the software integrator to make sure each part of the code is fulfilling the requirements individually and then merge them into one codebase and make it work. This work sound maybe easy, but most of the cases it is challenging. Furthermore it takes a lot of time. Just imagine, you are getting the work of several weeks from 10 developers and you need to make them work together...it’s a nightmare, right? Luckily there’s a strategy which make it easy, it is called continuous integration.
What is continuous integration?
The idea is continuous integration is not to wait until the chaos is becoming too large, but integrate the software as soon and as often as possible. Every day, or even after each change. Of course manually it is not possible, so the idea is to automate as much as you can. First of all you need a version control system, which is following the actual status of the code.
There are several checks which you can run automatically on your code base:
- Build the whole code
It is especially useful if you are working on multiple compilation targets, so you can build it for all of them
Run unit test
As already discussed in my post about Test Driven Development it is also good to cover your code with unit tests, so that you make sure that the others who are changing your code are not breaking the already working functionality
Run higher level tests
You can create also automatized component or integration tests and you can run them any time to check if the already implemented functionality is still working
Run static code analyzer
There are several tools which can run a static check in your code. It can detect potential bugs (like usage of uninitialized variables etc.) and it can also check for the expected coding style or give suggestions for more effective code
Run dynamic checks
There are several tools which can detect typical missusages like memory leaking or hanging of the program
Do measurements
You can always measure the runtime of critical scenarios or the memory usage of your code. You can even measure the code coverage
Run additional scripts up to your needs
It is always project dependent. For example in one of my projects we had a script which collected all TODO comments from the code, so that we could make sure before the release that all of them are already resolved.
You can run all these checks in any state of the code and give feedback immediately to the developer team though a status monitor tool or by e-mail notifications. So that the team has always an idea about the current state of the code base. It is a good strategy to keep all tests green.
Scale your system
To run all these checks on your code can take a really long time if you are working with a huge code base, especially because compilers and static code analyzers are usually extra slow. So it makes sense to run everything all the time, because it would cause a too long feedback loop. If you are getting the notification first after one day that your code broke something that does not help a lot. So you need to scale your integration framework.
A set of checks needs to be run after each change of the code base (after each commit), like compilation and unit test.
You can also set up a nightly job which is running several scripts every night on the code, so that the developers are getting feedback every morning: compilation, unit tests, higher level check, some part of the static analyzers.
Plus you can set up a job which is running all the checks, but this one you are letting run only for example once a week or before a software release.
Use frameworks
You could implement it all yourself with some scripts, but fortunately there are several frameworks which are already supporting this kind of functionality.
One of the most popular is Jenkins. First of all it is free. It is also very flexible. You can set up several kinds of jobs with several trigger actions. You can setup also different kinds of notifications. It has a plugin for several external tool (GIT etc.), but you can integrate any tool with a short script on any scripting language (python, bash etc.). It is created nice reports and diagrams which you can just show to your management.
What I want to say here is that you can setup such a system easily and fast and it makes a lot of benefits.
Define your integration strategy
You need to also design your integration strategy. If you are integration every change immediately to the final software then what can you do if it is breaking the build? It needs to be fixed and in the meantime everyone is blocked. That’s why it is a better idea to run first the checkers and integrate it to the main version only if all checks based.
There are also processes where you are developing the different components independently and you are only merging some dedicated “release” versions of the to the main software.
And there are much more strategies. You need to decide what are you merging to the main software when and how. Most of the strategies has its own advantages and disadvantages. You need to define which one is fitting the most to your project.
