10

I work in the IT department of automation/dev company and we are just about to implement/add CI into our set of tools. And we are having difficulties to choose the one.

At the moment we've been thinking about these systems:

  • Jenkins
  • CircleCI
  • TravisCI

Question: What are key attributes of CI's soft which the systems should have and based on which we should compare them?

EDIT: We are expecting from CI the standard things: From building the applications, running tests (Unit/Integration/Performance/..), saving statistics and providing information either via email/on the page (reports).

Issue there is that we don't know which if there are more functionality and if, what is it? This is what I would be looking for under words "key attributes". The names above are just for reference, I'm looking for answer "why use this or that" over simplified "use this or that".

We are developing in .NET (C#) environment with some libraries written in C++/VBA.

030
  • 13,235
  • 16
  • 74
  • 173
Tatranskymedved
  • 211
  • 1
  • 7
  • 3
    It would help if you can elaborate on what you are looking for in your CI tool. What are your requirements, what are your need? For instance: what languages will you be building artifacts from? Are you interested in a SaaS model or do you prefer hosting your own CI? There are other CI tools out there, why did you nail it down to these three? Giving us more information will allow answers more tailored to your need, otherwise, you'll get generic answers that you can already Google: https://hackernoon.com/continuous-integration-circleci-vs-travis-ci-vs-jenkins-41a1c2bd95f5#.th6ck5zb7 – Alexandre Mar 01 '17 at 06:49
  • @Alexandre Thanks for follow up, I have edited the answer: honestly I don't want to know the difference between each system (or just systems above), I'm looking for an information about what functions are possible and which (from the experience of users) are the most beneficial. – Tatranskymedved Mar 01 '17 at 07:06
  • 1
    Look, your question is very broad because you have not tried these tools. You can easily install them in a container with few commands and try them out. If you have almost no experience, you can pretty much pick at random, get experience with one tool and then if it stops working for you, you can come and ask more specific question on what features you need and which tools might have those or which plugin you need for your current tool to get that feature. – Jiri Klouda Mar 07 '17 at 05:18
  • 2
    With the edit, I am going to for now close this as duplicate of this question: http://stackoverflow.com/questions/1438390/comparison-of-ci-servers – Jiri Klouda Mar 07 '17 at 05:20
  • @JiriKlouda Thank You for follow up! I had in mind while creating to make this not too broad nor opinion based. The difference: All systems considered as CI are doing same work which will most likely not change over time. If You put them on scale, will there always be one winner ? As not, what is making them "special"? This abstract definitions on key points should be always the same and thus not broad/changing. If question would be put in a way "what is better system", "which CI systems do You know" or "what are diff between this and that", that would fit for the flag. – Tatranskymedved Mar 07 '17 at 06:08
  • There is not going to be a clear winner. Some of the 10+ products might start falling by the wayside as the requirements increase, but at the start without any specifics it is hard to say. Most people will never get to a point where the lack of features would even show as significant. – Jiri Klouda Mar 07 '17 at 06:15
  • 1
    The problem with the question asking for key attributes in Continuous Integration software is that you will get the same answer as with selecting almost any others tool. For example avi's answer could be just as well applied to code review tool or version control system or wiki or pretty much anything. – Jiri Klouda Mar 07 '17 at 06:20
  • @JiriKlouda Ok, this is valid argument, I agree on that. – Tatranskymedved Mar 07 '17 at 07:24
  • 1
    good link for jenkins Vs teamcity this will help to descide which CI tool to go for https://www.linkedin.com/pulse/jenkins-vs-teamcity-amir-najjar – Ashish Karpe Mar 07 '17 at 12:24
  • @030 Sorry but I have to pretty much disagree with last edit related to "title change". Original question was related to "key attributes" and now it points to "which CI to use" ~ which software to recommend. As I see it, that part of edit is non-productive and against original question. – Tatranskymedved May 12 '21 at 18:03

2 Answers2

13

In general, there are five main differences between different CI software solutions.

  1. Cost: Is the software open source and free or proprietary? Does your DevOps team already have a budget for software or are you expected to compare free options?

  2. Maintenance: Is the software something you need to host on your own and maintain, or is it being offered as a service? If you need to host it on your own, do you have the team members available who can run the maintenance? If it is being provided as a service, where does your code reside?

  3. Security: Are there any security or compliance concerns about where the build machines are hosted? Are the permissions required by the CI software into your SCM well understood?

  4. Community: Does the CI software have a good community around it with the plugins and support for the technologies you use? Is it easy to get support or troubleshoot when things go wrong?

  5. UX: How easy is the software to use? Does the CI software support scripts stored within your SCM for the tasks you need to do? Is the reporting easy to understand or will it require some training?

avi
  • 1,279
  • 1
  • 13
  • 32
  • Your list misses the point that managed services can have arbitrary undocumented limitations or bugs that can be hard or impossible to circumvent. In my experience, circle CI and Jenkins have this kind of limitations that can entirely negate the benefits of a managed service. – Michaël Le Barbier Jul 31 '17 at 09:23
  • Very good list. I would also add Extensibility/Plugins. Even though you implicitly mention it in community, I would say they can be exclusive because some tools provide their own plugins while they can have a strong supporting community. – gmolaire Dec 05 '17 at 21:40
6

The SCM system you use can be essential in making your CI choice.

Using a private/intranet solution, for example, pretty much excludes CircleCI and TravisCI as these only support cloud-based GitHub and/or Bitbucket.

Jenkins has plugins supporting many SCM systems out there, see Which SCM tools does Jenkins support?. But using a less popular or a wrapped/customized SCM system may require development of a new plugin.

Also particularities of your building/testing framework/infrastructure may matter, for similar reasons (configuration, location/access method and compatibility/support).

Community
  • 1
Dan Cornilescu
  • 6,730
  • 2
  • 19
  • 44