CI/CD Toolchain Discussion

Modern development processes often involve various components known as a toolchain in order to support the automation of build, test and deployment of code.

Even assuming Visual Studio as your IDE, there are numerous ways to solve this issue, the simplest being straight TFS/MSBuild/Sitecore Packages, with the most complex involving many specialised components along the way: TFVC, Subversion, Git, Grunt, Jenkins, TeamCity, VSO, PowerShell, TDS, Unicorn, Octopus Deploy and so on.

I don't believe there is a right or wrong answer here - and frankly I can make arguments for lots of different models.  As always there are advantages and disadvantages.  It's a complex topic with zealots on every side of the fence. 

  • What is your team's stack?
  • What works well and what is still hard?
  • What sort of problems do you routinely face?
  • Was the setup hard?
  • What did you transition from - what did you learn along the way?

Share your experiences with your current toolchain and read about others.  Maybe we can figure out a "best practice" model for Sitecore, or maybe we'll just agree to disagree!

9 Replies

  • How about I kick it off?

    Our setup was originally VSS (no really, but like, an eon ago)  so naturally we transitioned to TFS and TFVC.

    Our current deployment stack makes use of MSBuild, PowerShell and Sitecore Packages.

    This allows us to manage, not just Sitecore content and code, but Azure infrastructure at the same time.

    We are currently on the market for some new tooling: winning this race at the moment is TFS-hosted Git, Unicorn, TeamCity, Octopus Deploy, Azure Resource Templates and Desired State Configuration (which is, of course, PowerShell fundamentally).  The TFS-hosted Git is probably the most contentious item there, and may end up being BitBucket or the like.

    The original stack was designed for simplicity - minimum tooling footprint and minimum infrastructure requirements for clients.

    The new tooling is designed to improve throughput, deliver feature branching and feature toggles more easily, and make the delivery of packages to INT and UAT more robust.  From UAT the process will become manually triggered, but will still operate through automation, giving customers a chance to take pause and signoff in UAT, and allowing us to accumulate changes for delivery into production, so as to not disrupt the live environments too often.

    Azure Resource Templates and DSC allow us to automate the provisioning of our Blue/Green environment for production releases.  Not just new code, new hardware! New everything!

     

  • In reply to Richard Hauer:

    Ended up being too long as a reply cardinalcore.co.uk/.../

    In short - I recommend all the tooling you don't use ;) :D
  • I will add our current stack to this - and a few comments on where we are going.

    We have used CI for a very long time (since 2003/2004) so some of our tooling are now split on old and new. We have not moved all projects - ohh yes, we do have some which has run for 12 years on Sitecore - luckily not version 4 any more :-)

    For version control - most of our code is still svn. We use git on some projects (bitbucket as well as own hosted) that have started more recently. We have tried tfs with some clients but I am on the side of Nate on that one... My choice if I had to do now is definitely git.

    For build - we have a lot still on cruisecontrol.net and some on teamcity, and we have tried bamboo and others. Teamcity would be my platform of choice.

    For deployment we have custom scripts but are moving everything we can to Octopus deploy. We script powershell nowadays - so no big deal here.

    For the infrastructure automation it is custom powershell - here we have things to improve.

    The same goes for Sitecore content. We have been at packages, tds and a bit of unicorn. We are not aligned a lot here in the team (we are 45 people in the sitecore dev team) - most did not like tds in the releases they tried so it is very project dependent. I would like to move to unicorn to a larger extend.

    Sitecore.ship would be natural as well with unicorn.

    That completes our list - and I seems to be very aligned with Nate on the choices to make if you are starting over without strings attached.

    Br
    Klaus
  • For us it is primarily Github, Team City, Octopus deploy, TDS and custom Powershell. Biggest drawback currently is the deployment speed of TDS why we consider to just automate the process of installing Sitecore Update packages.

    There are older projects not migrated to this setup yet - but I am sure we will get there :)

    Best Regards
    Sebastian
  • I sense a lot of hostility towards TFS/TFVC (you don't need to be a Betazoid to see that). I suppose this is due to restrictions it places on you in terms of project hierarchy and structure. I guess we have been using it so long we don't really notice this any more - we just do it naturally the way it wants anyway.

    I have heard complaints that it has less superior merge tooling - do people know you can change the merge tool to your tool of choice? Even use the git one if you like? Personally I've never had any issues with the TFS merge tools but I suppose that's a matter of experience.

    I've no vested interest here - just curious why people seem to have so much animosity to a tool I've never had any significant issues with.
  • In reply to Richard Hauer:

    Merging in TF Version control is LIGHTYEARS behind GIT - it is AT BEST on par with subversion (I would possibly argue behind).

    I have said my reasoning for not being a TFS / TFVC fan, the primary of which is that I believe this tooling should work around you, not you around it.

    Otherwise, it has issues with ignores, permissions, the fact its integrated heavily to VS (which for example front end developers should not require). It's expensive, hardware intensive (in the case of TFS)... For me - that's enough reasons to explore the competition.

    I am happy to run a little call with you to run through my regular toolset if you ever need :D

  • In reply to Richard Hauer:

    Not quite hostility but too many scars of a tool which currently might be close to catching up - but when I started trying it out in tfs 2005 it was buggy, slow in updating your work area - and especially onky being able to have checkout of a work area was a strong for me.

    I saw a number of occasions where a get latest would kot update files and so forth.
    I never liked the build server facilities - it scales badly and was cumbersome to configure for continous integration and deployment.

    The times I tried never versions I never felt it did the same for us as the toolset we had in place. I undoubtly did not give it a fair chance - but I still dont like the structures it enforces.

    Br
    Klaus
  • In reply to Richard Hauer:

    Hi,

     

    I learned my way through the CI-CD tool chain set over last two year for very large Sitecore based platform initiative. Trying to blog about it now..

    Let the follow image speak more.. 

    What these lines/arrows means and how the overall Dev-Ops can work will explain further.. in coming article in that series..

  • For continues deployment and integration we used:

    1. TFS for source control

    2. Octopus with Powershell to trigger Package deployment and file and DLLs distribution.

    4. Custom script to perform continues package deployment.

     

    I've posted example of package deployment custom script here: borisbrodsky.com/.../

    -Boris