Skip to content

Open Source

Converting Conditional Build Steps to Jenkins Pipeline

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. Introduction With all the new developments in Jenkins Pipeline (and Declarative Pipeline on the horizon), it’s easy to forget what we did to create "pipelines" before Pipeline. There are number of plugins, some that have been around since the very beginning, that enable users to create "pipelines" in Jenkins. For example, basic job chaining worked well in many cases, and the Parameterized Trigger plugin made chaining more flexible. However, creating chained jobs with conditional behavior was still one of the harder things to do in Jenkins. The Conditional BuildStep plugin is a powerful tool that has allowed Jenkins users to write Jenkins jobs with complex...
Categories: Open Source

Jenkins Upgrades To Java 8

In the next few months, Jenkins will require Java 8 as its runtime. Back in last November, we discussed interesting statistics showing that Jenkins was now running Java 8 on a majority of its running instances. Timeline Here is how we plan to roll that baseline upgrade in the next few months. Now: Announce the intention publicly. April, 2017: Drop support for Java 7 in Jenkins weekly. With the current rhythm, that means 2.52 will most likely be the first weekly to require Java 8. June 2017: First LTS version requiring Java 8 is published. This should be something around 2.60.1. If you are still running Java 7, you will not be...
Categories: Open Source

SCM API turns 2.0 and what that means for you

Due to regressions discovered after release it is not recommended to upgrade the plugins listed below at this time. We are announcing the SCM API 2.0.x and Branch API 2.0.x release lines. Downstream of this there are also some great improvements to a number of popular plugins including: GitHub Branch Source BitBucket branch source Git Mercurial Pipeline Multibranch GitHub Organization Folders There are some gotcha’s that Jenkins administrators will need to be aware of. Always take a backup of your JENKINS_HOME before upgrading any plugins. We want to give you the whole story, but the take home message is this: When updating the SCM API and/or Branch API plugins to the 2.0.x release lines, if you have any of the GitHub Organization Folders, GitHub...
Categories: Open Source

Blue Ocean Dev Log: January Week #2

As we get closer to Blue Ocean 1.0, which is planned for the end of March, I figured it would be great to highlight some of the good stuff that has been going on. It’s been a busy-as-usual week as everyone comes back from vacation. A couple of new betas went out this week. Of note: input to Pipelines is now supported, a much asked for feature (see below) A new French translation Some optimisations (especially around reducing number of HTTP calls). We have started using gtmetrix.com to measure changes on dogfood to get some numbers around optimisations on the web tier. And a grab bag of other great bug fixes. Also a bunch...
Categories: Open Source

Declarative Pipeline Syntax Beta 2 release

This week, we released the second beta of the new Declarative Pipeline syntax, available in the Update Center now as version 0.8.1 of Pipeline: Model Definition. You can read more about Declarative Pipeline in the blog post introducing the first beta from December, but we wanted to update you all on the syntax changes in the second beta. These syntax changes are the last compatibility-breaking changes to the syntax before the 1.0 release planned for February, so you can safely start using the 0.8.1 syntax now without needing to change it when 1.0 is released. A full syntax reference is available on the wiki as well. Syntax Changes Changed "agent" configuration...
Categories: Open Source

Jenkins World 2017 Call for Papers is Open

The largest Jenkins event, Jenkins World is coming to San Francisco, California on August 28 - 31, 2017, at the Marriott Marquis. This conference will feature two days of hands-on training, workshops, and certification exams followed by two more days with five tracks of technical sessions from Jenkins and DevOps experts from around the world. Inspire your peers and colleagues by sharing your expertise and experience as one of the Jenkins World speakers. The Call for Papers is open, last day for submitted a proposal is March 5th, 2017. Compared to Jenkins World 2016, what’s new for 2017? Two tracks are now dedicated to "show and tell." These...
Categories: Open Source

Security warnings in Jenkins

Jenkins 2.40 was released earlier this week, and readers of the changelog will have noticed that it now includes the ability to show security warnings published by the configured update site. But what does that mean? In the past, we’ve notified users about security issues in Jenkins and in plugins through various means: Emails to the jenkinsci-advisories mailing list (which I recommend you subscribe to), blog posts, and, recently, emails to the oss-security mailing list. But I still wanted to increase the reach of our notifications, to make sure Jenkins admins are informed quickly about possible security problems on their instances. The logical next step was to...
Categories: Open Source

Learning plugin development by improving the LIFX notifier plugin

This is a cross post by Veaceslav Gaidarji, open source developer and contributor to the Jenkins and Bitrise projects. Some time ago I encountered a LIFX smart bulbs. These are the bulbs with a chip inside - 50% bulb, 50% chip. There are mobile applications for easy configuration and remote control of the bulb. Nothing special here, it simply works and is very convenient to have such bulbs in dormitory. Brilliant idea time 99% of ideas which come to our minds either were already implemented by someone else or they are useless. — Veaceslav Gaidarji And as it always happens, the developer inside me generated an idea which, as it always happens, was implemented...
Categories: Open Source

Thank you for an amazing 2016

I do not think it is an exaggeration to say: 2016 was the best year yet for the Jenkins project. Since the first commit in 2006, the project has reached a number of significant milestones in its ten years but we have never experienced the breadth of major milestones in such a short amount of time. From Jenkins 2 and Blue Ocean to the Google Summer of Code and Jenkins World, I wanted to take a moment and celebrate the myriad of accomplishments which couldn’t have happened without the help from everybody who participates in the Jenkins project. The 1,300+ contributors to the jenkinsci GitHub organization, the 4,000+ members of the !forum/jenkinsci-dev">developers mailing list, the 8,000+ members...
Categories: Open Source

Continuous Delivery with Jenkins and Puppet Enterprise

This is a guest post by Carl Caum, who works at Puppet and created the Puppet Enterprise Pipeline plugin. During PuppetConf 2016, myself and Brian Dawson from CloudBees announced the Puppet Enterprise plugin for Jenkins Pipeline. Let’s take a look at how the plugin makes it trivial to use Puppet to perform some or all of the deployment tasks in continuous delivery pipelines. Jenkins Pipeline introduced an amazing world where the definition for a pipeline is managed from the same version control repository as the code delivered by the pipeline. This is a powerful idea, and one I felt complemented Puppet’s automation strengths. I wanted to make it trivial to control...
Categories: Open Source

Announcing the beta of Declarative Pipeline Syntax

Last week we released version 0.7.1 of the Pipeline-Model-Defintion plugin and wanted to crown it as the official Beta version of the Declarative Pipeline syntax. Although it has been available in the update center since August, we continue to solidify the syntax. We feel this release is getting very close to the final version and should not change much before 1.0. However, it is still a Beta so further tweaks are possible. A release (0.8.0) is planned for early January 2017 which will finalize the syntax with the following changes: JENKINS-40524, JENKINS-40370, JENKINS-40462, JENKINS-40337 What is Declarative Pipeline? All the way back at Jenkins World in September, Andrew Bayer presented a sneak peak of a new syntax...
Categories: Open Source

Monthly JAM Recap - November 2016

As we near the end of the year, the number of November JAMs show that the Jenkins community isn’t slowing down for holiday season. We had a number of excellent events hosted around the world this November with plenty of great stories and presentations shared by the various members of the world-wide Jenkins community. Melbourne, Australia JAM Melbourne JAM leaders, Raisa and Bhuva hosted Blue Ocean for the inaugural meeting. Attendees learned the values of Blue Ocean, a project that rethinks the user experience of Jenkins, modeling and presenting the process of software delivery by surfacing information that is important to development teams with as few clicks as possible, while still staying true to...
Categories: Open Source

Test for RSS please ignore.

Sonar - Fri, 12/09/2016 - 15:29
Categories: Open Source

Upcoming December Jenkins Events

Happy Holidays! A special shout out to all JAM leaders who continue to keep local activities going in December. Online JAM December 14 | Live Demos: Pipeline, Git, and Blue Ocean North America December 7 | Seattle JAM: Jenkins at Microsoft December 14 | Los Angeles JAM: Jenkins Days December 14 | Guadalajara JAM: Jenkins & Docker Australia December 14 | Melbourne JAM: Meeting at AWS Office Links Start a JAM in your city if there isn’t one already. Become a JAM member Become an online JAM member Speak or sponsor at a JAM. Contact us at jenkinsci-jam@googlegroups.com Become a Jenkins project contributor...
Categories: Open Source

Cognitive Complexity, Because Testability != Understandability

Sonar - Wed, 12/07/2016 - 13:35

Thomas J. McCabe introduced Cyclomatic Complexity in 1976 as a way to guide programmers in writing methods that “are both testable and maintainable”. At SonarSource, we believe Cyclomatic Complexity works very well for measuring testability, but not for maintainability. That’s why we’re introducing Cognitive Complexity, which you’ll begin seeing in upcoming versions of our language analyzers. We’ve designed it to give you a good relative measure of how difficult the control flow of a method is to understand.

Cyclomatic Complexity doesn’t measure maintainability

To get started let’s look at a couple of methods:

int sumOfPrimes(int max) {              // +1
  int total = 0;
  OUT: for (int i = 1; i <= max; ++i) { // +1
    for (int j = 2; j < i; ++j) {       // +1
      if (i % j == 0) {                 // +1
        continue OUT;
      }
    }
    total += i;
  }
  return total;
}                  // Cyclomatic Complexity 4
 
  String getWords(int number) { // +1
    switch (number) {
      case 1:                   // +1
        return "one";
      case 2:                   // +1
        return "a couple";
      default:                  // +1
        return "lots";
    }
  }        // Cyclomatic Complexity 4

These two methods share the same Cyclomatic Complexity, but clearly not the same maintainability. Of course, this comparison might not be entirely fair; even McCabe acknowledged in his original paper that the treatment of case statements in a switch didn't seem quite right:

The only situation in which this limit [of 10 per method] has seemed unreasonable is when a large number of independent cases followed a selection function (a large case statement)...

On the other hand, that's exactly the problem with Cyclomatic Complexity. The scores certainly tell you how many test cases are needed to cover a given method, but they aren't always fair from a maintainability standpoint. Further, because even the simplest method gets a Cyclomatic Complexity score of 1, a large domain class can have the same Cyclomatic Complexity as a small class full of intense logic. And at the application level, studies have shown that Cyclomatic Complexity correlates to lines of code, so it really doesn't tell you anything new.

Cognitive Complexity to the rescue!

That's why we've formulated Cognitive Complexity, which attempts to put a number on how difficult the control flow of a method is to understand, and therefore to maintain.

I'll get to some details in a minute, but first I'd like to talk a little more about the motivations. Obviously, the primary goal is to calculate a score that's an intuitively "fair" representation of maintainability. In doing so, however, we were very aware that if we measure it, you will try to improve it. And because of that, we want Cognitive Complexity to incent good, clean coding practices by incrementing for code constructs that take extra effort to understand, and by ignoring structures that make code easier to read.

Basic criteria

We boiled that guiding principle down into three simple rules:

  • Increment when there is a break in the linear (top-to-bottom, left-to-right) flow of the code
  • Increment when structures that break the flow are nested
  • Ignore "shorthand" structures that readably condense multiple lines of code into one
Examples revisited

With those rules in mind, let's take another look at those first two methods:

                                // Cyclomatic Complexity    Cognitive Complexity
  String getWords(int number) { //          +1
    switch (number) {           //                                  +1
      case 1:                   //          +1
        return "one";
      case 2:                   //          +1
        return "a couple";
      default:                  //          +1
        return "lots";
    }
  }                             //          =4                      =1

As I mentioned, one of the biggest beefs with Cyclomatic Complexity has been its treatment of switch statements. Cognitive Complexity, on the other hand, only increments once for the entire switch structure, cases and all. Why? In short, because switches are easy, and Cognitive Complexity is about estimating how hard or easy control flow is to understand.

On the other hand, Cognitive Complexity increments in a familiar way for the other control flow structures: for, while, do while, ternary operators, if/#if/#ifdef/..., else if/elsif/elif/..., and else, as well as for catch statements. Additionally, it increments for jumps to labels (goto, break, and continue) and for each level of control flow nesting:

                                // Cyclomatic Complexity    Cognitive Complexity
int sumOfPrimes(int max) {              // +1
  int total = 0;
  OUT: for (int i = 1; i <= max; ++i) { // +1                       +1
    for (int j = 2; j < i; ++j) {       // +1                       +2 (nesting=1)
      if (i % j == 0) {                 // +1                       +3 (nesting=2)
        continue OUT;                   //                          +1
      }
    }
    total += i;
  }
  return total;
}                               //         =4                       =7

As you can see, Cognitive Complexity takes into account the things that make this method harder to understand than getWords - the nesting and the continue to a label. So that while the two methods have equal Cyclomatic Complexity scores, their Cognitive Complexity scores clearly reflect the dramatic difference between them in understandability.

In looking at these examples, you may have noticed that Cognitive Complexity doesn't increment for the method itself. That means that simple domain classes have a Cognitive Complexity of zero:

                              // Cyclomatic Complexity       Cognitive Complexity
public class Fruit {

  private String name;

  public Fruit(String name) { //        +1                          +0
    this.name = name;
  }

  public void setName(String name) { // +1                          +0
    this.name = name;
  }

  public String getName() {   //        +1                          +0
    return this.name;
  }
}                             //        =3                          =0

So now class-level metrics become meaningful. You can look at a list of classes and their Cognitive Complexity scores and know that when you see a high number, it really means there's a lot of logic in the class, not just a lot of methods.

Getting started with Cognitive Complexity

At this point, you know most of what you need to get started with Cognitive Complexity. There are some differences in how boolean operators are counted, but I'll let you read the white paper for those details. Hopefully, you're eager to start using Cognitive Complexity, and wondering when tools to measure it will become available.

We'll start by adding method-level Cognitive Complexity rules in each language, similar to the existing ones for Cyclomatic Complexity. You'll see this first in the mainline languages: Java, JavaScript, C#, and C/C++/Objective-C. At the same time, we'll correct the implementations of the existing method level "Cyclomatic Complexity" rules to truly measure Cyclomatic Complexity (right now, they're a combination of Cyclomatic and Essential Complexity.)

Eventually, we'll probably add class/file-level Cognitive Complexity rules and metrics. But we're starting with Baby Steps.

Categories: Open Source

The Tweets You Missed in November

Sonar - Mon, 12/05/2016 - 12:05

Here are the tweets you likely missed last month!

"SonarQube 6.x series: Focused and Efficient", by @bellingard https://t.co/XBND3qPUA1

— SonarQube (@SonarQube) November 3, 2016

SonarQube JavaScript 2.18: 10 new rules and significant improvements to the type-based analysis, seehttps://t.co/ZjCRncEUDw pic.twitter.com/UUs3IWHmi5

— SonarQube (@SonarQube) November 21, 2016

SonarLint for IntelliJ 2.4 hides false-positive and won't-fix issues in connected mode. https://t.co/dRURlXJ0Sk pic.twitter.com/JVHM2kJsgu

— SonarLint (@SonarLint) November 17, 2016

SonarLint for Eclipse 2.3 hides false-positive and won't-fix issues in connected mode. https://t.co/rTXlxtAjKZ pic.twitter.com/1O6zKA5GVl

— SonarLint (@SonarLint) November 28, 2016

SonarLint for Visual Studio 2.8 Released with even more powerful path-sensitive dataflow engine see https://t.co/z7VETuMBrl @VisualStudio pic.twitter.com/pDAvY7lqmU

— SonarLint (@SonarLint) November 29, 2016

Categories: Open Source

NUnit-Summary Becoming an “Official” NUnit Application

NUnit.org - Thu, 09/22/2016 - 23:39

NUnit-Summary is an “extra” that I’ve maintained personally for some time. It uses built-in or user-supplied transforms to produce summary reports based on the results of NUnit tests.

I have contributed it to the NUnit project and we’re working on updating it to recognize NUnit 3 test results. The program has never had a 1.0 release, but we expect to produce one soon.

This old post talks about the original nunit-summary program.

Categories: Open Source

An Engine Extension for Running Failed Tests – Part 1: Creating the Extension

NUnit.org - Thu, 09/22/2016 - 20:47

In a recent online discussion, one of our users talked about needing to re-run the NUnit console runner, executing just the failed tests from the previous run. This isn’t a feature in NUnit but it could be useful to some people. So… can we do this by creating an Engine Extension? Let’s give it a try!

The NUnit Test Engine supports extensions. In this case, we’re talking about a Result Writer extension, one that will take the output of a test run from NUnit and create an output file in a particular format. In this case, we want the output to be a text file with each line holding the full name of a failed test case. Why that format? Because it’s exactly the format that the console runner already recognizes for the --testlist option. We can use the file that is created as input to a subsequent test run.

Information about how to write an extension can be found on the Writing Engine Extensions page of the NUnit documentation. Details of creating a ResultWriter extension can be found on the Result Writers page.

To get started, I created a new class library project called failed-tests-writer. I made sure that it targeted .NET 2.0, because that allows it to be run under the widest range of runtime versions and I added a package reference to the NUnit.Engine.Api package. That package will be published on nuget.org with the release of NUnit 3.5. Since that’s not out yet, I used the latest pre-release version from the NUnit project MyGet feed by adding https://www.myget.org/F/nunit/api/v2 to my NuGet package sources.

Next, I created a class to implement the extension. I called it FailedTestsWriter. I added using statements for NUnit.Engine and NUnit.Engine.Extensibility and implemented the IResultWriter interface. I gave my class Extension and ExtensionProperty attributes. Here is what it looked like when I was done.

using System;
using System.IO;
using NUnit.Engine;
using NUnit.Engine.Extensibility

namespace EngineExtensions
{
    [Extension, ExtensionAttribute("Format", "failedtests")]
    public class FailedTestsWriter : IResultWriter
    {
        public void CheckWritability(string outputPath)
        {
            using (new StreamWriter(outputPath, false, Encoding.UTF8)) { }
        }

        public void WriteResultFile(XmlNode resultNode, string outputPath)
        {
            using (var writer = new StreamWriter(outputPath, false, Encoding.UTF8))
            {
                WriteResultFile(resultNode, writer);
            }
        }

        public void WriteResultFile(XmlNode resultNode, TextWriter writer)
        {
            foreach (XmlNode node in resultNode.SelectNodes("//test-case[@result='Failed']")) // (3)
                writer.WriteLine(node.Attributes["fullname"].Value);
        }
    }
}

The ExtensionAttribute marks the class as an extension. In this case as in most cases, it’s not necessary to add any arguments. The Engine can deduce how the extension should be used from the fact that it implements IResultWriter.

As explained on the Result Writers page, this type of extension requires use of the ExtensionPropertyAttribute so that NUnit knows the name of the format it implements. In this case, I chose to use “failedtests” as the format name.

The CheckWriteability method is required to throw an exception if the provided output path is not writeable. We do that very simply by trying to create a StreamWriter. The empty using statement is merely an easy way to ensure that the writer is closed.

The main point of the extension is accomplished in the second WriteResultFile method. A foreach statement selects each failing test, which is then written to the output file.

Testing the Extension

That explains how to write the extension. In Part 2, I’ll explain how to deploy it. Meanwhile, I’ll tell you how I tested my extension in it’s own solution, using nunit3-console.

First, I installed the package NUnit.ConsoleRunner from nuget.org. I used version 3.4.1. Next, I created a fake package subdirectory in my packages folder, so it ended up looking like this:

packages
    NUnit.ConsoleRunner.3.4.1
    NUnit.Engine.Api.3.5.0-dev-03211
    NUnit.Extension.FailedTestsWriter
        tools
            failed-tests-writer.dll

Note that the new extension “package” directory name must start with “NUnit.Extension.” in order to trick the console-runner and engine into using it.

With this structure in place, I was able to run the console with the --list-extensions option to see that my extension was installed and I could use a command like

nunit3-console mytests.dll --result:FailedTests.lst;format=failedtests

to actually produce the required output.

Categories: Open Source

Back to Blogging!

NUnit.org - Thu, 09/22/2016 - 02:50

My blog has been offline for a long time, as you can see. The last prior post was in 2009!

Recently, I found a backup copy of the old blog and was able to re-establish it. Watch for some new posts in the near future.

Categories: Open Source

Software Testing Latest Training Courses for 2012

The Cohen Blog — PushToTest - Mon, 02/20/2012 - 05:34
Free Workshops, Webinars, Screencasts on Open Source Testing Need to learn Selenium, soapUI or any of a dozen other Open Source Test (OST) tools? Join us for a free Webinar Workshop on OST. We just updated the calendar to include the following Workshops:
And If you are not available for the above Workshops, try watching a screencast recording.

Watch The Screencast

Categories: Companies, Open Source