Skip to content

Open Source

Eating The Dog Food… In Public

Sonar - Thu, 02/16/2017 - 10:55

At SonarSource, we’ve always eaten our own dog food, but that hasn’t always been visible outside the company. I talked about how dogfooding works at SonarSource a couple years ago. Today, the process is much the same, but the visibility is quite different.

When I wrote about this in 2015, we used a private SonarQube server named “Dory” for dogfooding. Every project in the company was analyzed there, and it was Dory’s standards we were held to. Today, that’s still the case, but the server’s no longer private, and it’s no longer named “Dory”.

Today, we use next.sonarqube.com (nee Dory) for dogfooding, and it’s open to the public. That means you can follow along as, for instance, we run new rule implementations against our own code bases before releasing them to you. We also have a set of example projects we run new rules against before they even make it to Next, but seeing a potentially questionable issue raised against someone else’s code hits a different emotional note than seeing it raised against your own.

Of course, that’s the point of dogfooding: that we feel your pain. As an example, take the problem of new issues raised in the leak period on old code. Since we deploy new code analyzer snapshots on Next as often as daily, it means we’re always introducing new rules or improved implementations that find issues they didn’t find before. And that means that we’re always raising new issues on old code. Since we enforce the requirement to have a passing quality gate to release, this causes us the same problem you face when you do a “simple” code analyzer upgrade and suddenly see new issues on old code. Because we do feel that pain, SonarQube 6.3 includes changes to the algorithm that sets issue creation date so that issues from new rules that are raised on old code won’t be raised in the leak period.

Obviously, we’re not just testing rules on Next; we’re also testing changes to SonarQube itself. About once a day, a new version of SonarQube itself is deployed there. In fact, it happens so often, we added a notification block to our wallboard to keep up with it:

By running the latest milestone on our internal instance, each UI change is put through its paces pretty thoroughly. That’s because we all use Next, and no one in this crowd is meek or bashful.

Always running the latest milestone also means that if you decide to look over our shoulders at Next, you’ll get a sneak peek at where the next version is headed. Just don’t be surprised if details change from day to day. Because around here, change is the only constant.

Categories: Open Source

Say Hello to the Blue Ocean Pipeline Editor

Back in September 2016 we announced the availability of the Blue Ocean beta and the forthcoming Visual Pipeline Editor. We are happy to announce that you can try the Pipeline Editor preview release today. What is it? The Visual Pipeline Editor is the simplest way for anyone wanting to get started with creating Pipelines in Jenkins. It’s also a great way for advanced Jenkins users to start adopting pipeline. It allows developers to break up their pipeline into different stages and parallelize tasks that can occur at the same time - graphically. The rest is up to you. A pipeline you create visually will produce a Declarative...
Categories: Open Source

Declarative Pipeline: Notifications and Shared Libraries

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. Declare Your Pipelines! Declarative Pipeline 1.0 is here! This is the third post in a series showing some of the cool features of Declarative Pipeline. In the previous post, we converted a Scripted Pipeline to a Declarative Pipeline, adding descriptive stages and post sections. In one of those post blocks, we included a placeholder for sending notifications. In this blog post, we’ll repeat what I did in "Sending Notifications in Pipeline but this time in Declarative Pipeline. First we’ll integrate calls to notification services Slack, HipChat, and Email into our Pipeline. Then we’ll refactor those calls into a single Step in a...
Categories: Open Source

Declarative Pipeline: Publishing HTML Reports

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. Declare Your Pipelines! Declarative Pipeline 1.0 is here! This is the second post in a series showing some of the cool features of Declarative Pipeline. In the previous blog post, we created a simple Declarative Pipeline. In this blog post, we’ll go back and look at the Scripted Pipeline for the Publishing HTML Reports in Pipeline blog post. We’ll convert that Pipeline to Declarative syntax (including properties), go into more detail on the post section, and then we’ll use the agent directive to switch our Pipeline to run in Docker. Setup For this post, I’m going to use the blog/add-declarative/html branch of my fork of the hermann...
Categories: Open Source

Blue Ocean Dev Log: February Week #2

We’re counting down the weeks until Blue Ocean 1.0, which is planned for the end of March. If you hadn’t picked up on the hint in my previous post, most of the Blue Ocean development team is in Australia, where it is currently the middle of summer. As I write this it is about 1000 degrees outside. Emergency measures such as air-conditioning and beer have been deployed in order to continue Blue Ocean development. This week featured a new beta with the SCM API changes; many bug fixes, and some version bumps went out in beta 22. We also got some fresh new designs coming soon, though not in time for beta...
Categories: Open Source

Hacking with IntelliJ

Selenium - Wed, 02/08/2017 - 13:02

Jetbrains have a programme for Open Source projects which allows them to receive IntelliJ IDEA licenses. As part of that programme, which the Selenium project has participated in for many years, they’ve asked us to provide a fair and balanced review of IntelliJ. I’ll attempt to do that, and I’ll try and state my biases up-front so you’re aware of them.

I’ve been using Jetbrain’s IntelliJ to hack on the Selenium code since I started working on it slightly over ten years ago. It’s still my favourite IDE for my Java work, and it’s plenty of fun to use. For some time, I’ve been using the (free) Community Edition, which is ample for many coding needs.

Most of my work is in Java, and that’s where I know IDEA best. I dabble in Ruby and Python, and I’ve written a reasonably large amount of Javascript, all in IDEA.

The Pros:

In common with other good IDEs, IDEA has the ability to work seamlessly with many different languages. If you’re a polyglot programmer, being able to stay in the same tool for much of your work makes life a lot easier. On the Selenium project, we use Java, C#, Ruby, Python, and Javascript extensively. I don’t do any C#, and I mainly focus on Java, but the support for JS, Ruby, and Python is lovely and seems to work well. The built in type detection and code navigation features are impressive (particularly for untyped languages such as JS)

Of course, the feature that made IDEA so awesome in the first place is the range of refactoring options it offers. These are great, and always have been. One nice feature I’ve noticed as we move to a Java 8 future (finally!) is that it offers suggestions to help migrate to new features where they make sense (and, I’ll be honest, sometimes when they don’t). It’s made making use of lambdas a lot easier.

For a while, IDEA was becoming slower and more bloated, but I’m pleased to see that, partly thanks to the work of developers from Facebook, the latest releases feel snappier and handle larger projects more efficiently. One thing I appreciate is how open Jetbrains were to receiving patches to their core product: it displays a level of respect for external contributors that I feel is important (of course, I would think that: I work on OSS for fun!)

There’s a nice wide range of plugins available for IDEA. I’ve hooked up the Buck plugin and made use of it. Without an extensions API, this plugin wouldn’t have been possible, but having them there is incredibly useful and makes the IDE even more capable.

Finally for the plus points of the IDE, I love that the IDE tracks new versions of Java relatively closely — it’s fun to see what new language features we’ll be able to use in the future!

The Cons:

Although it’s a fine product, there are some niggles to be had.

Most annoyingly, the built in code analysis doesn’t always warn that some Java classes won’t compile. The most recent example was where IDEA didn’t flag that some lambdas couldn’t be used since the choice of method to use was ambiguous. This may be because the Java language continues its slothful way forward, and the compiler improves with each release — certainly these same files compiled just fine with older Java releases.

When an error does happen, I’ve yet to find the magic setting to allow IDEA to keep going as far as possible. One of the features I like about Eclipse is that it’ll compile as much as it can, even if there are invalid source files. When doing TDD, this allows you to move just a little bit faster as unit tests can run and pass so long as they don’t touch faulty code. I dearly wish this same capability was present in IDEA!

On the Selenium project, we use Buck for our builds. The Buck plugin doesn’t (yet!) allow me to build and run tests within the IDE, yet Buck performs some steps that can’t be repeated by the IDE that are required for a successful build. IDEA offers the ability to run an Ant step before a build is run, and it would be extremely useful if this was generalised to “any shell command”. Most of the time, it’s fine, but it’s irksome to forget to run things!

On the whole, I love IntelliJ an awful lot. It’s a fast and capable IDE, and the company behind it supports OSS. What’s not to love?


Categories: Open Source

Monitor Jenkins jobs with the Datadog plugin

This is a guest post by Emily Chang, Technical Author at Datadog. A modified version of this article was originally posted on the Datadog blog. If you’re using Jenkins to continuously integrate changes into your projects, it’s helpful to be able to quickly identify build failures and assess their impact on other components of your stack. Datadog’s plugin helps users monitor and alert on the performance of their Jenkins builds, right alongside the rest of their infrastructure and applications. As shown in the out-of-the-box dashboard below, the Datadog plugin provides a bird’s-eye view of job history and trends. You can use Datadog to: Set alerts...
Categories: Open Source

Declarative Pipeline for Maven Projects

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. Declare Your Pipelines! Declarative Pipeline 1.0 is here! This is first in a series of blog posts that will show some of the cool features of Declarative Pipeline. For several of these posts, I’ll be revisiting some of my previous posts on using various plugins with (Scripted) Pipeline, and seeing how those are implemented in Declarative Pipeline. To start though, let’s get familiar with the basic structure of a Declarative Pipeline by creating a simple Pipeline for a Maven-based Java project - the Jenkins JUnit plugin. We’ll create a minimal Declarative Pipeline, add the settings needed to install Maven and the JDK, and...
Categories: Open Source

Google Summer Of Code 2017: Call for mentors

On behalf of the GSoC Org Admin team I am happy to announce that we are going to apply to Google Summer of Code (GSoC) again this year. In GSoC high-profile students work in open-source projects for several months under mentorship of organization members. We are looking for mentors and project ideas. So yes, we are looking for you :) Conditions As a mentor, you will be asked to: lead the project in the area of their interest actively participate in the project during student selection, community bonding and coding phases (March - August) work in teams of 2+ mentors per 1 each student dedicate a consistent and significant amount...
Categories: Open Source

The Tweets You Missed in January

Sonar - Mon, 02/06/2017 - 11:09

Here are the tweets you likely missed last month!

SonarQube 6.2 released: read the news and see in it screenshots! https://t.co/MaLzAnorc9https://t.co/qhVuMdnvSD pic.twitter.com/Lqvc7IklFf

— SonarQube (@SonarQube) January 10, 2017

Governance 1.2 dissociates overall health of an application portfolio and risk identified on its projects https://t.co/Ztt9a0AMQ4 pic.twitter.com/bVBhkKal1p

— SonarQube (@SonarQube) January 12, 2017

SonarPython 1.7 brings support for Cognitive Complexity https://t.co/r5Mi3ga0jw pic.twitter.com/1kwBUvaoVR

— SonarQube (@SonarQube) January 27, 2017

SonarC++ 4.4 Released: SonarLint for Eclipse CDT support, improved dataflow engine and 4 new ruleshttps://t.co/eQRS02gGN3 pic.twitter.com/MFxomdHCXF

— SonarQube (@SonarQube) January 12, 2017

SonarJS 2.19 Released: 14 new rules, including 2 rules detecting invalid calls to built-in methods https://t.co/RaZOecudjn #javascript pic.twitter.com/CvENmeD882

— SonarQube (@SonarQube) January 12, 2017

Detecting Type Issues in #javascript with SonarJS, see https://t.co/ugos9C3uXf pic.twitter.com/R2RTEt9jQ3

— SonarQube (@SonarQube) January 11, 2017

SonarLint for IntelliJ 2.7 shows issues context and highlights corresponding locations https://t.co/wvnnyVEjA4 pic.twitter.com/GMGBO3Vvwx

— SonarLint (@SonarLint) January 31, 2017

Categories: Open Source

SCM API 2.0 Release Take 2

In January we announced the release of SCM API 2.0. After the original release was published we identified four new high-impact issues. We decided to remove the new versions of the plugins from the update center until those issues could be resolved. The issues have now been resolved and the plugins are now available from the update center. Summary for busy Jenkins Administrators Upgrading should make multi-branch projects much better. When you are ready to upgrade you must ensure that you upgrade all the required plugins. If you miss some, just upgrade them and restart to fix the issue. And of course, it’s always a good idea...
Categories: Open Source

Declarative Pipeline Syntax 1.0 is now available

This is a guest post by Patrick Wolf, Director of Product Management at CloudBees and contributor to the Jenkins project. I am very excited to announce the addition of Declarative Pipeline syntax 1.0 to Jenkins Pipeline. We think this new syntax will enable everyone involved in DevOps, regardless of expertise, to participate in the continuous delivery process. Whether creating, editing or reviewing a pipeline, having a straightforward structure helps to understand and predict the flow of the pipeline and provides a common foundation across all pipelines. Pipeline as Code Pipeline as Code was one of the pillars of the Jenkins 2.0 release and an essential part of implementing continuous delivery (CD). Defining all of the stages of...
Categories: Open Source

Blue Ocean Dev Log: February Week #1

With only a couple of months left before Blue Ocean 1.0, which is planned for the end of March, I have been highlighting some of the good work being finished up by the developers hacking on Blue Ocean. This week was a grab bag of important behind-the-scenes features and finalising the preview of the editor. The merge of the SCM API changes also made it in. The editor has the new sheets style of editing (there will be blogs and more on this in the next few weeks): Some highlights: Fix to async loading of resources like translations, so screens don’t "flash" when they are loaded (i18n improvement) Links in notifications can be configured...
Categories: Open Source

Security updates for Jenkins core

We just released security updates to Jenkins, versions 2.44 and 2.32.2, that fix a high severity and several medium and low severity issues. For an overview of what was fixed, see the security advisory. For an overview on the possible impact of these changes on upgrading Jenkins LTS, see our LTS upgrade guide. I strongly recommend you read these documents, as there are a few possible side effects of these fixes. Subscribe to the jenkinsci-advisories mailing list to receive important notifications related to Jenkins security....
Categories: Open Source

Best Practices for Scalable Pipeline Code

This is a guest post by Sam Van Oort, Software Engineer at CloudBees and contributor to the Jenkins project. Today I’m going to show you best practices to write scalable and robust Jenkins Pipelines. This is drawn from a combination of work with the internals of Pipeline and observations with large-scale users. Pipeline code works beautifully for its intended role of automating build/test/deploy/administer tasks. As it is pressed into more complex roles and unanticipated uses, some users hit issues. In these cases, applying the best practices can make the difference between: A single master running hundreds of concurrent builds on low end hardware (4 CPU cores and 4 GB...
Categories: Open Source

Blue Ocean Dev Log: January Week #4

As we get closer to Blue Ocean 1.0, which is planned for the end of March, I have started highlighting some of the good stuff that has been going on. This week was 10 steps forward, and about 1.5 backwards…​ There were two releases this week, b19 and b20. Unfortunately, b20 had to be released shortly after b19 hit the Update Center as an incompatible API change in a 3rd party plugin was discovered. Regardless, the latest b20 has a lot of important improvements, and some very nice new features. A first cut of the "Create Pipeline" UX, seen above, allowing you to create Git based Multibranch Pipelines like you have never...
Categories: Open Source

Blue Ocean Dev Log: January Week #3

As we get closer to Blue Ocean 1.0, which is planned for the end of March, I have started highlighting some of the good stuff that has been going on, and this week was a very busy week. A new Blue Ocean beta (b18) was released with: Parametrized pipelines are now supported! i18n improvements Better support for matrix and the evil (yet somehow still used) Maven project type (don’t use it!) SSE fixes for IE and Edge browsers An alpha release of the Visual Editor for Jenkinsfiles on top of Declarative Pipeline has snuck into the "experimental" update center. Andrew will be talking about Declarative Pipelines at FOSDEM next week. Parametrized Pipelines You would know this...
Categories: Open Source

Converting Conditional Build Steps to Jenkins Pipeline

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. Introduction With all the new developments in Jenkins Pipeline (and Declarative Pipeline on the horizon), it’s easy to forget what we did to create "pipelines" before Pipeline. There are number of plugins, some that have been around since the very beginning, that enable users to create "pipelines" in Jenkins. For example, basic job chaining worked well in many cases, and the Parameterized Trigger plugin made chaining more flexible. However, creating chained jobs with conditional behavior was still one of the harder things to do in Jenkins. The Conditional BuildStep plugin is a powerful tool that has allowed Jenkins users to write Jenkins jobs with complex...
Categories: Open Source

Jenkins Upgrades To Java 8

In the next few months, Jenkins will require Java 8 as its runtime. Back in last November, we discussed interesting statistics showing that Jenkins was now running Java 8 on a majority of its running instances. Timeline Here is how we plan to roll that baseline upgrade in the next few months. Now: Announce the intention publicly. April, 2017: Drop support for Java 7 in Jenkins weekly. With the current rhythm, that means 2.52 will most likely be the first weekly to require Java 8. June 2017: First LTS version requiring Java 8 is published. This should be something around 2.60.1. If you are still running Java 7, you will not be...
Categories: Open Source

SCM API turns 2.0 and what that means for you

The regressions discovered after release have now been resolved and this post has been updated with the correct plugin version numbers. See this post for more details. We are announcing the SCM API 2.0.x and Branch API 2.0.x release lines. Downstream of this there are also some great improvements to a number of popular plugins including: GitHub Branch Source BitBucket branch source Git Mercurial Pipeline Multibranch GitHub Organization Folders There are some gotcha’s that Jenkins administrators will need to be aware of. Always take a backup of your JENKINS_HOME before upgrading any plugins. We want to give you the whole story, but the take home message is this: When updating the SCM API and/or Branch API plugins to the 2.0.x release lines,...
Categories: Open Source