Skip to content

Open Source

JUC Israel report

This year marks the 3rd annual Jenkins User Conference in Israel. While the timing of the event turned out to be less than ideal for reasons beyond our control, that didn't stop 400 Jenkins users from showing up at the "explosive" event at a seaside hotel near Tel Aviv.

Shlomi Ben-Haim kicked off the conference by reporting that JUC Israel just keeps getting bigger, and that we sold out 2 weeks earlier and the team had to turn down people who really wanted to come in. The degree of adoption of Jenkins is amazing in this part of the world, and we might have to find a bigger venue next year to accomodate everyone who wants to come.

IMG_9716

It turns out most of the talks were in Hebrew, so it was difficult for me to really understand what's going on, but the talks ranged from highly technical ones like how to provision Jenkins from configuration management (the server as welll as jobs), all the way to more culture focused one like how to deploy CD practice in an organization. Companies large and small were well represented, and I met with a number of folks who actively contribute to the community.

There were a lot of hall way conversations, and those of us at the booth had busy time.

Thanks everyone who came, thanks JFrog for being on the ground for the event (and congratulations for the new round of funding) and CloudBees for hosting the event. Please let us know if there are things we can do better, and see you again next year!

IMG_9777
Categories: Open Source

.NET in SonarQube: bright future

Sonar - Thu, 07/10/2014 - 11:12

A few months ago, we started on an innocuous-seeming task: make the .NET Ecosystem compatible with the multi-language feature in SonarQube 4.2. What followed was a bit like one of those cartoons where you pull a string on the character’s sweater and the whole cartoon character starts to unravel. Oops.

Once we stopped pulling the string and started knitting again (to torture a metaphor), what came off the needles was a different sweater than what we’d started with. The changes we made along the way – fewer external tools, simpler configuration – were well-intentioned, and we still believe they were the right things to do. But many people were at pains to tell us that the old way had been just fine, thank you. It had gotten the job done on a day-to-day basis for hundreds of projects, and hundreds-of-thousands of lines of code, they said. It had been crafted by .NETers for .NETers, and as Java geeks, they said, we really didn’t understand the domain.

And they were right. But when we started, we didn’t understand how much we didn’t understand. Fortunately, we have a better handle on our ignorance now, and a plan for overcoming it and emerging with industry leading C# and VB.NET analysis tools.

First, we’re planning to hire a C# developer. This person will be first and foremost our “really get .NET” person, and represents a real commitment to the future of SonarQube’s .NET plugins. She or he will be able to head off our most boneheaded notions at the pass, and guide us in the ways of righteousness. Or at least in the ways of .NETness.

Of course it’s not just a guru position. We’ll call on this person to help us progressively improve and evolve the C# and VB.NET plugins, and their associated helpers, such as the Analysis Bootstrapper. He (or she) will also help us fill the gaps back in. When we reworked the .NET ecosystem there were gains, but there were also loses. For instance, there are corner cases not covered today by the C# and VB.NET plugins which were covered with the old .NET Ecosystem.

We also plan to start moving these plugins into C#. We’ve realized that just can’t do the job as well in Java as we need to. But the move to C# code will be a gradual one, and we’ll do our best to make it painless and transparent. Also on the list will be identifying the most valuable rules from FxCop and ReSharper and re-implementing them in our code.

At the same time, we’ll be advancing on these fronts for both C# and VB.NET:

  • Push “cartography” information to SonarQube.
  • Implement bug detection rules.
  • Implement framework-specific rules, for things like SharePoint.

All of that with the ultimate goal of becoming the leader in analyzing .NET code. We’ve got a long way to go, but we know we’ll bring it home in the end.

Categories: Open Source

Planned changes in Jenkins User Conference contact information collection



One of the challenges of running Jenkins User Conferences is to ballance the interest of attendees and the interest of sponsors. Sponsors would like to know more about attendees, but attendees are often weary of getting contacted. Our past few JUCs have been run by making it opt-in to have the contact information passed to sponsors, but the ratio of people who opt-in is too low. So we started thinking about adjusting this.

So our current plan is to reduce the amount of data we collect and pass on, but to make this automatic for every attendee. Specifically, we'd limit the data only to name, company, e-mail, and city/state/country you are from. But no phone number, no street address, etc. We discussed this in the last project meeting, and people generally seem to think this is reasonable. That said, this is a sensitive issue, so we wanted more people to be aware.

By the way, the call for papers to JUC Bay Area is about to close in a few days. If you are interested in giving a talk (and that's often the best way to get feedback and take credit on your work), please make sure to submit it this week.

Categories: Open Source

Workflow plugin tutorial: writing a Step impl

The other day I was explaining how to implement a new workflow primitive to Vivek Pandey, and I captured it as a recording.

The recording goes over how to implement the Step extension point, which is the workflow equivalent of BuildStep extension point. If you are interested in jumping on the workflow plugin hacking, this might be useful (and don't forget to get in touch with us so that we can help you!)

Categories: Open Source

What's new in ApprovalTests.Net v3.7

Approval Tests - Thu, 07/03/2014 - 22:56
[Available on Nuget]
AsyncApprovals - rules and exceptions[Contributors: James Counts]In the end, all tests become synchronous. This means for a normal test we recommendHowever, If you are looking to test exceptions everything changes and you might want to use Removed BCL requirement[Contributors: James Counts &  Simon Cropp]HttpClient is nice way of doing web calls in .Net. Unfortunately, at this time the BCL in nuget does unfortunate things to your project if you do not wish to use the HttpClient. This is a violation of a core philosophy of approvaltests 
"only pay for the dependencies you use
HttpClient was add ApprovalTests 3.6. Thanks to Simon for pointing and troubleshooting this error. It has now been removed. 
Wpf Binding Asserts[Contributors: Jay Bazuzi]This is a bonus from v3.6It is a very hard thing to detect and report Wpf Binding Error. To even get the reports to happen you have to fiddle with the registry and then read and parse logs.No More!  Now to you use BindsWithoutError to ensure that your Wpf binding are working.

Categories: Open Source

Jenkins User Event & Code Camp 2014, Copenhagen

This is a guest post from Adam Henriques.

On August 22nd Jenkins CI enthusiasts will gather in Copenhagen, Denmark for the 3rd consecutive year for a day of networking and knowledge sharing. Over the past two years the event has grown and this year we are expecting a record number of participants representing Jenkins CI experts, enthusiasts, and users from all over the world.

The Jenkins CI User Event Copenhagen has become cynosure for the Scandinavian Jenkins community to come together and share new ideas, network, and harness inspiration from peers. The program offers invited as well as contributed speaks, tech talks, case stories, and facilitated Open Space discussions on best practice and application of continuous integration and agile development with Jenkins.

The Jenkins CI Code Camp 2014

The Jenkins CI User Event will be kicked off by The Jenkins CI Code Camp on August 21st, the day before the User Event. Featuring Jenkins frontrunners, this full day community driven event has become very popular, where Jenkins peers band together to contribute content back to the community. The intended audience is both experienced Jenkins developers and developers who are looking to get started with Jenkins plugin development.

For more information please visit the Jenkins CI User Event 2014, Copenhagen website.

Categories: Open Source

JUC Berlin summary

IMG_9194

After a very successful JUC Boston we headed over to Berlin for JUC Berlin. I've heard the attendance number was comparable to that of JUC Boston, with close to 400 people registered and 350+ people who came.

The event kicked off at a pre-conference beer garden meetup, except it turned out that the venue was closed on that day and we had to make an emergency switch to another nearby place, and missed some people during that fiasco. My apologies for that.

But the level of the talks during the day more than made up for my failing. They covered everything from large user use cases from BMW to Android builds, continuous delivery to Docker, then of course workflow!

One of the key attractions of events like this is actually meeting people you interact with. There are all the usual suspects of the community, including some who I've met for the first time.

Most of the slides are up, and I believe the video recordings will be uploaded shortly, if you missed the event.

Categories: Open Source

Pictures from JUC and cdSummit

I've uploaded pictures I've taken during JUC Boston and JUC Berlin.

JUC Berlin pictures starts with pre-conference beer garden meet-up. See Vincent Latombe gives a talk about Literate plugin. I really appreciated his coming to this despite the fact that the event was only a few days before his wedding:

In JUC Boston pictures, you can see some nice Jenkins lighting effect, as well as my fellow colleague Corey Phelan using World Cup to lure attendees into a booth:

IMG_8721 IMG_8745

Pictures from the cdSummits are also available here and here.

If you have taken pictures, please share with us as your comment here so that others can see them.

Categories: Open Source

Jenkins Office Hours: dotCi

Surya walked us through the dotCI source code yesterday, and a bunch of ideas about how to reuse pieces are discussed. The recording is on YouTube, and my notes are here.

Categories: Open Source

Jenkins Office Hours: dotCi

Tomorrow in Jenkins office hours, Surya Gaddipati will be going over DotCi, a package of features that integrates Jenkins closely with GitHub, configuration via .ci.yml file in source tree, built-in Docker support and MongoDB backend.

I think there's a number of interesting pieces here that could be split into individual plugins for reuse, and possible alignment with existing efforts like Script Security plugin or Literate plugin.

To record the show, this event will be in a different hangout from the usual one, but the time is the same. Looking forward to seeing you!

Categories: Open Source

Jenkins User Meet-up in London?

I'll be visiting London in early September, and if possible I'd love to organize some get together of Jenkins users/devs. I wonder if anyone is interested in hosting the event?

I think it just needs to fit 20 or so people, so all we need is a single conference room somewhere in London. If you think you might be able to help, please drop us a note at the events list.

Categories: Open Source

With great power comes great configuration

Sonar - Thu, 06/26/2014 - 16:16

We’ve got an ambitious vision for the C/C++ plugin this year. To fulfill it, we started with some under-the-cover improvements to the parser and the internal data model. Those improvements were really just a means to an end, but they’ve had the effect of markedly improving our ability to parse and analyze C and C++ code.

Unfortunately, they came with a downside: a higher analysis configuration burden. For instance, in order to correctly expand macros in the code (and we can, now), we need to know what the macro means. Which means that the macro definition needs to be passed in to the analysis.

Just contemplating the configuration update required for a single large system made me queasy, and I wasn’t the only one. So we set the main plugin aside for a little while this spring and wrote a build wrapper, which will eavesdrop on the tool of your choice (e.g. Make or MSBuild) to gather all the extra configuration data for you.

The build wrapper supports the Clang, GCC and MSVC compilers, and is available in 32-bit and 64-bit versions for Windows and Linux and a 64-bit version is available for OS X. Using it couldn’t be simpler. You drop it somewhere on your machine (make sure it’s executable on ‘nix systems), and prepend your build command with it:


build-wrapper --out-dir [output directory] make

Of course, it needs to be a full build that the wrapper is eavesdropping on, so ideally this command would have come after a make clean. And for MsBuild it would be something like:


build-wrapper --out-dir [output directory] msbuild /t:rebuild [other options]

The output directory is where the build wrapper writes its data files, creating the directory if it doesn’t exist. Currently, the build wrapper simply adds its files to the specified directory, but that behavior could change in the future (E.G. someday it might start by issuing rm [output directory]/*).

The build wrapper writes two files: build-wrapper.log and build-wrapper-dump.json. The .log file is just that – a log that Support may ask for if you ever contact them with questions. The .json file is the one that’s actually used during analysis. This screenshot of the build-wrapper-dump.json from my Linux build of CMake should give you an idea what these files look like:
build-wrapper-dump.json

I’m only posting a brief screenshot because the full file is 43,614 lines long (plus a blank line at the end). I’m not saying that all the information in the file is absolutely required for analysis, but it would have taken me a very long time to identify and specify the pieces that are.

Once the build is complete, and your .json file is written, it’s time to kick off a SonarQube analysis. But first you’ll need to tell SonarQube where to find all that extra configuration data the build wrapper just logged. In your sonar-project.properties add the following:

sonar.cfamily.build-wrapper-output=[output directory]

I end up with a properties file that’s only six lines long (including whitespace), and SonarQube has everything it needs to analyze my project:

sonar.projectKey=cmake-linux-clang
sonar.projectName=CMake Linux CLang build
sonar.projectVersion=1.0

sonar.sources=Source
sonar.cfamily.build-wrapper-output=build

If you haven’t used the build-wrapper on your C/C++ projects yet, you should give it a try and let us know how it goes. Hopefully, it will help you drastically improve the quality of your analyses while dramatically decreasing the configuration.

But you won’t have to tell people it was easy.

Categories: Open Source

JUC Boston, what a day!



We kicked off this year's Jenkins User Conference world tour in Boston this Wednesday. The event was well-attended with more than 450 people registered and 400+ people showed up. So big thank you for everyone who came!

Workflow plugin that Jesse presented was a big hit and lit up twittersphere, and while I was only able to listen to parts of sessions as people had questions and comments for me, ones that I've seen were great. Alyssa told me that the sponsors were happy too, which is also important to keep events like this going.

Perhaps the biggest hit of all was the "get drunk on the code show by Steven Christou. When I got in, he packed 30 or so people in the room learning how to write a simple Jenkins plugin, and all the beer bottles were long gone!

One of the "fun" activities we did during the event was a trivia quiz. I'm happy to announce the winners here — Tamara from IBM and Prabhu from Staples. Congrats for your Amazon gift cards!

During the show, I've heard from several people that they'd love to see more regular local meet-ups. Duncan had shown interest in organizing, and Jesse is a Bostonian, so please encourage them to get one going

Categories: Open Source

JUC Europe Sneak Peak: Integrated Pipelines

This is a guest post from Markos Rendell, a Senior Manager at Accenture.



I am very much looking forward to the Jenkins User Conference in Berlin next week which I will be attending with a three other members of my team. We are all very passionate about automation, infrastructure-as-code, configuration management and of course… Jenkins.

My team and I specialize in implementing continuous delivery for large scale transformation deliveries. We work with a wide range of technologies from open source, packaged products, through to software-as-a-service. We work with physical infrastructure, private cloud, public cloud and platforms-as-a-service, but there is one almost uniquely common factor… using Jenkins.

At the conference I will be expecting to exchange views with others using Jenkins at similar scale and am particularly interested in sessions covering using Jenkins with Docker and making Jenkins more resilient.

I am also looking forward to presenting this lightening talk where I will be demoing ways in which we’ve extended Jenkins to implement complex integrated pipelines for large-scale software implementations. See here for a sneak preview.

Categories: Open Source

What's new in ApprovalTests.Net v3.6

Approval Tests - Wed, 06/18/2014 - 11:17
I finally got a new version released. It's been a lot time in the making and I wanted to highlight some of the new feature. There's a bunch so hold on tight:

VerifyNoAbandonedFilesOne issue that pops up with approvals is sometimes old approval files end up abandoned in the source tree. Now you can write a simple test that will prevent this



Asp ScrubbersIt's great that ApprovalTests can so easily capture web pages, but it can still be hard to get those pages as consistent as a Unit test needs them to be. Inspired by Geoff Bache's TextTest, scrubbers have been included.

Scrubbers are simple Functions that take a input string and return a clean version of it. Here's an example to remove the GUID used by asp for forms

SimpleLogger.MarkSimpleLogger is a easy way to both capture behavior for locking test as well as see performance data for simple profiling but it was also hard to get everywhere a method would exit. Now you don't have to, simply use a using around the method like this.

StatePrinterApprovalsLocking down legacy code can be hard when none of the objects have a good ToString. Enter StatePrinterApprovals. Using the nuget package StatePrinter you can easily print and capture even highly complex and recursive object graphs.Rest Calls Finally there is a simple object to create your own Rest Calls that are both easy to Mock (because they implement ILoader) and easy to Test. Here's the Testing Logicand here the code to test it.

VerifyJsonIt's always been easy to verifyJson, but now it will be pretty printed as well

Serializable Theory TestUsing Theory based testing, you can now test that any object can be serialized and consumed to recreate the original object. It's as simple as 

WindowRegistry.AssertsYou can also easily assert that WindowsRegistry Values are in place with

WriterFactoryApprovalTests is more extensible with the introduction of WriteFactory. An easy way to insert your own FileWriters if you need them
ApprovalDataScenariosFinally, the last big change is the introduction of DataScenarios. This allows you to use data driven test cases to produce multi approved files.
Categories: Open Source

Get together at beer garden for JUC Berlin

There'll be a number of active community people in the event, so let's take advantages of that and meet up. And there's no better place to do it than a beer garden in summer!

If you are coming to JUC Berlin, I've just set up an RSVP page for a beer garden get together the day before, and another dinner afterward.

Looking forward to seeing you!

Categories: Open Source

JUC speaker sneakpeak: A build ecosystem for loosely compiled code



We're getting excited about the Boston and Berlin JUC's in the next two weeks! Here's a preview of Forest Handford's upcoming JUC-US East Lightning Talk on June 18...

When MEDITECH migrated to Subversion from a home-grown first generation version control system we needed a way to get the code compiled and sent to the running server. We selected Jenkins as our build server, with the hope of eventually using it for CI.

A MEDITECH application consists of hundreds of source files. Each source file translates to an object code file that the interpreter executes. This is one of the last major projects I worked on prior to leaving MEDITECH to work at Carbonite. In my Lightning Talk, "A Build Eco-System for Loosely Compiled Code," I'll discuss the toughest challenges my team had in getting Jenkins to work as our build server and how we eventually overcame them.

Staff from both Carbonite and MEDITECH will be in attendance. Both companies are hiring!

You'll find more great talks in the full JUC-US East agenda and the JUC-Europe agenda.

Categories: Open Source

Two Weeks Till Jenkins User Conference Berlin

Following right on the heels of our US-East Jenkins User Conference in Boston, we have JUC Europe in Berlin on June 25. Like the East Coast conference, the Berlin one is almost full, so sign up while you can.

Our venue in Berlin is KOSMOS, a building with a fascinating history. The building was inaugurated as a cinema in 1961. With 1001 seats, it was the largest, most modern and most popular film theatre in the former GDR and has since been extensively modernized in line with the requirements of historically listed buildings.

We have an excellent line-up of speakers filling up two conference tracks.

Once again, we have some fabulous sponsors to thank. Without them, there would be no JUC.

This year we’ve introduced a new Community sponsorship level, which allows non-corporate groups like JUGSs to help support the conference as well. (Drop a note to juc-oc-ext AT cloudbees DOT com if your group is interested).

We are very grateful to all of our sponsors – thank you! Hopefully see everyone at JUC.

PS – if you are coming to Berlin for JUC, check out the CD Summit on June 24 at the same venue. There’s also one on June 19 in NYC. The summit will focus on how continuous integration with Jenkins streamlines processes and automates testing and deployment, providing the foundation you need for continuous delivery.

Categories: Open Source

Thinking about moving on to Servlet 3.0



One of the current efforts under way in the dev list is driven by Tom Fennelly et al, who is working on introducing a series of small ball improvements to the user interface in Jenkins. If this is something you are interested in (and who aren't?), you should see Kevin Burke's manifest that sets out the plan of attack, and This mega thread on the dev list for the discussion.

There are numerous sub-conversations born out of this, and one of them is the minimum required servlet spec version for Jenkins.

Jenkins devs are thinking about ways to update page contents post load, for example so that the list view will keep updating as stuff happens. WebSocket was discussed as an option, and then server-side events, which seems to be the current favorite.

To use any of those async HTTP features, we need servlet 3.0. Unfortunately, if we are to do it, Jenkins will not run on earlier versions of the container. There's no graceful fallback that works with servlet 2.5 containers due to the way servlet 3.0 is written.

So I looked into the impact of this change to the users.

It turns out that the most users run Jenkins through java -jar jenkins.war, which are already running servlet 3.0 compatible Winstone 2.x (based on Jetty 8.) And people running newer version of Jenkins tends to run newer version of containers. When I look at people who are running >=1.509 and later, 70% of them run on servlet 3 compatible container. The number for >=1.532 is 84%, then for >=1.554 it's 94%.

When I look at which container is dragging us down as of >= 1.554, you see that there's a sizable Tomcat6 deployments (2.5%). If we start requiring Servlet 3.0 these people will be in a nasty surprise. Then there's about 1.8% who claims to be running on Winstone 0.9.10, which is really puzzling, but I'm assuming these people are getting OEM-ed Jenkins of a sort (multiple large companies are known to do this), so these people will likely be able to update to Winstone 2.x automatically by virtue of getting a new jenkins.war from their upstream. So all in all I'd say if we start requiring servlet 3.0 today, there'll be about 3% user base who will be impacted.

This post is a trial balloon to see the community reaction to this idea. If you have reasons to argue against us moving to servlet 3.0, we'd like to hear from you — please share your thoughts on our issue tracker!

Categories: Open Source

Other events for JUC visitors: CD seminar



If you will be on the US East Coast or in Berlin for JUC, some of the JUC sponsors are organizing separate events called Continuous Delivery Seminar, which might be of interest to you. These events focus more on higher-level business value questions as well as vendor solutions that are difficult in community-focused JUC.

  • New York City on June 19, the day after JUC US East — headlined by Forrester Research analyst Kurt Bittner
  • Berlin on June 24, the day before JUC Europe — headlined by Jan Hagen, author of Confronting Mistakes: Lessons from the Aviation Industry when Dealing with Errors

Read more about the events here. The events are free and I've heard that there'll be some souveniors. I'm one of the speakers, and I'll be talking about Jenkins, as always!

Categories: Open Source