I am quite proud to announce a new free e-book about test strategy.
It contains ideas Henrik Emilsson and I have have discussed for years.
It is not a textbook, but it contains many examples and material that hopefully will inspire your own test strategies (the careful reader will recognize stuff from this blog and inspiration from James Bach’s Heuristic Test Strategy Model.)
Reader requirement: Understand Swedish.
Download Den Lilla Svarta om Teststrategi
Protecting your valuable digital property is extremely important. Encrypting client/server communication is one way to make sure your data is secure from potential hackers. Surround SCM 2014.1 introduces improved encryption methods and a new, stronger security option: RSA key exchange.Configuring encryption and server security
Communication between Surround SCM clients and the Surround SCM Server should always be encrypted. At a minimum, make sure the Encrypt communication between clients and the server option is enabled in the global General server options.
If you need stronger encryption, you can use RSA key exchange, which is a public key algorithm that uses separate keys for encryption and decryption. You may want to use RSA key exchange if:
- Your organization stores sensitive information in Surround SCM.
- Your network is potentially insecure.
- Users log in to client applications from outside your network.
- Users are authenticated to Surround SCM using LDAP, single sign-on, or external authentication.
To use RSA key exchange, select Use RSA key exchange in the global General server options. This generates the required public key that must be added to clients that connect to the server. Click Download Public Key File to download an XML file that includes the Surround SCM Server address, port number, and public key. Make sure to save the key file in a secure location.
After RSA key exchange is enabled, the public key must be added to server connections in clients so users can access Surround SCM. This process is slightly different for different client types.
Distribute the key file to all users who use the Surround SCM Client or CLI. Client users must import the file to their server connection settings. For example, on the Surround SCM Login dialog box, click Setup. Select the server connection to add the public key to and click Edit. Click Import, select the key file, and click Open. Click OK to save the changes.
CLI users simply need to save the key file and enter the full path to it in the -z option with commands instead of entering the server connection information.
If users use Surround SCM Web or proxy servers, only the Surround SCM administrator needs to import the key file using the Surround SCM Registry Utility. For example, click Web Options or Proxy Options in the registry utility and then click Import on the Server Options dialog box. Select the key file and click OK to save the changes.
The same encryption and key exchange principles apply to the Seapine License Server. Always enable encryption to make sure communication is secure between the license server, admin utilities, API, and other Seapine product servers. To enable encryption and RSA for the license server, click Server Options in the license server admin utility and select the Server category.More information
For more information about encryption and key exchange, see the following help topics:
- How encryption, authentication, and key exchange works
- Setting general global server options
- Configuring RSA key exchange
If you also use TestTrack, see our related blog post about securing TestTrack client/server communication.Share on Technorati . del.icio.us . Digg . Reddit . Slashdot . Facebook . StumbleUpon
Sauce is thrilled to announce that we’ve integrated with Siesta by Bryntum!
Sauce is a cloud-based tool that enables you to securely test your web and mobile apps across 385+ browser/OS/platform/device combinations.
How does Siesta work with Sauce?
Siesta’s integration with Sauce is super easy since they’ve done most of the work under the hood. The main difference from the usual way you run WebDriver tests in Siesta is the addition of a –saucelabs argument, which will define the necessary credentials for connecting to Sauce and the –cap arguments which will specify the OS and browser combinations on which you want your tests to run. When you run Siesta tests using WebDriver, a standard command might look like this:
For running the same test on Sauce’s hosted browsers, you’ll need to add your Sauce credentials and desired capabilities, like so:
__SIESTA_DIR__/bin/webdriver http://localhost/myproject/tests/harness.html --saucelabs SL_USERNAME,SL_KEY --cap browserName=firefox --cap platform=windows
This command will then arrange a few things behind the scenes, like setting up the Sauce Connect tunnel so these tests can run locally and securely, and direct Sauce to execute the test specs on the latest version of Firefox on Windows. Of course, there are many desired capabilities and platforms you can choose to make sure you’re running tests according to the coverage that you need, and you can see all the different options on the Sauce Labs Platforms page.
For more information and the nitty gritty details, go ahead and check out the instructions provided in the Siesta docs: http://bryntum.com/docs/siesta/#!/guide/saucelabs_integration
What does the integration mean?
For more information, check out the Brynthum blog post.
- Michael Sage, Principal Technology Evangelist, Sauce Labs
Michael Sage is a Principal Technology Evangelist at Sauce Labs who helps software teams develop, deliver, and care for great apps. He’s spent over 15 years as a solutions architect and consultant with software companies including Mercury Interactive, Hewlett Packard, and New Relic. He lives in San Francisco, CA.
LibreOffice, is a leading an open source office suite developed by the Document Foundation. The project has been a member of the Coverity Scan Service, which allows open source projects to use our award winning static analysis solution for free, since 2012. The LibreOffice team has been very busy in the last several months improving their code quality and security. Since last November, they have fixed more than 6,000 defects in their code through the service and achieved a defect density rate of just .08% which is significantly lower than like-sized open source and commercial projects. In our 2013 Coverity Scan Report, we found open source projects greater than 1 million lines of code had an average defect density rate of .65% and the same size commercial projects had a .71% defect density rate.
By fixing more than 6,000 defects in less than one year through the Coverity Scan Service, we estimate that they saved almost 43,000 developer hours. Those savings arise from both the automated nature of the analysis, which reduces the effort required to find problems, and the way in which we report issues which makes it easy to recognize the real problems, understand what is wrong, and devise fixes.
The most common defects the LibreOffice project fixed were:
- Error handling issues 2,271
- Null pointer dereferences 1,796
- Uninitialized members 1,145
Read the new Coverity Scan LibreOffice Project Spotlight to learn more.
We would like to applaud the LibreOffice team for their commitment to quality and thank them for their support. Sign up to register your open source project in Coverity Scan. If you would like to keep track of LibreOffice, register to become a project observer.
The post LibreOffice: Improving Quality Through the Coverity Scan service appeared first on Software Testing Blog.
To read more, visit our blog at blog.sonatype.com.
Note: The following is a guest submission to the uTest Blog from Sanjay Zalavadia.
By considering the performance of in-development software from the perspective of the end user, QA teams can better address disruptive issues.
Software testing can often be an arduous and stressful process. Even in traditional waterfall production methods, quality assurance teams are typically faced with a months-long period colloquially known as the “death march” as developments near release. During these moments, QA management and teams hunker down and toil away, attempting to address as many remaining coding flaws as possible before the software goes into production. The proliferation of agile development principles has only escalated this trend as QA members are constantly working to identify areas of improvement during the entire course of development.
It’s understandable if QA objectives become a little shortsighted under these conditions and testers place all of their focus on finding bugs and coding errors. However, testing managers need to remain cognizant of the ultimate goal of any successful development process: optimizing the end user experience.
QA performance cannot be measured by the number of bug reports generated, but by the satisfaction of software users following a product’s release. To that end, it is advantageous to consider the viewpoint of the consumer and incorporate user feedback into the development process.
Usability critical to software performance
In a truly agile software development project, user feedback is a critical component of the production cycle, helping to guide tester and developer efforts to improve the performance of the application.
By considering how individuals engage with a piece of software and what problems may commonly occur or will be most disruptive to the user experience, developers and QA teams can better focus on addressing those issues. That fact is that despite the best efforts of software testers, coding flaws are essentially an inevitability. No software is 100 percent perfectly written, but the most successful programs are often those that perform at an optimal level with a bare minimum of usability issues.
In a Software Testing Help post, quality assurance expert Santhosh Kumar Ponnusamy outlined several of the traits characterizing a successful tester. In particular, he highlighted the openness to consider the end user viewpoint and the staunch commitment to improving consumer satisfaction.
“Every product is developed for customers,” Ponnusamy wrote. “Customers may or may not be technical persons. If you don’t consider the scenarios from their perspective, you will miss many important bugs. So put yourself in their shoes. Know your end users first. Their age, education, even the location, can matter most while using the product. Make sure to prepare your test scenarios and test data accordingly. After all, a project is said to be successful only if the end user is able to use the application successfully.”
IT Business Edge’s John Storts explained that successfully incorporating user feedback can often be the difference between an effective agile development project and a more troubled production process. He noted how during an attempt to implement agile principles, one of his past employers quickly fell back into the routine of a waterfall approach. Despite a lot of lip service being paid to agile development early in the design process, once the ball got rolling, company members from management down to testers returned to traditional models of operation.
“Our software builds, or ‘iterations,’ managed to roll in bug fixes and feature requests from the developers, other project team members and in-house software testers, but I rarely heard of end user requests making it into pre-release versions,” Storts stated. “Communication of this kind during early development stages is one major distinguishing factor between waterfall and agile methods.”
For the most benefits from user feedback, it’s often wise for QA teams to first develop a concept of persona to guide the testing process. It can take a concerted effort from QA leadership to properly integrate user feedback with testing efforts, but the shift will ultimately pay off with better performing software releases.
Sanjay Zalavadia is the VP of Client Services for Zephyr, who offers a real-time, full-featured test management system. Learn more about Zephyr right here.
This story came in from Joseph – one of our fellow dynaTrace users and a performance engineer at a large fleet management service company. Their fleet management software runs on .NET, is developed in-house, is load tested with JMeter and monitored in Production with dynaTrace. A usage and configuration change of their dependency injection library […]
The post Detecting Bad Deployments on Resource Impact and Not Response Time: Hotspot Garbage Collection appeared first on Compuware APM Blog.
Carmakers, their suppliers, and developers of automotive components can quickly and cost-effectively prove compliance with the ISO 26262 standard through the use of an integrated product development management solution.
Our new guide, “Managing ISO 26262 Compliance with Seapine Software,” provides a brief overview of ISO 26262. You’ll also learn how Seapine’s integrated product development management solution makes verification easier, less error prone, and more cost effective by automating the creation, management, maintenance, and documentation of requirements traceability.
Download your free copy now to learn how Seapine’s solution, which includes TestTrack and Surround SCM, will help your team meet the requirements and challenges of proving compliance with ISO 26262.Share on Technorati . del.icio.us . Digg . Reddit . Slashdot . Facebook . StumbleUpon
Challenges in automation which testers face often lead to subsequent failures. Learn how to respond to these common challenges by developing a solid business case for increased automation adoption by engaging manual testers in the testing organization, being technology agnostic, and stabilizing test scripts regardless of applications changes.
Learn Jim Trentadue’s explainations of a variety of automation perceptions and myths.
Don’t miss his session!
Thanks to those of you who attended our last webinar, How To Combine Front-End and Back-End Testing, featuring our Chief Technology Evangelist Michael Sage and BlazeMeter‘s VP of Customer Success, Ophir Prusak.
Everyone knows that front-end testing is crucial to make sure your web and mobile apps are meeting the needs of your users and customers. But how do you know what will happen to your front end when your web or mobile app is under heavy load?
Michael and Ophir set out to help answer this question, plus more. Together, they covered many valuable topics, including:
- The fundamentals of approaching performance vs. front-end testing
- Step-by-step instructions on getting real-world results from your front-end while applying load to the back-end
- Critical issues you need to know about performance testing
They also showed a real-world test in real time using JMeter and Selenium.
Missed the webinar? You can watch it in its entirety below.
Below you’ll find the top Q&A’s post-presentation:
Q: Why do we need JMeter? Why can’t we do performance testing with Selenium browsers?
A: [Ophir] Great question! The answer is really all to do with scalability. When you’re doing a test with Selenium, you’re usually using real browsers –and real browsers are very resource intensive. On the other hand, when you’re doing a test in JMeter by using virtual users, I can exponentially get a lot more users per machine. Just to give you an idea of ballpark numbers, on a single low-end Amazon EC2 machine, I can get 3,4 or 5 virtual users using browsers in parallel until I hit a bottleneck on the machine. If I’m doing it with a JMeter, I can get 1,000 people in parallel. So you’re looking at around 250X more users when I’m doing a JMeter test per machine than Selenium. So if you’re looking at ten or 100 users, you can do it with Selenium but when you’re looking at tens of thousands of users, it just won’t be able to support it.
Q: Can I test behind the firewall?
A: [Michael] Yeah, absolutely We have a utility called ‘Sauce Connect’, which creates an encrypted tunnel between your environment inside your firewall or your DMZ and our grid and you get a dedicated virtual machine to act as the tunnel property. You run this utility and it appears as if the Sauce Labs grid is inside your network and it’s all done in an encrypted fashion.
[Ophir] At BlazeMeter, we also have the ability to run behind the firewall. It’s a different type of solution. It does require having something we call a BlazeMeter agent, which is basically a load generator which sits behind the firewall, which you can still control through your browser but you have a local machine which is creating the requests for you.
Lastly, please follow our friends at BlazeMeter at @blazemeter, Sauce Labs at @saucelabs, Michel Sage at @mondosage, and Ophir Prusak at @prutwo to keep up with the latest. Feel free to share this webinar using the hashtag #frontandbackendtesting.
Video captured by testers is an integral piece of the puzzle for developers; faster understanding of the problem you‘re encountering means faster identification the root issue to enact the necessary changes.
If you’ve ever tried to find a screen-mirroring tool for Android, you’ve likely waded through dozens of applications that at first seem like solid solutions, before ending in headaches and time wasted on confusing setup steps. Some even require your device to be rooted, which is no use if you’re participating in testing that prohibits using modified devices!
If only there were a simple way to project what you see on your Android device’s screen to a computer monitor! Alas, there are indeed a few tools you can start using today, but which is the best?
- Some testers have praised Droid@Screen for its intuitive, user-friendly interface while others down-rate its ability to stream video at a usable rate and its lack of an in-app video recorder.
- How about Mobizen? Touted as easy to install with good frame rates, yet the input method changes for different applications, resulting in some user confusion.
- MyMobiler is the go-to for some uTesters, who enjoy its wireless connection and keyboard control during recording but have experienced periodic disconnection problems.
- Heard of AndroidScreenCast? Some features of the app require your device to be rooted, so do the pros outweigh the cons?
Hop on over to our Tool Reviews section and check em’ out, then tell us what you think! Have any better suggestions for Android screen mirroring apps? Submit them to our Tool Reviews repository and share your hard-earned knowledge.
So who wins the Android screen mirroring tool showdown? Leave a comment below and let’s discuss!
- How to build a modern continuous delivery pipeline with Jenkins
- Connect Jenkins and Puppet such that Dev and Ops team can determine what happens on the other side of the house and closely interact to debug issues in production environments.
Webinar recording is here.
Following are answers to questions we received during the webinar:________________________________________________________________
Q: Is Puppet serving as the orchestrator for Jenkins?
A: Not quite - the tools run independently but communicate with each other. The demo will make it clear.
Q: Can JMeter be plugged in with Jenkins for Continuous testing?
A: Yes it can.
Q: When we say continuous testing do we mean automated testing here?
A: Continuous Testing = automated testing for each commits made in the source repository.
Q: What drivers or plugins are required? Can I get a website where I can get this info?
Q: With JMeter can we run a load test using the build in Jenkins, or how can we do continuous testing with this combination?
A: JMeter is going to used for load testing stage. It depends how you setup your workflow/pipeline. If you run perf test on every commit (you shouldn't) but you have continuous testing. You will have more testing stages ideally.
Q: Can Puppet work with VM's
A: Yes, Puppet can work with VMs. Puppet agents live at the OS level, and can be deployed to virtual machines or bare hardware. Puppet is agnostic to where or how it has been deployed. We do have some hooks and integrations around provisioning new VMs as well.
Q: I'm curious that I don't see AWS/EC2 under "Virtual & Cloud" for Puppet along with VMware, Xen, Azure ... is there a reason? Any concerns I should have about compatibility with EC2 infrastructure?
A: No, there are no concerns around EC2. Puppet runs great in EC2 and we have many customers running their infrastructure with Puppet in Amazon's cloud.
Q: Are you going to share these scripts somewhere?
A: Demo write up available on CloudBees developer wiki. The jenkinsci infrastructure is available at https://github.com/jenkinsci/infra-puppet
Q: I understand that Puppet helps create an MD5 hash file of the war file - build deployments. Could you provide a basic definition of what is Puppet and what is Docker?A: Puppet (stealing from the Puppet page)
Puppet Enterprise (PE) uses Puppet as the core of its configuration management features. Puppet models desired system states, enforces those states, and reports any variances so you can track what Puppet is doing.
To model system states, Puppet uses a declarative resource-based language — this means a user describes adesired final state (e.g. “this package must be installed” or “this service must be running”) rather than describing a series of steps to execute
Docker (stealing from Docker.io)
Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications. Consisting of Docker Engine, a portable, lightweight runtime and packaging tool, and Docker Hub, a cloud service for sharing applications and automating workflows, Docker enables apps to be quickly assembled from components and eliminates the friction between development, QA, and production environments. As a result, IT can ship faster and run the same app, unchanged, on laptops, data center VMs, and any cloud.
Q: Will this work with SVN too?
A: There is an equivalent version of Validated Merge for Jenkins that our team has pushed out in OSS.
Q: Will Validated merge with SVN repo too?A: See above.
Q: Is an equivalent to the gated repo available with subversion? It's a great idea; a while back I'd worked with a similar homegrown solution for Perforce.A: See above.
Q: What's the difference between open source Jenkins & CloudBees's version?A: See this link.
Q: Where I could get the quotation if I want to buy?A: Email email@example.com
Q: Does Puppet require root access for Unix host? What privileges would it require as a user?A: The Puppet agent typically runs as root in order to be able to fully configure the system, but it does not require those privileges. When running as a non-privileged user, it will only be able to manage aspects of the system the user has permissions for.
Q: When Harpreet was doing the Traceability demo, the Jenkins screen that showed the artifact deployment state had a field for 'Previous version' that was blank. Why was that empty? What value would normally be in there, the MD5 hash of the previous artifact?A: Those would change if I had checked in new code thus altering the MD5 hash. Since I was just rebuilding the same image in the demo, the hashes are same and hence no previous version.
Q: Is Puppet capable to work with IBM Solutions? like Websphere?A: Yes. In general, if it's possible to manage or modify an application from the command line of a system, it is possible to build a Puppet model for it. Check out forge.puppetlabs.com for 2500+ examples of pre-built community and supported modules.
Q: I read that about the agent, but what about the master? If not, can you run Puppet without a master?A: The master is effectively a web service, which does not require root privileges, so it too can be run without root. For testing and development, you can run Puppet in a stand-alone mode using the `puppet apply` family of commands.
Q: Does Puppet need vagrant to run or can we run it directly on the VM?
A: Puppet can be run directly on a VM. It does not have dependencies on Vagrant or any other specific virtualization/cloud management software.
Q: How does the facility match with the preccommit checkin provided by Visual Studio Env?A: I am not familiar with Visual Studio Env but documentation indicates that those are just environment variables that are in injected into builds, if so then Jenkins can understand environment variables.
-- Harpreet Singhwww.cloudbees.com
Harpreet is vice president of product management at CloudBees.
Follow Harpreet on Twitter. -- Reid Vandewiele
Reid is a technical solutions engineer at Puppet Labs, Inc.
About two weeks ago I got an Ad in my mail for an iPhone Application called Breeze, a simple App that counts how many steps you take during the day.
What they do a little different from other Apps I saw before is that they set you a personal target for the number of steps you should take each day.
They do this based on information they have from you, either from another app by this same company (the app I use to track my running), or from the information stored on your iphone about your movement patterns.
You can track your progress during the day, you can also get “motivational messages” as you make progress (by walking to the cafe in the corner to eat your lunch…), and as you meet your daily goal they display for you a congratulations message with a small trophy and some cheap screen confetti.
It may sound silly, but since I installed this App I am trying am trying to make an effort to meet my daily goals. I’ve even started to take my dog for longer walks in the mornings and evenings .
The principle is simple…
Anything you measure will improve.
If you weigh yourself once a week, you will eat more salads and less hamburgers for lunch. If you hang a board where you mark if your kids finished their school chores before 6:00 PM each day, you will see how they start making homework before you even need to remind them.
Many times there is no need to define a reward for the actions, the actual fact that you are seeing your achievements in front of your eyes will serve as the motivational factor to generate the change.
This is the same principle that make Kanban Boards such an effective tool, by giving visibility to everyone in the team about the stuff that is progressing and the things that are stuck in your project.
Taken to testing…(?)
This is also something we can use in our favor in, both to change our own behaviour and to help us modify the behaviour of others in our favor.
On a personal level, you can set daily or weekly goals of learning new techniques X amount of time, and spending on test design and execution Y amount of time, and providing feedback to the Product Team on Z number of features.
Then create a calendar that you hang it on your cube or office, where you will mark with green all the days where you met that goal and with red all the days when you didn’t.
I dare you to do this, and report how many days are green and how many days are red at the end of 1 month!
On a group level, in PractiTest we have a Kitchen Monitor where we display different graphs with information on important things happening in the team and project.
Sometimes this may be about test coverage that is lacking in some areas of the product, other times it may be about bugs accumulating in a given feature to be released.
The interesting thing is that whenever you place this information in front of everyone to see, even if you don’t say a word about it out loud, the status of these areas tends to automatically improve
But be careful with the other side of the coin
Just like every other tool, there are side effects to the use of metrics.
The two side effects that I think are the most important are: “Metric Tiredness” & the “Not Measured Syndrome”.
Metric Tiredness refers to measurements that are defined and left to run for months or years without making any changes or adaptations.
Just like your ears will get used to the noise of the Air Conditioning in your office (and you will notice its absence when the power goes down!), so will your team learn to ignore the metrics after a while. Many times these static metrics can also become irrelevant over time.
This means that you need to make sure to refresh your metrics, even artificially, once in a while to keep them fresh and to keep your team interested on them.
The Not Measured Syndrome refers to the fact that just as what you measure tends to improve, these improvements come at the expense of the things that you don’t measure. So be careful to choose what is really important.
Also, if you are thinking that in order to work around this issue you will measure a lot of things together, I will add here another side-effect to measures and that is that if you try to measure too many things at once, then all your metrics will become irrelevant because you won’t be able to focus your team on the specific things that are important to you.
Do you have any type of Public Measurement or Kitchen Monitor in your team?
Share with us what do you measure and any important tips you’ve learned about them from your personal experience!
The more .NET developers we meet, the more we grow to appreciate what true craftsmen they are, whether behind the keyboard or behind the lens. Keep up the great work and thanks for all you do for .NET.Nikhilesh Singh
Nikhilesh Singh is a programmer by profession and photographer by hobby. He has been part of team building software solutions for more than 13 years and has loved learning, sharing, caring, mentoring and managing teams. He is a Microsoft Certified Technology Specialist and Professional Developer and has built both thick as well as thin client solutions. He is passionate about honing skills to be a better software craftsman, be it using awesome tools like NCover or reading and watching Uncle Bob’s articles and webcasts. He loves to learn about functional programming and has been pursuing it quite a lot lately. When not coding, you will find him with his tripod and camera photographing his beautiful surroundings or participating in webcasts. You can follow him on Twitter at @nikhilsingh.Hector Romano
Hector Romano, based in beautiful Buenos Aires, is a developer who has a love for creativity and enjoys experimenting with various techniques and patterns such as MVC and MVVM in web fields, as well as using service-oriented architecture (SOA) and aspect oriented (AOP) programming architecture. He has various Microsoft certifications to his credit and also enjoys using Agile development methodologies like SCRUM. He has a particular spot in his heart for developing database software, creating interconnections for different platforms and working in languages using web services. Learn more about him by following him on twitter @RomanoTulioHec.