Skip to content

Hiccupps - James Thomas
Syndicate content
James Thomas
Updated: 1 hour 53 min ago

Speaking Easier

Wed, 01/18/2017 - 22:50

I've been thinking about public speaking and me.

Wind back a year or so. Early in November 2015 I presented my talk, my maiden conference talk, the first conference talk I'd had accepted, in fact the only conference talk I had ever submitted, on a massive stage, in a huge auditorium, to an audience of expectant software testers who had paid large amounts of money to be there, and chosen my talk over three others on parallel tracks. That was EuroSTAR in Maastricht. I was mainlining cough medicine and menthol sweets for the heavy cold I'd developed and I was losing my voice. The thing lasted 45 minutes and when I was finished I felt like I was flying.

Wind back another year or so. At the end of July 2014 I said a few words and gave a leaving present to one of my colleagues in front of a few members of staff in the small kitchen at work. I was healthy and the only drug I could possibly be under the influence of was tea (although I do like it strong). The thing lasted probably two minutes and when I was finished I felt like I'd been flushed out of an aeroplane toilet at high altitude.

Wind forward to today, January 2017. In the last couple of years, in addition to EuroSTAR, I have spoken a few times at Team Eating, the Linguamatics brown bag lunch meeting, spoken to a crowded kitchen for another leaving presentation, spoken to the whole company on behalf of a colleague, spoken at several Cambridge tester meetups, spoken at all three of the Cambridge Exploratory Workshops on Testing, spoken at the Midlands Exploratory Workshop on Testing, spoken at the UK Test Management Forum, and spoken at a handful of local companies, and opened the conference (yes, really) at the most unusual wedding I've ever been to.

I'm under no illusion that I'm the greatest public speaker in the world; I'm probably not even the greatest public speaker in my house. But, and this is a big one, I'm now confident enough about my ability to stand in front of people and speak that it's no longer the ordeal it had turned into. In fact, at times I have even enjoyed it.

Now back to 2014 and that kitchen. I stood stiffly, statically, squarely front of the fridge. Someone tapped the glass for quiet and as I spoke my scrap of paper wobbled, and my voice trembled, and my knees knocked.

The worse I felt about the delivery, the worse the delivery seemed to get, and the worse I felt, and the worse it seemed to get ... After stumbling back to my desk I decided enough was enough: I was going to do something about my increasing nervousness at speaking in public. And so, on the spur of the moment, I challenged myself to speak at a testing conference.


I found that the EuroSTAR call for papers was open, and I wrote my proposal, and got some comments on it from testers I respect, and I rewrote my proposal, and I sent it off, and I crossed my fingers without being quite sure whether I was hoping to be accepted or not. Then, if I'm honest, I made very little progress for a couple of months, until I came across Speak Easy.

Speak Easy team inexperienced speakers with experienced mentors to help with any aspect of conference presentations. It sounded relevant so I signed up and, within a few days, James Lyndsay got in touch. In our first exchange, this is what I told him I wanted:
  • Tips, strategies, heuristics for keeping the nerves in check - ultimately, I'd like to be able to stand in front of anyone and feel able to present.
  • Tips for building, crafting, structuring presentations and talks - I imagine that confidence in the material will help confidence in delivery.
  • Any other relevant suggestions.

Amongst other things, he asked me questions such as what did I mean by nerves? When did I get them? And what was I currently using to moderate them?

Amongst other things, he gave me a suggestion: "having confidence in your material can help, but not as much as knowing the stuff".

Amongst other things, he assigned me a task: visualising a variety of scenarios in which I was required to speak in front of different audiences (people I knew, experts in my field, experts in an unfamiliar field, ...) from different positions (presenter, audience member, ...).

Amongst other things he had me watch several talks, concentrating on the breathing patterns of the speakers rather than their words.

Based on my responses, he proposed further introspection or experimentation. In effect, he explored my perception of and reaction to my problem with a range of different tools, looking for something that might provide us with an "in". In retrospect, I think I could have done more of this myself. But, again in retrospect, I think I was too close to it, too bound up in the symptoms to be able to see that.

Amongst other things, and a little out of the blue, for both of us, he mentioned that I might look into Toastmasters on the basis of Tobias Mayer's blog post, Sacred Space, published just a few days previously. So I did. In fact, I went to the next meeting of Cambridge City Communicators, which was the following week, and I stood up to speak.

I reported back to James afterwards: I was thrown an "agony aunt" question and had to answer it there and then, with no prep time. I was nervous, I was pleased that I didn't gabble, I deliberately paused, and my voice didn't (I don't think) shake.  They told me that I was very static (they are hot on body language and gesture) and I ummed a little. But my personal feedback is that although I was able to some extent to overcome the shakes and the thumping chest, I wasn't my natural self.  I was concentrating so much on the medium that the message was very average. So I think I want to tune my goal in Speak Easy: I want to feel like myself when speaking in front of a group.

I can't emphasise how big a deal this last point was for me. It changed what I wanted to change. I realised that I could live with being nervous if it was me that was nervous and not someone else temporarily inhabiting my body.


And that was just as well, because during this period I got an email from EuroSTAR. I'd been accepted. Joy! Fear!

So I signed up to Toastmasters and committed myself to stand up and speak at every meeting I attended, and to do so without notes from the very beginning, and to do it wholeheartedly. I learned a few things:
  • I can write a full draft and then speak it repeatedly to make it sound like it should be spoken.
  • That rehearsal lets me smooth out the places where I stumble initially, and find good lines that will be remembered and used again. 
  • Experimenting with how much rehearsal I need to get the balance between natural and stilted right was useful because I can now gauge my readiness when preparing (to some extent).
  • Standing and sitting to speak are different for me. Standing is much more nerve-wracking, even alone, so now I try to practice standing up. 
  • I can squeeze rehearsal into my day, if I try. For instance, I'll put my headphones on and (I hope) appear to be having a phone conversation to anyone I walk past as I do a lap of the Science Park at lunch times.
  • Speaking without notes from the start forced me to find ways to learn the material.
  • Doing it more helps, so I sought out opportunities to speak.

I attended Toastmasters religiously every two weeks and kept up my goal of speaking at every meeting in some capacity. The possibilities include scheduled talks, ad hoc "table topics" where people volunteer to speak on a subject that's given to them there and then, and various functional roles. Whatever I was doing, I'd look for a way to prepare something for it, or dive into the unexpected with full enthusiasm.

I frequently didn't enjoy either my performance or my assessment of my performance, but I found that I could see incremental improvement over time. I used James as a sounding board, reporting back to him every now and again about problems I'd had or victories that I felt I'd won, or about the positive things that attending Toastmasters was giving me:
  • The practice: to get up and speak on a regular basis in front of a bunch of people for whom, ultimately, it made no difference whether I was good, bad or indifferent.
  • The formality:  I found that the ceremony and rigidity removed uncertainty, allowing me to focus more on the speaking.
  • The common aim: the people there all want to improve as speakers, and want others to improve as speakers too, and that gives a strong sense of solidarity and security. 
  • The feedback: in addition to slips of paper filled in by each member for each speaker there is feedback on every speech from another Toastmaster, delivered by them as a speech in itself.

Talking of feedback, a summary of the advice I was given in the eight or nine months I was there might be: speak clearly, don't be afraid to pause, include variety in my voice, use my hands to emphasise and illustrate points, use some basic structural and rhetorical devices, stop rocking backwards and forwards and shuffling my feet, stop touching my nose

Other than the last couple, which are habits I had no idea I had, this is standard advice for beginner speakers. What's useful, I found, is to get it applied to you regularly about some speaking you've just been doing, rather than reading it in a blog post when you haven't been anywhere near presenting for months.

But enough of that, because suddenly it was the start of November and I was in a taxi, in a plane, in a taxi, on a train, in a taxi, in front of a stage at a conference centre in Maastricht waiting to deliver my talk.

And then I was on the stage. And I had a headset mic on - which I had never done before.  And I was coughing, and the sound tech was coughing. And we shared my cough sweets. And I was being introduced. And I was stepping forward from the side and ... and ... and ... amazingly I found that I was smiling.

And I was interacting with the audience. And I was making a joke. And they were laughing. And I wasn't shaking. And my voice wasn't catching. And I was delivering my talk in what felt like a natural way, with pauses, at a natural pace ... and although I can't be sure what I was doing with my feet, I can say that my head was very, very big.


A few weeks later, I got an email from the organisers:
Thank you for contributing to the success of EuroSTAR Conference 2015, we hope you enjoyed the experience of speaking in Maastricht. We have amalgamated all the information from attending delegates and for your feedback scores and comments on your session are included below: Individual speakers were evaluated by delegates using a 1-10 basis (10 being excellent - 1 being poor).We categorize sessions by the following standards:
  • 9.00 – 10.00 Outstanding
  • 8.00 – 8.99 Excellent
  • 7.00 – 7.99 Good
  • 6.00 – 6.99 Low Scoring
  • Under 6.00 – Below minimum standard acceptable
Your score was  5.90 out of 67 respondents which according to above table, came in the Below Minimum bracket. The track session presentation overall average score (40 track sessions) was 7.51 Comments on Forms below:-
  • Well, fun but what am I going to do with this?? (+ some jokes don’t work on non-British people).
  • Accent!
  • as hard to understand if you're not a native in English language
  • The core ideas turned out more interesting than I expected, but needs post processing by me.
  • Good presentation but very specific to native speakers.  Really good work done on linking patterns but I think will not reach wide audience 

And I'd got similar comments directly too. I'd known that including jokes themselves (in a talk called Your Testing is a Joke, about the links between testing and joking) was a risk to non-native speaker comprehension from my practice runs, and I'd changed the talk to reduce it. It's also indisputable that I have an accent (I'm from the Black Country and it shows) and I think that having a heavy cold probably contributed to any lack of clarity.

So it wasn't great getting this kind of feedback - duh! - but knowing what I wanted prevented me from being discouraged: on that stage on that day, however it came across to anyone else, I was myself.

Thankfully, usefully, I did also get some positive feedback from attendees at the conference and the content of my talk was validated by winning the Best Paper prize. But even without those things I think I'd have been able to take significant positives in spite of the audience reviews.

Back at work, I quickly had an opportunity to exorcise a demon by doing another leaving presentation. I treated it as I would a Toastmasters talk and wrote a draft in full, which I then repeated until I'd smoothed it out sufficiently. And then in the kitchen I wasn't rubber-legging and I wasn't heart-pounding and I wasn't knee-knocking, and tapped the glass and I spoke without notes and I got a laugh and I ad-libbed. And, sure, I stumbled a bit, but I was still there and doing it and doing it well. Or, at least, well enough.

I've been thinking about public speaking and me.

I wouldn't want to claim anything too grand. I haven't cracked the art of presenting. I still get nerves. I am not suggesting that you must do the same things as I did. I am not claiming that I haven't had some setbacks, and I don't have a magic wand to wave. But if I tried to summarise what I've done, I guess I'd say something like this:
  • I decided I wanted to change.
  • I found out what I wanted to change to.
  • I was open to ways to help me get there.
  • I looked for, or made, openings.
  • I reflected on what I was doing.
  • I stuck at it.

And I made my change happen.

Images: Black Country T-ShirtsCambridge City Communicators
Categories: Blogs

Without Which ...

Wed, 01/11/2017 - 08:40

This week's Cambridge Tester meetup was a show-and-tell with a theme:
Is there a thing that you can't do without when testing? A tool, a practice, a habit, a method that just works for you and you wouldn't want to miss it? Here's a list, with a little commentary, of some of the things that were suggested:
  • Testability: mostly, in this discussion, it was tools for probing and assessing a product.
  • Interaction with developers: but there's usually a workaround if they're not available ..
  • Workarounds
  • The internet: because we use it all the time for quick answers to quick questions (but wonder about the impact this is having on us).
  • Caffeine: some people can't do anything without it.
  • Adaptability: although this is like making your first wish be infinite wishes.
  • People: Two of us suggested this. I wrote my notes up in Testing Show
  • Emacs
  • Money: for paying for staff, tools, services etc.
  • Visual modelling: as presented, this was mostly about system architecture, but could include e.g. mind maps.
  • Notebook and pen: writing gives clarity
  • Phone: for playing games as a break from work.
  • Explainability: "it's my job to eradicate inexplicability."
  • Freedom/free will: within the scope of the mission
  • Problems: because we'll be out of a job without them.

Categories: Blogs

Testing Show

Wed, 01/11/2017 - 08:22

This week's Cambridge Tester meetup was a show-and-tell with a theme:
Is there a thing that you can't do without when testing? A tool, a practice, a habit, a method that just works for you and you wouldn't want to miss it? Thinking about what I might present I remembered that Jerry Weinberg, in Perfect Software, says "The number one testing tool is not the computer, but the human brain — the brain in conjunction with eyes, ears, and other sense organs. No amount of computing power can compensate for brainless testing..."

And he's got a point. I mean, I'd find it hard to argue that any other tool would be useful without a brain to guide its operation, to understand the results it generates, and to interpret them in context.

In show-and-tell terms, the brain scores highly on "tell" and not so much on "show", at least without a trepanning drill. But, in any case, I was prepared to take it as a prerequisite for testing so I thought on, assuming I could count on my brain being there, and came up with this:
The thing I can't do without when testing is people. Why? Well, first and foremost, software is commissioned by people, and built by people, and functions to service the needs of people. Without those people there wouldn't be software for me to test. As a software tester I need software and software needs people. And so, by a transitive relationship, I need people.

Which is a nice line, but a bit trite. So I thought some more.

What do people give me when I'm testing? Depending on their position with respect to the software under test they might provide
  • background data such as requirements, scope, expectations, desires, motivations, cost-benefit analyses, ...
  • test ideas and feedback on my own test ideas
  • insight, inspiration, and innovation
  • reasons to test or not to test some aspects of the system
  • another perspective, or perspectives 
  • knowledge of the mistakes they've made in the past, so perhaps I need not make them   
  • coaching
  • the chance to improve my coaching
  • satisfy a basic human need for company and interaction
  • ...

There are methodologies and practices that recognise the value of people to other people. For example, XP, swarming, mobbing, pairing, 3 Amigos, code reviews, peer reviews, brainstorming, ... and then there are those approaches that provide proxies for other people such as personas, thinking hats, role playing, ...

Interactions with others needn't be direct: requirements, user stories, books, blogs, tweets, podcasts, videos, magazines, online forums, and newsletters, say, are all interactions. And they can be more or less formal, and facilitated,  like Slack channels, conferences, workshops, and even meetups. They're generally organised by people, and the content created by people for other people, and the currency they deal in is information. And it's information which is grist to the testing mill.

And that's an interesting point because, although I do pair test sometimes, for the majority of my hands-on testing I have tended to work alone. Despite this, the involvement of other people in that testing is significant, through the information they contribute.

Famously, to Weinberg and Bolton, people are crucial in both a definition of quality and indeed a significant proportion of everything else too.
  •  Quality is value to some person.
  •   X is X to some person at some time.

Fair enough, you might ask with a twinkle in your eye, but didn't Sartre say "Hell is other people"?

Yes he did, I might reply, and I've worked with enough other people now to know that there's more than a grain of truth in that. (Twinkling back atcha!) But in our world, for our needs, I think it's better to think of it this way: software is other people.

Edit: I've listed some of the other things that were suggested at the meetup in Without Which.
Categories: Blogs

State of Play

Wed, 01/04/2017 - 08:23
The State of Testing Survey for 2017 is now open. This will be the fourth iteration of the survey and last year's report says that there were over 1000 respondents worldwide, the most so far.

I think that the organisers should be applauded for the efforts they're putting into the survey. And, as I've said before, I think the value from it is likely to be in the trends rather than the particular data points, so they're playing a long game with dedication.

To this end, the 2016 report shows direct comparisons to 2015 in places and has statements like this in others:
We are starting to see a trend where testing teams are getting smaller year after year in comparison with the results from the previous surveys.I'd like to see this kind of analysis presented alongside the time-series data from previous years and perhaps comparisons to other relevant industries where data is available. Is this a trend in testing or a trend in software development, for instance?

I'd also like to see some thought going into how comparable the year-to-year data really is. For example: is the set of participants sufficiently similar (in statistically important respects) that direct comparisons are possible? Or do some adjustments need to be made to account for, say, a larger number of respondents from some part of the world or from some particular sector than in previous years. Essentially: are changes in the data really reflecting a trend in our industry, or perhaps a change in the set of respondents, or both, or something else?

While I'm wearing my wishing hat I'd be interested in questions which ask about the value of the changes that are being observed. For example, are smaller teams resulting in better outcomes? What kind of outcomes? For who? I wonder whether customers or consumers of testing could be polled too, to give another perspective, with a different set of biases.
Categories: Blogs

What We Found Not Looking for Bugs

Sat, 12/31/2016 - 22:05
This post is a conversation and a collaboration between Anders Dinsen and me. Aside from a little commentary at the top and edits to remove repetition and side topics, to add links, and to clarify, the content is as it came out in the moment, over the course of a couple of days.

A question I asked about not looking for bugs at Lean Coffee in Cambridge last month initiated a fun discussion. The discussion suggested it’d be worth posing the question again in a tweet. The tweet in turn prompted a dialogue.

Some of the dialogue happened on public Twitter, some via DM, and on Skype, and yet more in a Google doc, at first with staggered participation and then in a tight synchronous loop where we were simultaneously editing different parts of the same document, asking questions and answering them in a continuous flow. It was at once exhilarating, educational and energising.

The dialogue exposes some different perspectives on testing and we decided to put it together in a way that shows how it could have taken place between two respectful, but different, testers.

James: Testing can’t find all the bugs, so which ones shouldn’t we look for? How?

Anders: My brain just blew up. If we know which bugs not to look for, why test?

James: Do you think the question implies bugs are known? Could they be expected? Suspected?

Anders: No, but you appear to know some bugs not to find.

James: I don't think I'm making any claims about what I know, am I?

Anders: Au contraire, "which bugs" seems quite specific, doesn't it?

James: By asking "which" I don't believe I am claiming any knowledge of possible answers.

Anders: I think this is a valid point.

Testing takes place in time, and there is a before and an after. Before, things are fundamentally uncertain, so if we know bugs specifically to look for, uncertainty is an illusion.

That testing takes place in time is obvious, but still easily forgotten like most other things that relates to time.

In our minds, time does not seem as real as it is. Remember, that we can just as vividly imagine the future and remember the past as we can experience the current. In our thoughts, we jump back and forth between imagination, the current and memory of the past, often without even realizing that we are in fact jumping.

When I test, I hope an outcome of testing will be test results which will give me certainty so that I can communicate clearly to decision makers and help them achieve certainty about things they need to be certain about to take decisions. This happens in time.

So before testing, there is uncertainty. After testing, some kind of certainty exists in someone (e.g. me, the tester) about the thing I am testing.

Considering that, testing is simple, but it follows that, expecting and even suspecting bugs implies some certainty, which will mislead our testing away from the uncertain.

James: I find it problematic to agree that testing is simple here - and I’ve had that conversation with many people now. Perhaps part of it is that "testing" is ambiguous in at least two interesting senses, or at least at two different resolutions:
  • the specific actions of the tester
  • a black box into which stakeholders put requirements and from which they receive reports

These are micro and macro views. In The Shape of Actions, Harry Collins talks about how tasks are ripe for automation when the actors have become indifferent to the details of them. I wrote on this in Auto Did Act, noting that the perspective of the actor is significant.

I would want to ask this: from whose perspective is testing simple? Maybe the stakeholder can view testing as simple, because they are indifferent to the details: it could be off-shore workers, monkeys, robots, or whatever doing the work so long as it is "tested".

I am also a little uncomfortable with the idea of certainty as you expressed it. Are we talking about certainty in the behaviour of the product under test, or some factor(s) of the testing that has been done, or something else?

I think I would be prepared to go this far:
  • Some testing, t, has been performed
  • Before t there was an information state i
  • After t there is an information state j
  • It is never the case that i is equal to j (or, perhaps, if i is equal to j then t was not testing)
  • It is not the case that only t can provide a change from i to j. For example, other simultaneous work on the system under test may contribute to a shared information state.
  • The aim of testing is that j is a better state than i for the relevant people to use as the basis for decision making

Anders: But certainty is important, as it links to someone, a stakeholder, a human. Certainty connotes a state of knowledge in something that has a soul, not just a mathematical or mechanical entity.

This leads me to say that we cannot have human testing without judgement.
Aside: It’s funny that the word checking, which we usually associate with automatic testing, might actually better describe at least part of human testing, as the roots of ‘check’ are the same as the game of chess, the Persian word for king. The check is therefore the king’s judgement, a verdict of truth, gamified in chess, but in the real world always something that requires judgement. But that was a stray thought ... What’s important here is that some way or another testing is not only about information.

I accept that as testers, we produce information, even streams of tacit and explicit knowledge testing and some of that can be mechanistically or algorithmically produced, but if we are to use it as humans and not only leave it to the machines to process, we must not only accept what we observe in our testing, we must judge it. At the end of the day (or the test) at least we must judge whether to keep what we have observed to ourselves, or if we should report it.

James: I did not define what I mean by an information state. If you pushed me to say something formal, I might propose it’s something like a set of assertions about the state of the world that is relevant to the system under test, with associated confidence scores. I might argue that much of it is tacitly understood by the participants in testing and the consumption of test results. I might argue that there is the potential for different participants to have different views of it - it is a model, after all. I might argue that it is part of the dialogue between the participants to get a mutual understanding of the parts of j that are important to any decisions.

This last sentence is critical. While there will (hopefully) be some shared understanding between the actors involved, there will also be areas that are not shared. Those producing the information for the decision-maker may not share everything that they could. But even if they were operating in such a way as to attempt to share everything that was relevant to the decision, their judgement is involved and so they could miss something that later turns out to be important.
Aside: I wonder whether it is also interesting to consider that they could over-share and so cloud the decision with irrelevant data. It is a real practical problem but I don’t know whether it helps here. If it does, then the way in which information is presented is also likely to be a factor. Similarly, the decision-maker may have access to information from other sources. These may be contemporary or historical, from within the problem domain or not, ...

So, really, I think that having two information states - pre and post t - is an oversimplification. In reality, each actor will have information states taking input from a variety of sources, changing asynchronously. The states i and j should be considered (in a hand-wavy way) the shared states. But we must remember that useful information can reside elsewhere.

Anders: I feel this is too much PRINCE2, where people on the shop floor attach tuples of likelihood and consequence-scores to enumerated risks, but essentially thereby hiding important information needed to make good, open-eyed decisions about risks.

James: Perhaps. I have been coy about exactly what this would look like because I don't have a very well-formed or well-informed story. In Your Testing is a Joke, I reference Daniel Dennett who proposes that our mental models are somewhat like the information state I've described. But I don't think it's possible or desirable to attempt to do this in practice for all pieces of information, if it were even possible to enumerate all pieces of information

Anders:I have witnessed such systems in operation and had to live with consequences of them. I have probably developed a very sceptical attitude due to that.

But we should not forget that testing is a human activity in a context and it is my human capacity to judge what I observe in testing and convey messages about it to stakeholders.

James: I’m still not comfortable with the term "certainty".

I might speculate that certainty as you are trying to use it could be a function of the person and the information states I’m proposing. Maybe humans have some shared feeling about what this function is, but it can differ by person. So perhaps a dimension of the humanity in this kind of story is in the way we "code" the function that produces certainty from any given information state.

The data in the information state can be produced by any actor, including a machine, but the interpretation of that information to provide confidence (a term I'm more comfortable with, but see e.g. this discussion) is of course a human activity. (But recent advances in AI suggest that perhaps it won’t necessarily always be so, for at least some classes of problem.)

Anders: Can I please ask you to join "team human", i.e. that all relevant actors (except the tools we use and the item under test) are humans with human capabilities, i.e. real thoughts and perhaps most importantly gut feelings?

Can you accept that fundamentally, a test result produced by a human is not produced by mechanistically, but human interpretation of what the human senses (e.g. sees), experience, imagination, and ultimately judgement?

James: Think of statistics. There are numerous tools that take information and turn it into summaries of the information. Some of them are named to suggest that they give confidence. (Confidence intervals, for example, or significance.) Those tools are things that humans can drive without thought (so essentially machines.)

Anders: I fundamentally cannot follow you there. Nassim Taleb is probably the most notable critic of statistics interpreted as something that can give confidence. His point (and mine) is that confidence as a mathematical term should not be confused with real confidence, that which a person has.

James: I think we are agreeing. Although the terms are named in that way, and may be viewed in that way by some - particularly those with a distant perspective - the results of running those statistical methods on data must inherently be interpreted by a human in context to be meaningful, valuable.

Anders: Ethically, decisions should be taken on the basis of an understanding of information. Defining "understanding" is difficult though, but there must be some sort of judgement involved, and then I’m back at square one: I use all my knowledge, experience and connect to my values, but by the end of the day, what I do is in the hands of my gut feeling.

James: Perhaps another angle is that data can get into (my notion of) an information state from any source. This can include gut, experiment, hearsay, lies. I want each of the items of data to have some level of confidence attached to them (in some hand-wavy way, again).

The humanistic aspect that you desire can be modelled here. It’s just not the only or even necessarily the most important factor, until the last step where judgement is called for.

Anders: This leads me to think about kairos: That there is a moment in which testing takes place, the point in time where the future turns to become the past. Imagine your computer clock shows 10.24 am and you know you have found a bug. When is the right time to tell it to the devs? They are in a meeting now about future features. Let’s tell them after lunch.

Kairos for communicating the bug seems to be "after lunch".

But it is not just about communication, there could even be a supreme moment for performing a test. It could be one that I have just had the idea for, one I have sketched out yesterday in a mind map, noted on a post-it, or prepared in a script months ago.

Kairos in testing could be the moment when our minds are open to the knowledge stream of testing so we can let it help us reach certainty about the test performed.

James: I am interested in the extent to which you can prepare the ground for kairos. What can someone do to make kairos more likely? As a tester, I want to find the important issues. Kairos would be a point at which I could execute the right action to identify an important issue. How to get to that moment, with the capacity to perform that action?

Anders: There is, to me, no doubt that kairos is a "thing" in testing in the human-to-human relating parts of what we do: communication, particularly; but also in leadership. A sense of kairos involves having an intuition of what is opportune to communicate in a given moment, and when is an opportune moment to communicate what seems important to you, but of course it could also be about having a sense of some testing to carry out at a particular moment to cause a good effect on the project.

Whether kairos is a thing in what is happening only between the tester and the object being tested (and possibly other machines), I would doubt, or if it was, we would certainly reach far beyond of the original meanings of kairos.

James: I think this is tied to your desire for a dialogue to be only between two souls, as we discussed on Skype. We agreed then that it is possible for one person to have an internal dialogue, and so two souls need not be necessary in at least that circumstance. I’d argue it's also not necessary in general. (Or we have to agree some different definition of dialogue.)

Anders: I do appreciate that some testers have a "technical 6th sense", e.g. when people experience themselves as "bug magnets". I think, however, that that comes from creative talents, imagination, technical understanding, and understanding of the types of mistakes programmers make, more than about human relations or "relations" to machines. I think it would then be better to talk about "opportune conditions", which, I think, would then probably be the same as "good heuristics".

James: From Wikipedia: In rhetoric, kairos is "a passing instant when an opening appears which must be driven through with force if success is to be achieved."

Whether at a time or under given conditions (and I'm not sure the distinction helps), it seems that kairos requires the speaker and listener (to give the roles anthropomorphic names for a moment) to both be in particular states:
  • the speaker must be in a position to capitalise on whatever opportunity is there, but also to recognise that it is there to be acted upon.
  • the listener must (appear to the speaker to) be in a state that is compatible with whatever the speaker wants to do.

Whether or not the opportunity is acted upon, I think these are true. Notice that they include both time and conditions. Time can exist (forgetting metaphysical concerns) without conditions being true, but the conditions must necessarily exist in a time. So I argue that if you want to tie to conditions you are necessarily tying to time also. If I follow your reasoning, then I think this means you might be open to kairos existing in human-machine interactions?

A difference that is apparent at several points in our dialogue here, I think, is that I want to make (software) testing be about more than interaction of a human with a product. I want it to include human-human interactions around the product. (See e.g. Testing All the Way Down and The Anatomy of a Definition of Testing.)

It’s my intuition that many useful techniques of testing cross over between interactions with humans and those with machines. And so I am interested in seeing what happens when you try to capture them in the same model of testing. And in the course of our discussion I’ve realised that I’ve been thinking along these lines for a while - see Going Postel or even Special Offers, for example.

I think that you want to separate these two worlds more distinctly than I do, and reserve more concepts, approaches and so on for humans only. But I think we have a shared desire to recognise the humanity at the heart of testing and to expect that human judgement is important to contextualise the outcomes of testing.

Anders: Yes you are right, I want to separate the two worlds, and I realise now that the reason is that I hope testers will more actively recognise humanity and especially what it means being human. Too often, testers try to model humanity using terminology and understandings which are fundamentally tied to the technical domain.

This leads to a lot of (hopefully only unconscious) reductionism in the testing world. It’s probably caused by leading thinkers in testing having very technical, not humanistic backgrounds.

So I am passionate that we do not confuse the technical moment in time in which I hit the key on my keyboard to start automatic test suite thereby altering the states of the system under test and the testing tools used, but not yet influencing any humans with the kairos of testing which is only tied to the human relations we have, including those we have with ourselves, and not to any machines.

Kairos happens when we let it happen.

Kairos is when we look down on the computer screen, sense what is on the screen, allow it to enter our minds, and start figuring out what happened and what that might mean.

Categories: Blogs

On Being Capable

Tue, 12/20/2016 - 07:58

When Karo asked whether it'd be OK if she nominated me along with Chris George and Neil Younger as meetup heroes for a UKSTAR competition I said I was sure we'd all be flattered.
Know any Software Meetup Heroes? I nominated @chrisg0911 @norry_twitting @qahiccupps - with a heartfelt thanks!— karo. stoltzenburg (@karostol) December 14, 2016
I guess I didn't really expect it to go anywhere and I certainly didn't expect that I'd feel somewhat embarrassed if it did.

But it has.

And so there you go, I learned something about myself. Again.

I've read the short-listed nominations and Emma, Oana, Alexandru, Leigh, Tony, and Hugh all look like great candidates doing great work for their local testing communities. I'd love you to go and read about them and vote for a hero right now.

Except that as I write this, it looks like, with delightful irony, that might not be possible ...
@qahiccupps there a bug in voting format?Ppl i know couldn't vote even once— Hugh McCamphill (@hughleo01) December 19, 2016 Which I suppose means that you have a little spare time in which I can urge you to get involved if you aren't already!

If you're reading this and you're from Cambridge, join the local tester meetup group run by Karo and Chris and come along to the morning Lean Coffee or the evening sessions. You can read my notes from recent meetings to get a flavour of what they're about.

If you attend just one of the Cambridge meetups in the 6 months before we announce a CEWT - Cambridge Exploratory Workshop on Testing - you'll automatically get an invite to that too.

If you're not from these parts I encourage you to find your own local meetup, with its own heroes, and get in there. You could do a lot worse than starting at the Ministry of Testing meetup group.

And if there's nothing happening locally for you why not set something up? Here's some peer workshop wisdom that proved enormously helpful to me when I was getting CEWT off the ground.
Categories: Blogs

One Way to Test

Tue, 12/20/2016 - 07:06

I came across this quote in Managing the Unmanageable, attributed to Doug Linder:
A good programmer is someone who looks both ways before crossing a one-way street.It made me chuckle - churlishly, childishly - as I imagined a developer crossing testing off their list because each time they'd happened to cross the street they'd implemented they'd checked it was working. Well, perhaps that some aspect of it wasn't not working, at that time, for that person, etc etc.

Reflecting as I write this, I wonder if I'd been having a bad day...

Anyway, I offered the quote to the Test team at Linguamatics yesterday, along with mince pies, and posed a different question as part of our annual festive Testing Can be Fun session (see also The So in Absolute, Last Orders, Further Reading, Testing is Like Making Love):
What might a “good” tester say or do, when encountering a one-way street?Ten minutes allowed, and as many mince pies as you can eat. Stick your answers in the comments if you like.
Categories: Blogs

Cambridge Lean Coffee

Thu, 12/15/2016 - 08:02

This month's Lean Coffee was hosted by Cambridge Consultants. Here's some brief, aggregated comments and questions  on topics covered by the group I was in.

How to get work in testing having been a developer for 25 years?
  • The questioner is an experienced developer/consultant who consistently sees "poor quality" development.
  • You don't need a formal background; it's possible to learn testing on the job.
  • The job market seems to be about 'technical testers' these days, so a developer could be suited to it.
  • Are you applying for roles and being rejected. (Not yet; this is a recent idea.)
  • What do you mean by testing? ("Separation of concerns, loose coupling, SOLID, good requirements. Unit testing is just there for the taking ... you just do it.")
  • They sound like full life-cycle or architectural ideas that might enable testing or reduce the need for it? ("Yes.")
  • Think about what motivates the person you're pitching to. What do they care about? Ask what they're worried about, the risks they perceive.
  • Testing is a stigma for some people.
  • Perhaps don't try to sell testing, so much as the value that testing can bring.
  • Testing for its own sake is tedious.
  • What is the context that you're trying to sell testing into?
  • In some cases, testing might be the wrong thing to suggest. For example a startup might need to move fast to get to market.
  • Remember that it doesn't matter how valuable testing is to you, the key is how valuable it is to them.

Test Managers must have been testers.
  • Are we talking about technical management or line management? (The questioner was more interested in line management.)
  • Other things being equal, I'd rather have a good people manager than a tester as a manager.
  • Testers will benefit from access to someone with technical knowledge, if not their manager.
  • A good manager can give the value proposition from the company perspective. Someone focused on testing might not do that so well.
  • A good line manager understands your needs and helps you through challenges in all areas (not just your discipline).
  • A non-testing manager can offer a useful alternative perspective, force you to speak in plain language.
  • A non-testing manager might not understand the value that you've given on projects (and does salary review, appraisal etc) but a good manager will ask relevant people for that feedback.
  • What's the best thing a manager has done/does for you?
  • ... (non-tester) pushed me to develop myself; in particular he saw that I could benefit from public speaking experience.
  • ... (non-tester) trusts me to get on with stuff - but asks me hard questions
  • ... (tester) supported me; gave me time to learn
  • ... (tester) defended me from company crap and allowed me to do good work that needed doing
  • Can we differentiate people who see value in testing and in testers?
  • Line management is about people not activities.

How detailed should exploratory testing be?
  • The questioner has been accused of going "too deep" when testing, after finding bugs outside the mission scope.
  • ET is about learning the product; about iterating, debriefing and focusing.
  • Look at Explore It!
  • Sometimes the mission is "I just want you to check the feature".
  • Sometimes people don't want to hear about bugs because they might e.g. stop the product shipping.
  • Sometimes people assume that "I found a bug" means "you must fix the bug I found".
  • Are there other things that you could have done that would have been more valuable?
  • What did your accusers expect from you?
Edit: Katrina Clokie followed up on this question in The Testing Pendulum: Finding balance in exploration.
We can't find all the bugs, so which ones shouldn't we look for? How?
  • Think about the cost to the organisation if an issue comes to light. What do the stakeholders care about?
  • Quality is in the eye of the stakeholder.
  • Don't look for the bugs that the customer is likely to find.
  • You shouldn't look for the cases that aren't important.
  • Is that very practical advice? How do you know?
  • Yes, it is practical advice, it can force you to think about or find out which are the important cases.
  • ... for example, performance is not important, so we won't look for bugs there.
  • ... which isn't to say we won't find them in passing, of course.
  • But testing is a way to uncover the things that are important.
  • ... ideally it will be a continual dialogue with stakeholders which focuses the investigation.
  • If you're not going to do anything with the information, then don't look for it. There's no value in reporting if no action will result.
  • But sometimes the aggregation of bugs in an area is itself significant, e.g. one typo on a page vs 300 typos on every page.
  • That's an interesting negative ("shouldn't") because normally we focus on the things we are doing or should do.
  • Isn't the premise here questionable? Do testers really generally go out looking for specific bugs?
  • Perhaps testers will be focusing more on the areas of potential risk and ways in which those risks might be exposed?
  • But you might know of, say, a repeated anti-pattern within the development team that you would look for explicitly.
Edit: Me and Anders Dinsen followed up this question in What We Found Not Looking for Bugs.Image:
Categories: Blogs

Well Read

Fri, 12/09/2016 - 00:08

This week, Maaret Pyhäjärvi published How to write 180 blog posts in a year.  Maaret's blog is one that I make a point of reading whenever Feedly tells me there's a new post there. Why? Because her posts are thoughtful, often deeply thoughtful. Here's a couple of paragraphs from Thinking you're the best:
For years, I prepared in the previous night for every relevant meeting. I went in with a ready-made plan, usually three to prep my responses for whatever might emerge in the meetings. Back in school, my Swedish teacher made me translate things out loud every class, because of my "word-perfect translations". Truth is I had them pre-translated with great effort because I was mortified with the idea of having to do  that work on the fly. Through my own experiences, I've grown to learn that the pre-prep was always my safety blanket. I did not want to look bad. I did not want to be revealed. I was the person who would rather use 3 days on a half-an-hour task. And I would say it was for my "learning". It was for my "personality". But truth is, it was for my fear of not being perfect.My feed aggregates over 200 testing blogs and a bunch from other areas. I skim the titles regularly. I read from the list most days. I've got much benefit by reading from a wide range of sources across a long period of time. But these blogs, for a variety of reasons, I'll read every time they show up:
Categories: Blogs

Mum's the Word

Tue, 11/29/2016 - 08:18

A few weeks ago I put out an appeal for resources for testers who are pulled into live support situations:
Looking for blogs, books, videos or other advice for testers pulled into real-time customer support, e.g. helping diagnose issues #testing— James Thomas (@qahiccupps) October 28, 2016 One suggestion I received was The Mom Test by Rob Fitzpatrick, a book intended to help entrepreneurs or sales folk to efficiently validate ideas by engagement with an appropriate target market segment. And perhaps that doesn't sound directly relevant to testers?

But it's front-loaded with advice for framing information-gathering questions in a way which attempts not to bias the the answers ("This book is specifically about how to properly talk to customers and learn from them"). And that might be, right?

The conceit of the name, I'm pleased to say, is not that mums are stupid and have to be talked down to. Rather, the insight is that "Your mom will lie to you the most (just ‘cuz she loves you)" but, in fact, if you frame your questions the wrong way, pretty much anyone will lie to you and the result of your conversation will be non-data, non-committal, and non-actionable. So, if you can find ways to ask your mum questions that she finds it easy to be truthful about, the same techniques should work with others.

The content is readable, and seems reasonable, and feels like real life informed it. The advice is - hurrah! - not in the form of some arbitrary number of magic steps to enlightenment, but examples, summarised as rules of thumb. Here's a few of the latter that I found relevant to customer support engagements, with a bit of commentary:
  • Opinions are worthless ... go for data instead
  • You're shooting blind until you understand their goals ... or their idea of what the problem is
  • Watching someone do a task will show you where the problems and inefficiencies really are, not where the customer thinks they are ... again, understand the real problem, gather real data
  • People want to help you. Give them an excuse to do so ... offer opportunities for the customer to talk; and then listen to them
  • The more you’re talking, the worse you’re doing ... again, listen

These are useful, general, heuristics for talking to anyone about a problem and can be applied with internal stakeholders at your leisure as well as with customers when the clock is ticking. (But simply remembering Weinberg's definition of a problem and the Relative Rule has served me well, too.)
Given the nature of the book, you'll need to pick out the advice that's relevant to you - hiding your ideas so as not to seem like you're needily asking for validation is less often useful to a tester, in my experience - but  as someone who hasn't been much involved in sales engagements I found the rest interesting background too.Image: Amazon
Categories: Blogs