Skip to content

Hiccupps - James Thomas
Syndicate content
James Thomas
Updated: 18 hours 10 sec ago

Giving 'Back

Thu, 09/22/2016 - 06:51

The Test team book club at Linguamatics is currently reading What Did You Say? The Art of Giving and Receiving Feedback. Here's a couple of quotes that I picked out for our last session:
  • If you’re really interested in helping people, you’ll do well to start your feedback by opening your own motives to inspection.
  • Even when it’s given at the receiver’s request, feedback describes the giver more than the receiver.
  • When the data and their model don’t match, most people discard the data.

I recall an instance when, engaged in discussion with a colleague I'll call Russell, about the data analysis he was presenting, I spotted an opportunity to offer feedback. It was about something that I knew Russell wanted to change. It was about something that I knew was open to me to give feedback on, because we had talked about it. It was about something that I thought would be beneficial for Russell in multiple ways and, I hoped, would provide some insight into a particular behaviour pattern that he had.

However, it was also the first time that I had seen this particular thing. A data set of size one. I had no evidence, yet, that it would lead to the end point that Russell desired to alter. A data set of size zero.

Against this: my instinct, my gut, and my experience. And a sense of goodwill, built up over time, over repeated interactions, over sometimes difficult sessions where I had tried to demonstrate that I do care to assist and support and advise because I want to help Russell to be the best he can be, in the respects that matter to him and for his work.

But I was still cautious. I have unwittingly burned and been burned enough times over the years to know that each of these conversations carries with it risks. Risks of misreading the context, risks of misreading the agreements, risks of misreading the mood, risks, risks, risks, ...

But I went ahead anyway. The potential benefit and the goodwill in the bank outweighed the risks, I calculated, on this occasion. And I gave my feedback. And Russell agreed with me. And I breathed a deep internal sigh of relief.

Comparing this anecdote to the quotes I pulled from the book:
  • My motives, I think, were good: I wanted to help Russell achieve a personal goal.
  • But the feedback does reflect something about me: an interest in reducing unnecessary complexity, an interest in making presentation clear, the ego that is required to believe that my colleagues will want to listen to any advice from me, ...
  • In this case, it turned out my suggestion didn't contradict Russell's model but exposed it, and in any case I had little concrete data to present.

I use this episode as an example not because it ended well, particularly, but because it's an illustration for me of how much I have been influenced by What Did You Say? in the couple of years since I first read it. I consciously I go about my day-to-day business, doing my best to be careful about when I choose to offer feedback, about when I deliberately choose not to, and about picking up and picking up on any feedback that's coming my way in return.

I try to treat this as a testing task where I can, in the sense that I try hard to observe my own actions and the responses they generate, and I think about ways in which they might be related and how I might approach things differently in the next exchange, or at another time, with this person, or someone else.

Easier said than done, of course, so I'll finish with another quote from the book, another quote that I've taken to heart and act on, that regularly helps guide me with pretty much everything that I've said above:
Don’t concentrate on giving feedback; concentrate on being congruent–responding to the other person, to yourself, and to the here-and-now situation. Don’t go around hunting for opportunities to give feedback, because feedback is effective only when the need arises naturally out of congruent interactions.Some details have been changed.
Image: Leanpub

Categories: Blogs

Reporting, a Novel Approach

Tue, 09/13/2016 - 07:41
 There's a girl in the park playing with an enormous bunch of balloons. She's running around, clearly very happy to have such a pretty and fun toy. She seems entranced by the way the balloons have life of their own: they hold themselves up, needing no support from her, and animatedly jostle one another as she moves. Her grip on the strings, twined together in her fist, is quite loose, and she's in danger of losing them if she's not careful. And, of course, she isn't and she does. The balloons float up and up and up from her released grasp, past a tall tree in which two nude men are arm wrestling. On their wrists each sports a watch showing ten minutes past ten, despite the time being 12:57. With their free arms they reach out and catch the balloons as they bobble by, bursting every last one of them, and smiling.In the second exercise of Mira Nair's Storytelling workshop, which ran at last night's Cambridge Tester Meetup, we were asked to write a story that included three items chosen at random from a selection. I had balloons, entwined arms with hands clasped, and a clock showing 10:10. Prior to that we'd been provided with a story structure and asked to fill it with content. Later, to take existing stories and compress them to tweet length.

The workshop's practical aspects focused mostly on structure, including on the STAR mnemonic which is intended to help interviewees give good answers to behaviour questions such as "Give me an example of when you ...". The letters stand for Situation, Task, Action, and Result and a story according to them should run like this:

  • Situation: define the background
  • Task: explain the mission
  • Action: describe the work done
  • Result: enumerate the outcomes 

The first exercise in the workshop gave us that skeleton and asked us to fit a recent episode to it. Some found it liberating ("The structure draws the story out of whatever I put in") while others struggled ("I couldn't find something that I thought would make a good story"). At work, Mira suggests, we'll more often have the problem of something to present and needing a structure to bring the best out of it than the reverse.

This was reinforced by the second exercise which provided content but, interestingly, didn't require any structure of us. More people found this more straightforward, the content anchoring everything else.

In the story I gave at the beginning I experimented with another narrative device Mira talked about, the False Start, in which an apparently predictable beginning leads to an unexpected ending and can result (with judicious use) in increased audience engagement.

The final exercise cut the content problem a different way: take an existing description (a couple of software bugs were provided) and summarise it in 140 characters or less. As testers, we author reports of potential issues regularly, and part of the skill of transmitting those reports is finding a way to quickly engage our audience, which will often mean extracting the essentials and conveying them efficiently.

Different approaches were taken here: I boiled the descriptions down as I might at work; others took the Twitter aspects and used hashtags as shorthand, effectively importing context cheaply, and others used humour to convey a sense of violated expectation.

One of the things I look for in these kinds of events is the questions they generate, the trains of thought I can follow at my leisure, the connections I can make, ... Here's a few:

  • what really distinguishes a story from any other prose, if that's possible? 
  • is the "storyness" in the eye of the author or the audience?
  • what techniques are there for picking out the relevant content for a story?
  • what aspects of storytelling are important beyond structure?
  • even with strong structure, stories can be poor, boring, uninformative.
  • readers assume much when reading a story, filling in missing details, assuming causation, intent and so on.
  • what about the inadvertent stories we tell all the time; that bored expression, that casual gesture, that throwaway remark?
  • stories don't need to be true, but my reports as a tester generally need to be true (to what I understand the situation I'm describing to be).
  • stories can be input to and output from testing.
  • don't forget the relative rule: the audience and the time are important to the effect a story will have.
  • there was discussion about tailoring stories for an audience ("stories should not be the same each time you tell them") but once written down, a story is static. 
  • I find that writing helps me to generate and understand the content. I'll often start writing before I know the story myself.
  • finding a perspective can help to make the story compelling, and that perspective can be the author's, the readers', or that of a third party.
  • I like the testing story heuristic I took from RST: status, how you tested, value/risks. But this is a content heuristic more than a delivery heuristic, although I find that order to generally be useful.

I have written before about the C's I look for in communication: conciseness, completeness, correctness, clarity, context. I realised in this workshop that I can add another: compelling.
Categories: Blogs

Tools: Take Your Pick Part 4

Tue, 08/30/2016 - 06:56

Back in Part 1 I started this series of posts one Sunday morning with a very small thought on tooling. (Thinking is a tool.) I let my mind wander over the topic and found that I had opinions, knowledge, ideas, and connections that I hadn't made explicit before, or perhaps only in part. (Mind-wandering is a tool.)

I wrote the stuff that I was thinking down. (Writing is a tool.) Actually, I typed the stuff that I was thinking up.1 I have recently been teaching myself to touch type in a more traditional style to (a) stop the twinges I was feeling in my right hand from over-extension for some key combinations and (b) become a faster, more consistent, typist so that my thoughts are less mediated in their transmission to a file. (Typing is a tool.)

I reviewed and organised my thoughts and notes. With each review, each categorisation, each classification, each arrangement, each period of reflection away from the piece of writing, I found more thoughts. (Reviewing and rationalisation and reflection are tools.) I challenged myself to explore ideas further, to tease out my intuitions, to try to understand my motivations, to dig deeper into whatever point it was I thought I was making. (Exploration is a tool.)

The boundaries between some of these tools are not clear some of the time. And that doesn't matter, some of the time. For me, in this work, it doesn't matter at all. My goal is to get my thoughts in order and hopefully learn something about myself, for me, and perhaps also something more general that I can share with my team, my colleagues, the wider readership of this blog.

That the boundaries are not clear is an interesting observation, I find, and it came about only because I was trying to list a set of tools used somewhat implicitly here. (Lists are tools.) Not knowing which tool we are using suggests that we can use tools without needing to know that we are using them at all. Part 2 started off with this:

   When all you have is a hammer, everything looks like a nail.

And I discussed how this does not necessarily mean that every use of the hammer is mindless. But I now also wonder whether it's possible to proceed without even realising that you have a hammer. With non-physical tools - such as those I've listed above - this seems to me a distinct risk. Side-effects of it might include that you don't know how you arrived at solutions and so can't easily generalise them, you don't realise that you are missing out large parts of the search space and so can't consider it; you don't provide yourself with the opportunity to look for improvement, ...

I think that reflection and introspection help me to mitigate those risks, to some extent. Although, of course, some of the risks themselves will be unknown unknowns and so less amenable to mitigation without additional effort being made to make them known. (Another problem to be solved. But which tool to use?)

The more I practise that kind of reflection the more practised I become at recognising simultaneously the problem and meta problems around it, or the problem space and the context of which is it a part, or the specific problem instance and the general problem class. I have had my fingers burned trying to verbalise these things to others, and I probably now over-compensate through clunky conversational tactics to try to make clear that I'm shifting modes of thinking.

Another thought on sub-conscious problem-solving approaches: perhaps the recognition (correct or not) of an instance of a nail-like problem triggers a particular hammer (tacit or explicit):
When it looks enough like a nail, I hit it.But every decision to use a tool is also an opportunity to make a different decision. Deciding to think about how and why a decision was made gives insight that can be useful next time a decision is made, including the realisation that a decision is being made.

Part 1 of this essay was in the domain of cleaning,  Part 2 more general and theoretical, and Part 3 focused back on work. They are grouped that way and written in that order because I found it helpful and convenient, but the way the thoughts came to me was messier, non-linear, fragmentary. I used tools to note down the thoughts: my phone, scraps of paper, my notebook, text files on the computer, ... Tools to preserve the output of tools to provide input to tools that will themselves generate output on the topic of tools.

I find myself chuckling to and at myself as I write this last part of the summary. While attempting to pull together the threads (attempting to pull together threads is a tool) I realise that in this piece which is ostensibly about hammers and nails - and having observed that perhaps we sometimes don't recognise the hammer - there may actually be no nail.

When I started out I had no particular problem to solve - beyond my own interest in exploring a thought - and so no particular need for a tool, although I have deployed several, deliberately and not. But. ironies aside, that's fine with me: quite apart from any benefits that may accrue (and there may be none, to me or you, y'know) the process itself is enjoyable, intellectually challenging, and satisfying for me. And exercising with my tools keeps me and them in some kind of working order too.

Footnote1.  Writing down but typing up? There's a thought for another day.
Categories: Blogs

Tools: Take Your Pick Part 3

Tue, 08/30/2016 - 05:47

In Part 1 of this series I observed my behaviour in identifying problems, choosing tools, and finding profitable ways to use them when cleaning my bathroom at home. The introspection broadened out in Part 2 to consider tool selection more generally. I speculated that, although we may see someone apparently thoughtlessly viewing every problem as a nail and hence treatable with the same hammer, that simple action can hide deeper conscious and unconscious thought processes. In Part 3 I find myself with these things in mind, reflecting on the tools I use in my day-to-day work.

One class of problems that I apply tools to involves a route to the solution being understood and a desire to get there quickly. I think of these as essentially productivity or efficiency problems and one of the tools I deploy to resolve them is a programming or scripting language.

Programming languages are are tools, for sure, but they are also tool factories. When I have some kind of task which is repetitive or tiresome, or which is substantially the same in a bunch of different cases, I'll look for an opportunity to write a script - or fabricate a tool - which does those things for me. For instance, I frequently clone repositories from different branches of our source code using Mercurial. I could type this every time:

$ hg clone -r branch_that_I_want

... and swear a lot when I forget that this is secure HTTP or mistype localrepo again. Or I could write a simple bash script, like this one, and call it hgclone:


hg clone -r $1$2

and then call it like this whenever I need to clone:

$ hgclone branch_that_I_want repo_that_I_want

Now I'm left dealing with the logic of my need but not the implementation details. This keeps me in flow (if you're a believer in that kind of thing) or just makes me less likely to make a mistake (you're certainly a believer in mistakes, right?) and, in the aggregrate, saves me significant time, effort and pain.

Your infrastructure will often provide hooks for what I sometimes think of as micro tools too. An example of this might be aliases and environment variables. In Linux, because that's what I use most often, I have set things up so that:
  • commands I like to run a particular way are aliased to always run that way.
  • some commands I run a lot are aliased to single characters.
  • some directory paths that I need to use frequently are stored as environment variables.
  • I can search forwards and backwards in my bash history to reuse commands easily.

One of the reasons that I find writing (and blogging, although I don't blog anything like as much as I write) such a productive activity is that the act of doing it - for me - provokes further thoughts and connections and questions. In this case, writing about micro tools I realise that I have another kind of helper, one that I could call a skeleton tool.

Those scripts that you return to again and again as starting points for some other piece of work, they're probably useful because of some specific piece of functionality within them. You hack out the rest and replace them in each usage, but keep that generally useful bit. That bit is the skeleton. I have one in particular that is so useful I've made a copy of it with only the bits that I was reusing to make it easier to hack.

Another class of problem I bump into is more open-ended. Often I'll have some idea of the kind of thing I'd like to be able to do because I'm chasing an issue. I may already have a tool but its shortcomings, or my shortcomings as a user, are getting in the way. I proceed here in a variety of ways, including:
  • analogy: sometimes I can think of a domain where I know of an answer, as I did with folders in Thunderbird.
  • background knowledge: I keep myself open for tool ideas even when I don't need tools for a task. 
  • asking colleagues: because often someone has been there before me.
  • research: that frustrated lament "if only I could ..." is a great starting point for a search. Choosing sufficient context to make the search useful is a skill. 
  • reading the manual: I know, old-fashioned, but still sometimes pays off.

On one project, getting the data I needed was possible but frustratingly tiresome. I  had tried to research solutions myself, had failed to get anything I was happy with, and so asked for help:
#Testers: what tools for monitoring raw HTTP? I'm using tcpdump/Wireshark and Fiddler. I got networks of servers, including proxies #testing— James Thomas (@qahiccupps) March 26, 2016 This lead to a couple of useful, practical findings: that Fiddler will read pcap files, and that chaosreader can provide raw HTTP in a form that can be grepped. I logged these findings in another tool - our company wiki - categorised so that others stand a chance of finding them later.

Re-reading this now, I notice that in that Twitter thread I am casting the problem in terms of the solution that I am pursuing:
I would like a way to dump all HTTP out of .pcap. Wireshark cuts it up into TCP streams. Later, I recast the problem (for myself) in a different way:
I would like something like tcpdump for HTTP.The former presupposes that I have used tcpdump to capture raw comms and now want to inspect the HTTP contained within it, because that was the kind of solution I was already using. The latter is agnostic about the method, but uses analogy to describe the shape of the solution I'm looking for. More recently still, I have refined this further:
I would like to be able to inspect raw HTTP in real time, and simultaneously dump it to a file, and possibly modify it on the fly, and not have to configure my application to use an external proxy (because that can change its behaviour).Having this need in mind means that when I happen across a tool like mitmproxy (as I did recently) I can associate it with the background problem I have. Looking into mitmproxy, I bumped into HTTPolice, which can be deployed alongside it and used to lint my product's HTTP.  Without the background thinking I might not have picked up on mitmproxy when it floated past me; without picking up on mitmproxy I would not have found HTTPolice or, at least, not found it so interesting at that time.

Changing to a new tool can give you possibilities that you didn't know were there before. Or expose a part of the space of possible solutions that you hadn't considered, or change your perspective so that you see the problem differently and a different class of solutions becomes available.

Sometimes the problem is that you know of multiple tools that you could start a task in, but you're unsure of the extent of the task, or the time that you'll need to spend on it, whether you'll need to work and rework or this is a one-shot effort and other meta problems of the problem itself. I wondered about this a while ago on Twitter:
With experience I become more interested in - where other constraints permit - setting up tooling to facilitate work before starting work.— James Thomas (@qahiccupps) December 5, 2015
And where that's not possible (e.g. JFDI) doing in a way that I hope will be conducive to later retrospective tooling.— James Thomas (@qahiccupps) December 5, 2015
And I mean "tooling" in a very generic sense. Not just programming.— James Thomas (@qahiccupps) December 5, 2015
And when I say "where other constraints permit" I include contextual factors, project expectations, mission, length etc not just budget— James Thomas (@qahiccupps) December 5, 2015
Gah. I should've started this at Perhaps tomorrow.— James Thomas (@qahiccupps) December 5, 2015
I wonder if this is irony.— James Thomas (@qahiccupps) December 5, 2015 A common scenario for me at a small scale is, when gathering data, whether I should start in text file, or Excel, or an Excel table. Within Excel, these days, I usually expect to switch to tables as soon as it becomes apparent I'm doing something more than inspecting data.

Most of my writing starts as plain text. Blog posts usually start in Notepad++ because I like the ease of editing in a real editor, because I save drafts to disk, because I work offline. (I'm writing this in Notepad++ now, offline because the internet connection where I am is flaky.) Evil Tester wrote about his workflow for blogging and his reasons for using offline editors too.

When writing in text files I also have heuristics about switching to a richer format. For instance, if I find that I'm using a set of multiply-indented bullets that are essentially representing two-dimensional data it's a sign that the data I am describing is richer than the format I'm using. I might switch to tabulated formatting in the document (if the data is small and likely to remain that way), I might switch to wiki table markup (if the document is destined for the wiki), or I might switch to a different tool altogether (either just for the data or for everything.)

At the command line I'll often start in shell, then move to bash script, then move to a more sophisticated scripting language.  If I think I might later add what I'm writing to a test suite I might make a different set of decisions to writing a one-off script. If I know I'm searching for repro steps I'll generally work in a shell script, recording various attempts as I go and commenting them out each time so that I can easily see what I did that lead to what. But if I think I'm going to be doing a lot of exploration in an area I have little idea about I might be more interactive but use script to log my attempts.

At a larger scale, I will try to think through workflows for data in the project: what will we collect, how will we want to analyse it, who will want to receive it, how will they want to use it? Data includes reports: who are we reporting to, how would they like to receive reports, who else might be interested? I have a set of defaults here: use existing tooling, use existing conventions, be open about everything.

Migration between tools is also interesting to me, not least because it's not always a conscious decision. I find I've begun to use Notepad++ more on Windows whereas for years I was an Emacs user on that platform. In part this is because my colleagues began to move that way and I wanted to be conversant in the same kinds of tools as them in order to share knowledge and experience. On the Linux command line I'll still use Emacs as my starting point, although I've begun to teach myself vi over the last two or three years. I don't want to become dependent on a tool to the point where I can't work in common, if spartan, environments. Using different tools for the same task has the added benefit of opening my mind to different possibilities and seeing how different concepts repeat across tools, and what doesn't, or what differs.

But some migrations take much longer, or never complete at all: I used to use find and grep together to identify files with certain characteristics and search them. Now I often use ack. But I'll continue to use find when I want to run a command on the results of the search, because I find its -exec option a more convenient tool than the standalone xargs.

Similarly I used to use grep and sed to search and filter JSON files. Now I often use jq when I need to filter cleanly, but I'll continue with grep as a kind of gross "landscaping" tool, because I find that the syntax is easier to remember even if the output is frequently dirtier.

On the other hand, there are sometimes tools that change the game instantly, In the past I used Emacs as a way to provide multiple command lines inside a single connection to a Linux server. (Aside: putty is the tool I use to connect to Linux servers from Windows.) When I discovered screen I immediately ditched the Emacs approach. Screen gives me something that Emacs could not: persistence across sessions. That single attribute is enough for me to swap tools. I didn't even know that kind of persistence was possible until I happened to be moaning about it to one of our Ops team. Why didn't I look for a solution to a problem that was causing me pain?

I don't know the answer to that.

I do know about Remote Desktop so I could have made an analogy and begun to look for the possibility of command line session persistence. I suspect that I just never considered it to be a possibility. I should know better. I am not omniscient. (No, really.) I don't have to imagine a solution in order to find one. I just have to know that I perceive a problem.

That's a lesson that, even now, I learn over and over. And here's another: even if there's not a full solution to my problem there may be partial solutions that are improvements on the situation I have.

In Part 4 I'll try to tie together the themes from this and the preceding two posts.
Syntax highlighting:
Categories: Blogs

Tools: Take Your Pick Part 2

Mon, 08/29/2016 - 21:45

In Part 1, I described my Sunday morning Cleaning the Bathroom problem and how I think about the tools I'm using, the way I use them, and why.  In particular I talked about using a credit card as a scraper for the grotty build up around the sides of the bath and sink. On the particular Sunday that kicked this chain of thoughts off, I noticed myself picking the card up and using a corner of it vertically, rather than its edge horizontally, to remove some guff that was collecting around the base of a tap.

This is something I've been doing regularly for a while now but, before I got the scraper, it was a job I used an old toothbrush for. In Part 1 I recounted a number of conscious decisions around the way I keep the littlest room spic and span, but switching to use the card on the tap wasn't one I could recall making.

Observing myself taking a tool I'd specifically obtained for one purpose and using it for another put me in mind of this old saw:
When all you have is a hammer, everything looks like a nail.Until I looked it up1 just now I hadn't heard this saying called The Law of the Instrument nor come across the slightly subtler formulation that Wikipedia attributes to Maslow:
I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.Given the first of those two variants, it's easy to imagine that the universal application of the hammer is a mindless option - and we've all probably seen instances of that - but I think that, generally, tools are used in places where they are inappropriate or sub-optimal for a variety of reasons, and temptations, as Maslow would have it.

There are three explicit variables in play in this space: the problem, the tool, and the person using the tool. Here's one way I explored it, by considering the possible scenarios involving the tool and choice of tool, and trying to understand how the person might have got there:

I recognise the shape of the problem, and I have a tool that I know fits it
  •  ... but I use my favourite tool instead because I'm more familiar with it.
  •  ... but I use something else because of politics in the office, my boss, my colleagues, ...
  •  ... but I use something novel because I want to own this problem and be the expert in it.
  •  ... but I use something else to prevent an increase in our already large tool set.
  •  ... but I won't use it because of ethical or moral issues I have with the tool vendor.
  •  ... but I won't use it because of previous bad experiences with the tool, or others similar to it in some way.
  •  ... but the context changed since I last looked, and I didn't notice, so I'll continue to use the existing tool.
  •  ...
I recognise the shape of the problem, but I don't have a tool that I know fits it
  • ... so I'll use the tool that I have invested heavily in anyway because sunk cost fallacy
  • ... so I'll use the tool I do have that looks closest.
  • ... so I'll use the tool that I have in my hand right now because there's no context-switching cost.
  • ... so I'll continue to use the tool I am using now, despite its flaws, because I believe there is some benefit.
  • ... so I'll use a tool I do have because there's no time/budget/desire to look for others.
  • ... so I'll use something I do have because I'm uncertain of my ability to choose a new tool well.
  • ... so I'll continue to use a tool I have because I'm worried about the cost of getting a new tool wrong.
  • ... so I'll use whatever is to hand because I don't really care about doing a good job.
  • ...
I don't recognise the shape of the problem
  • ... so I'll try the tools I've got and see where they get me,
  • ... or make a tool,
  • ... or use no tool,
  • ... or try break the problem down into pieces that can be attacked with tools I do know.
  • ...

The latter class can be particularly galling because it contains two sub-cases:
  •  I don't recognise the shape of the problem, and - even if I did - I don't have a tool that fits it.
  •  I don't recognise the shape of the problem, but - if I did - I would find that I have a tool that fits it.

And much wailing and gnashing of teeth have been caused by the hindsight searchlight focusing on the second of those. Your wailing and gnashing of teeth, right? And the same is true of these scenarios:

I don't, or am not prepared to,  recognise the existence of a problem
  • ... so I make no decisions about tool use at all
  • ... which means that I might stay as I am or unconsciously drift into some other behaviour.
  • ...
I recognise that there is no problem
  • ... but I have an agenda that I am pushing and so force tool use anyway.
  • ... but I just love to try new things so I'll go ahead and use a tool for its own sake.
  • ... but I'm putting off some other work so I'll do needless work here.
  • ... but I haven't got enough to do so I'll try this tool out to look busy.
  • ...

As I enumerate these cases, I begin to think that they apply not just to the person with just the hammer, but to all of us, every time we do or not use any tool for any task.

In using any tool at all you are making a decision - implicitly or explicitly. When you enter three commands into the shell to get something to run you are either accepting that you will not use a script to do it for you, and avoid the typos, being in the wrong directory and so on, or you are missing out on the knowledge that a script could help you, perhaps because you don't care to avoid that time being spent on typing and typos.

In choosing to use the same tool that you always use for editing a file you are missing out on the chance to learn that there is something better out there. But also avoiding paying the costs of learning that new thing. Do you do that consciously?

I started trying to map these kinds of observations back onto my own exploration of ways in which I could satisfy my bathroom mission. As I did this, I came to realise that I have mostly cast the problem recognition and tool choice as something that is done from a position of knowledge of the problem. But my own experience shows me that it's less clear-cut than that.

In this area, I love Weinberg's definition of a problem:
A problem is a difference between things as desired and things as perceived.I love it not least because it reminds me that there are multiple levers that can be pulled when confronted with a problem. If I am struggling with the shape of the problem I can change it, change my view of it, change my desires about what it should be. Opening out this way in turn reminds me that exploration of the space is an incredibly useful way to begin to understand which of those levers I can and/or should be pulling: perhaps if I can remove the things that look like nails, I can even put down my hammer.
Sometimes I find that I can learn more about the shape of the problem by using the tools I have and discovering their weaknesses. Sometimes I can only imagine a possible solution by attempting to resolve the problem the wrong way. If I do that tacitly, deliberately, then I'm here:
I recognise the shape of the problem, but I don't have a tool that I know fits it ... so I will explore the problem space with the tools I have, deliberately experimenting and hoping to learn more about the tools, the space, the problem, myself.And I might find that I'm then saying "aha, if only I had something which could ..." or "oh, so perhaps I don't really need ..."

But this means deliberately deciding to spend some of whatever budget is available for problem solving on the meta task of understanding the problem. Stated as baldly as this it seems obvious that someone with a problem might consider that, doesn't it? But how many times have you seen something else happen?

How many times have you seen only a tiny fraction of the capacity of some tool being exploited? For anything more complicated than a hammer, it's easy not to know that there are capabilities not being used. The Law of the Instrument can be applied within tools too. If I don't know that Word can do mail merge, I might find myself buying a new tool to do it, for example.

On the other hand, creative reuse can be a good way to get additional value from an existing tool. A hammer can be used for things other than hitting a nail - as a door stop, as a lever, to encourage some seized machinery to become separated, as a counterbalance, as a pendulum weight, as a goal post, and might be a sufficiently good substitute for the "proper" tool in any given situation, at any given time. And, in fact, imagining ways to reuse a tool can itself be a useful way to get creative juices flowing.

But contexts change - the problem might alter, views of it might alter, the available tools might alter. Being open to reconsidering decisions is important in getting a good outcome. Doing nothing, reconsidering nothing - that pretty much guarantees at best standing still or perhaps applying a solution to a problem that no longer exists or applying the wrong solution to the problem that does.

Tool use is inherent in software development and the kinds of choices I've described above are being made all the time for all sorts of reasons, including those that I've given. It was interesting to me, as I enumerated these thoughts, that although in my bathroom cleaning example I have no reason to be anything other than on the level - there are no bathroom politics in our house and the stakes are not high in any dimension - and despite doing my best to be as clear to myself about what I'm thinking and trying at any given time as I can, I still proceeded to make choices unconsciously, to not take account of useful evidence, and to continue with one line of exploration past the point at which its utility was exhausted.

In Part 3 I'll try and recast these thoughts in terms of some practical examples from the world of work rather than bathroom cleaning.

Footnote1. Given where I come from, and its traditional rivalry with Birmingham, I'm amused that the hammer that's applied to every problem is sometimes called a Birmingham Screwdriver.
Categories: Blogs

Tools: Take Your Pick Part 1

Mon, 08/29/2016 - 21:26

It's early on a Sunday morning and I'm thinking about tools. Yes, that's right: Sunday morning; early; tools.

Sunday morning is bathroom cleaning morning for me1 and, alongside all the scrubbing, squirting, and sluicing I spend time evaluating the way I clean against the results I achieve. My goal is to do a good job (in terms of the cleanliness and appearance of cleanliness of the bathroom) balanced against expense, time and effort.

I've got a set of tools to help with this task. Some are traditional and obvious, including sponge, J-cloth, glass cleaner, bathroom cleaner, toilet brush, Marigolds, ... Some are reasonably well known but not mainstream, including citric acid crystals, squeegee, old toothbrush, ... and some are more niche, including a credit card, flannel, and car windscreen coating, ...

By tool I mean something as basic as this, from Oxford Dictionaries:
 A thing used to help perform a jobAnd I'm being generous in my interpretation of job too - pretty much anything that it is desired to accomplish is a job for the purposes of this discussion.

Harry Collins, in The Shape of Actions, distinguishes tools from proxies and novelties based on the extent to which they extend our capacity to do the job or can stand in for us in doing the job itself. I find this stimulating but I'm not going to be concerned with it here. (If you're interested, Michael Corum makes an admirable attempt to summarise what is a challenging book in this blog post. My thoughts on it are less comprehensive: here and here.)

I don't think there's any action in my cleaning the bathroom that doesn't employ some kind of tool by the definition I've chosen, so any consideration of how well I'm achieving my goal will implicitly include some element of how well the tool, and my use of the tool, is contributing to it. Which often means that I'm wondering whether I have the right set of tools and am getting the best out of them. Which is where I was on Sunday morning; cleaning the shower screen.

A few years ago, when we had a shower curtain, I didn't need to clean it every week but could wait until the soap and limescale grot had built up and then put it through the washing machine. Although this clearly meant that there was known uncleanliness most weeks it was a decent enough compromise, especially since we bought shower curtains with frosted patterns on them which were less likely to be obviously dirty. (The shower curtain is a tool.)

When we replaced the curtain with a folding glass screen, I simply transferred the shower curtain cleaning cycle to it. And that didn't work well. I found that I was spending a long time every few weeks trying to get stacks of accumulated crud off the thing. Which was very hard work. And the screen, with its clear glass, didn't hide any aspect of the problem either. In fact, it appeared to be more susceptible to splashes leaving a mark than the curtain, and it looked horrible most of the time.

So I experimented with the tools I was using. I explored different cleaning products - amongst them vinegar, newspaper, lemon juice, various glass sprays, and citric acid - and a variety of cloth and sponge applicators. I tried varying my technique, I tried varying the frequency - I now clean it every week - and I introduced the squeegee to remove some of the dirtiness after every shower.

This made an improvement in terms of result - the shower screen looked great on Sundays and decent for most of the week - but I was still putting more effort than I'd really like into maintaining the screen's appearance. And so I started trying to reframe the problem. Could we stop caring about cleanliness? If we could, the problem would simply go away! Yeah! But that suggestion wasn't well received by the bathroom project stakeholders.

So, could we stop the screen getting so dirty in the first place? I considered some options: a water filter, no screen (perhaps back to a curtain), a special screen (I didn't know what was possible), special soap (again, I didn't know what was possible), changing our behaviour (say, baths only), stopping the clag sticking to the screen, ...

The last of these seemed interesting because I could think of a way in which this was similar to a problem that I already understood. I have sometimes used an additive in our car's screenwash that makes water less likely to stick around on the windscreen, and wondered whether I could use it on the bathroom screen.

The cost of researching it - not least because I imagined I'd need to spend time working out what materials my shower was made of and checking for compatibility with any chemicals and so on - and the possible difficulty of application and the likely cost and the fear of getting it wrong and ruining the screen all conspired to put me off looking into it very eagerly. But the lack of desired returns from my other strategies meant that eventually I came back to it.

Making a cup of tea at work shortly afterwards, I was bellyaching to a colleague about the problem, and my idea of putting some additive into my cleaning water. And I got a lucky break! It turned out he had recently bought some stuff for coating his car windscreen which turned out to be suitable also for showers and so had applied it to both, and he was very happy with it.

Accepting his recommendation cut down my potential research effort, making the task more tractable in the short term. So I bought a bottle of the same product he'd used, read the instructions, watched the online videos about how to apply it, checked that I could unapply it, cleaned the screen very thoroughly, and finally applied it (a not insignificant investment in time). And I have been really pleased with the results. Ten or so weeks later, I'm still cleaning once a week but with the squeegee and the coating it's a much lighter clean and the screen looks good for the whole seven days.

Here's another example: initially I used washing up sponges for cleaning the bathroom but I found that they tended to leave tiny grains of their abrasive layer behind, which I then had to clean in some other way than with the sponge itself. So I started using an old washing up sponge (when the one from the kitchen needed replacing I would promote it to bathroom sponge) but that didn't have the scouring power I wanted. So then I wondered whether there were specialist bathroom-cleaning sponges - I know, so naive - and I now use one of them. But then I noticed that there was some accumulated soap stuff that the sponge struggled to remove, stuff that was hard to see against the white enamel of the bath until it had built up into a relatively thick layer.

I found that I could scratch this residue away with my nail and so I could generate a set of properties I might look for in a tool to do the same job better: flexible, strong, thin, easy to handle in confined spaces, non-damaging to enamel.

When confronted by a tool-selection problem with constraints, and without a solution, I find that going and poking around in my toolbox or my shed can be really helpful. I might not be able to imagine what kind of tool fits all of the requirements, but I can probably imagine which of those requirements might be satisfied by the tools, or potential tools that I can see in front of me.

I maintain several boxes of bits and pieces in the shed which are invaluable in fixing stuff and also for daughters' robot building projects:

Rummaging around in one of them, I came across an old credit card, which works perfectly as a scraper. As with the car windscreen coating, it turned out that others have been here before me but that doesn't negate the utility of the tool in my context, even if it does remind me that there were probably faster ways to this solution.

So, shiny bathrooms are great, and the journey to the symbiotic relationship of a suitable tool used in a suitable way to solve the right problem need not be linear or simple. But what here is relevant to work, and testing? Part 2 will begin to think about that.

Footnotes:1. I do the washing up every day too! That's right, I'm a well-adjusted modern man, as well as handsome, young and with a full head of my own hair!
Categories: Blogs

Fail Over

Thu, 08/18/2016 - 22:50
In another happy accident, I ended up with a bunch of podcasts on failure to listen to in the same week. (Success!) Here's a few quotes I particularly enjoyed.

In Failing Gracefully on the BBC World Service, David Mindell from MIT recalls the early days of NASA's Project Apollo:
The engineers said "oh it's going to have two buttons. That's the whole interface. Take Me To The Moon, that's one button, and Take Me Home is the other button" [but] by the time they landed on the moon it was a very rich interactive system ...The ultimate goal of new technology should not be full automation. Rather, the ultimate goal should be complete cooperation with the human: trusted, transparent, collaboration ... we've learned that [full autonomy] is dangerous, it's failure-prone, it's brittle, it's not going to get us to where we need to go.And NASA has had some high-profile failures. In another episode in the same series of programmes, Faster, Better, Cheaper, presenter Kevin Fong concludes:
In complex systems, failure is inevitable. It needs to be learned from but more importantly it needs to become a conscious part of everything that you do.Which fits nicely with Richard Cook's paper, How Complex Systems Fail, from which I'll extract this gem:
... all practitioner actions are actually gambles, that is, acts that take place in the face of uncertain outcomes. The degree of uncertainty may change from moment to moment. That practitioner actions are gambles appears clear after accidents; in general, post hoc analysis regards these gambles as poor ones. But the converse: that successful outcomes are also the result of gambles; is not widely appreciated. In the Ted Radio Hour podcast, Failure is an Option, Astro Teller of X, Google's "moonshot factory", takes Fong's suggestion to heart. His approach is to encourage failure, to deliberately seek out the weak points in any idea and abort when they're discovered:
... I've reframed what I think of as real failure. I think of real failure as the point at which you know what you're working on is the wrong thing to be working on or that you're working on it in the wrong way. You can't call the work up to the moment where you figure it out that you're doing the wrong thing failing. That's called learning. He elaborates in his full TED talk, When A Project Fails, Should The Workers Get A Bonus?:
If there's an Achilles heel in one of our projects we want to know it right now not way down the road ... Enthusiastic skepticism is not the enemy of boundless optimism. It's optimism's perfect partner.And that's music to this tester's ears.
Image: Old Book Illustrations
Categories: Blogs

Understanding Testing Understanding

Fri, 08/12/2016 - 07:40
Andrew Morton tweeted at me the other day:
Does being able to make a joke about something show that you understand it? Maybe a question for @qahiccupps— Andrew Morton (@TestingChef) August 9, 2016I ran an on-the-spot thought experiment, trying to find a counterexample to the assertion "In order to make a joke about something you have to understand it."

I thought of a few things that I don't pretend to understand, such as special relativity, and tried to make a joke out of one of them. Which I did, and so I think I can safely say this:
@TestingChef Wouldn't have thought so. For example ...

Einstein's law of special relativity says you /can/ have a favourite child.— James Thomas (@qahiccupps) August 9, 2016Now this isn't a side-splitting, snot shower-inducing, self-suffocating-with-laughter kind of a joke. But it is a joke and the humour comes from the resolution of the cognitive dissonance that it sets up: the idea that special relativity could have anything to do with special relatives. (As such, for anyone who doesn't know that the two things are unrelated, this joke doesn't work.)

And I think that set up is a key point with respect to Andrew's question. If I want to deliberately set up a joke then I need to be aware of the potential for that dissonance:
@TestingChef To intentionally make a joke, you need to know about some aspect of the thing. (e.g. Special Relativity is not about family)— James Thomas (@qahiccupps) August 9, 2016@TestingChef If you're prepared to accept that intention is not required then all bets are off.— James Thomas (@qahiccupps) August 9, 2016Reading it back now I'm still comfortable with that initial analysis although I have more thoughts that I intentionally left alone on the Twitter thread. Thoughts like:
  • What do we mean by understand in this context?
  • I don't understand special relativity in depth, but I have an idea about roughly what it is. Does that invalidate my thought experiment?
  • What about the other direction: does understanding something enable you to make a joke about it?
  • What constitutes a joke?
  • Do we mean a joke that makes someone laugh?
  • If so, who?
  • Or is it enough for the author to assert that it's a joke?
  • ...
All things it might be illuminating to pursue at some point. But the thought that I've been coming back to since tweeting that quick reply is this: in my EuroSTAR 2015 talk, Your Testing is a Joke, I made an analogy between joking and testing. So what happens if we recast Andrew's original in terms of testing?Does being able to test something show that you understand it?And now the questions start again...
Categories: Blogs

Know What?

Tue, 08/02/2016 - 23:07

I regularly listen to the Rationally Speaking podcast hosted by Julia Galef. Last week she talked to James Evans about Meta Knowledge and here's a couple of quotes I particularly enjoyed.
When discussing machine learning approaches to discovering structure in data and how that can change what we learn and how we learn it:
James: In some sense, these automated approaches to analysis also allow us to reveal our biases to ourselves and to some degree, overcome them. Julia: Interesting. Wouldn't there still be biases built into the way that we set up the algorithms that are mining data? James: When you have more data, you can have weaker models. When discussing ambiguity and how it impacts collaboration:
James: I have a recent paper where we explore how ambiguity works across fields ... the more ambiguous the claims ... the more likely it is for people who build on your work to build and engage with others who are also building on your work ...Really important work often ends up being important because it has many interpretations and fuels debates for generations to come ...  It certainly appears that there is an integrating benefit of some level of ambiguity.Image: 
Categories: Blogs