Skip to content

Hiccupps - James Thomas
Syndicate content
James Thomas
Updated: 14 hours 39 min ago

Feyn Arts

Sun, 08/02/2015 - 22:21

The other day, I said I was reading Surely You Must Be Joking, Mr Feynman! by Richard Feynman and was captivated by it. I've finished it now, and I've pulled out a handful of quotes.

I love this on bad (or as he puts it, cargo cult) science and how strongly it relates to the way I want to perform and report testing:
But there is one feature I notice that is generally missing in cargo cult science ... It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty - a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid - not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked - to make sure the other fellow can tell they have been eliminated. Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can - if you know anything at all wrong, or possibly wrong - to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition. In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.And I love this on drawing (one of many skills he acquires in the book, including speaking Portuguese, safe-cracking and drumming) where I'm thinking of the parallel it has with the way in which in testing we seek to bring together art, craft, intuition and science, theory and practice:
I noticed that the teacher didn't tell people much ... Instead, he tried to inspire us to experiment with new approaches. I thought of how we teach physics: We have so many techniques ... that we never stop telling the students how to do things. On the other hand, the drawing teacher is afraid to tell you anything. If your lines are very heavy, the teacher can't say, "Your lines are too heavy," because some artist has figured out a way of making great pictures using heavy lines. The teacher doesn't want to push you in some particular direction. So the drawing teacher has this problem of communicating how to draw by osmosis and not by instruction, while the physics teacher has the problem of always teaching techniques, rather than the spirit, of how to go about solving physical problems. They were always telling me to "loosen up," to become more relaxed about drawing. I figured that made no more sense than telling someone who's just learning to drive to "loosen up" at the wheel. It isn't going to work. Only after you know how to do it carefully can you begin to loosen up ... One exercise they had invented for loosening us up was to draw without looking at the paper ... I found that my drawing had a kind of strength ... which appealed to me. The reason I felt good about that drawing was, I knew it was impossible to draw well that way, and therefore it didn't have to be good - and that's really what the loosening up was all about. I had thought that "loosen up" meant "make sloppy drawings," but it really meant to relax and not worry about how the drawing is going to come out.And then this on the, ahem, danger of analogy:
Another time somebody gave a talk about poetry. He talked about the structure of the poem and the emotions that come with it; he divided everything up into certain kinds of classes. [...] Dr. Eisenhart ... said, "I'd like to know what Dick Feynman thinks about it in reference to theoretical physics." I got up and said, "Yes, it's very closely related. In theoretical physics, the analog of the word is the mathematical formula, the analog of the structure of the poem is the interrelationship of the theoretical bling-bling with the so-and-so" -- and I went through the whole thing, making a perfect analogy. The speaker's eyes were beaming with happiness. Then I said, "It seems to me that no matter what you say about poetry, I could find a way of making up an analog with any subject, just as I did for theoretical physics. I don't consider such analogs meaningful."
Categories: Blogs

Having a Feyn Old Time

Sat, 08/01/2015 - 07:14

I've been reading Surely You're Joking Mr Feynman! by Richard Feynman and I'm captivated the eyes-open way his anecdotes relate how he notices things, how he feels about things, how he feels about how he feels about things, what his interest in things is, and why, and how he is constantly motivated to experiment and learn and understand, and then share his understanding.
Image: Google Books

Categories: Blogs

Wan Solo

Tue, 07/14/2015 - 08:17
I asked three reviewers for comments on a piece of writing. One a work colleague; another I've spoken to on Skype but don't know well; the third I've met only on Twitter.

I got back three very different perspectives.

I got back dispute, question, criticism, suggestion, cross-reference, reinforcement and praise, in varying measure.

I got back the seeds of new thoughts.

I got back things I can choose to take or leave.

I got back the makings of a stronger piece than I would have created alone.

I got to say that even though I know this can be a great way to work, and beneficial to all concerned, putting myself out there and dealing with what comes back can still be a challenge.
Categories: Blogs

A CEWT Aspiration

Wed, 07/08/2015 - 09:52
Testing Ideas. The topic was deliberately ambiguous, not only because it reflects the situations in which we frequently work, but also to provoke and admit a wide range of discussion at the first Cambridge Exploratory Workshop on Testing. And while we got that, we also got some common themes. From my perspective, three stood out:
  • personas
  • analogy
  • timing
There's a wealth of thought on exploiting personas to guide testing (and even design) by, for instance, defining a set of user profiles that represent important parts of a product's user or customer base and then trying to put yourself in their mindset, have their concerns, use the product they way they would to accomplish the aims that they would have. The use of related tools like de Bono's Six Thinking Hats has  also been well covered in testing.

Across the presentations and discussion we got some interesting thoughts on what other kinds of personas could provoke ideas too:
  • putting yourself in the position of your colleagues; testing in the style of others
  • putting yourself in the position of some aspect of your own personality
  • putting yourself in the position of the software
and also on being aware potential limitations of aspects of an individual's own persona.  For example, it was suggested that perhaps those testers who like to plunge in and quit might need to exercise caution when testing a new idea (perhaps suggested in a meeting) because of the risks of being seen as too confrontational or negative (regardless of whether or not that is an intent) and risk alienating whoever suggested it and maybe losing or holding back something worthwhile. Testers who prefer to wait and reflect might be seen as allowing ideas to grow.

I wonder whether personas are a way of cutting across testing heuristics we're already familiar with such as SFDPOT and HICCUPPS. Maybe we can part-define certain kinds of users as being particularly interested in some of the areas identified by those mnemonics. For example, choosing to test like a marketing colleague might focus on product claims and (apparent) satisfaction of current user desires but care less about how data flows around the system.

Karo's idea of putting yourself in the position of the software is one I found really interesting. It feels related to the notion of offers that James Lyndsay's recent improv/testing workshop suggested. In that context, the software interacts with the tester by making offers (click on this button, enter text into this field, retrieve some data ...) and, in this one, the software is not only offering but additionally an agent with its own needs. The potential for a perspective change when you're blocked for ideas seems huge.

We talked about when and whether it is important to consciously adopt a persona and then more broadly about the advantages of consciously adopting any approach that you might feel you already just do naturally or is common sense. Once you're aware of it as a technique, you can choose to apply it in certain circumstances, can gain inspiration from the fact that it's in your toolbox when you're after a new angle of attack. If you only have access to something unconsciously, then you're always waiting to see whether you happen across it. Which isn't to say that you shouldn't exploit the stuff that just happens, but try to watch how you do what you do - and how others do it.

My own presentation was on analogy and gave a specific example - joking and testing - that I've been exploring for a while.  Analogies are incomplete and so are heuristic, but offer the potential for great value in bootstrapping, building and exploring models of the system under test.

Gabrielle made analogies between particular life skills and experiences and testing. For her, the risks associated with riding a motorbike, and the things she does to mitigate those risks, map well onto risk in the domain of testing; similarly, softer skills such as handling interactions with the various managers she's had at the charity shop she volunteers at.

Both Mike and Liz's talks included the question of where and when testing takes place in projects they've worked on. Giving testers permission to apply themselves in the design phase of a project, and getting buy-in from others on the project, was flagged up. Testing the gaps between stories and testing for gaps between stories seemed to be a couple of places where this is generally going to be relatively uncontroversial and could show how testers can add value.

We touched on the fact that ideas are sometimes kept deliberately ambiguous in order to keep all the stakeholders on board with a project. Each can feel that the thing being built will do what they want while it is still only an idea couched sufficiently vaguely.

The problem of when to try to squash the space of possibilities into a specific actuality was thought potentially difficult and links back to the point above about exercising caution when testing of ideas lest the progenitor of the ideas becomes disillusioned. It was suggested that presenting evidence of the current status and letting the stakeholders themselves recognise the way the wind is blowing might be a useful approach.

The timeliness of an idea and how that affects the traction it gets was something Neil talked about. He gave an example of a simple utility to collect logs and other trace files from multiple locations on a machine after a failure was observed. It was created by a tester who saw a need and taught themselves to code just enough to implement a solution.

The solution was shared with colleagues, who taught themselves enough code to modify and extend it and so on. It not only saved time, but its existence improved the skill set of the team as they inspired one another to do more with it.  Management saw the tool and asked for the test team to develop it for inclusion in the product for collecting the same kind of data on the customer side.  

An idea nurtured can bring unexpected value to unexpected people from unexpected places along the way.  Neil emphasised this by talking about how he's taken the local Lean Coffee meetups format into his own team meetings and got positive results.

There were stacks of things we didn't cover much but might have with more time or had the discussion gone different ways. This is a just flavour of them:
  • opportunity costs of ideas. Building a utility, learning to code etc are useful but what else wasn't done as a result?
  • sharing ideas. Persuading others that your ideas are worth pursuing. Persuading others that you're no longer convinced by an idea.
  • ownership of ideas. Who owns them? Who gets credit for them? How much does it matter?
  • meta-ideas. Trying to analyse where your ideas come from and looking for ways to get ideas from other places.
  • how to choose between ideas. Often the problem isn't coming up with ideas, it's a surfeit of them.
  • prototypes and pretotypes. Getting a physical thing in front of people can elicit more, more useful responses than describing the idea of the thing.
It's a tenet of Lateral Thinking that so long as the end result idea is valid in some way, the route to it doesn't matter so much. One of the things that motivate me to be involved in this kind of workshop and the other local meetups and to blog is the increasing realisation that idea begets idea begets idea begets idea begets idea (you get the idea) and even though everything along that chain might not be useful to me, I can often end up somewhere that is.

The act of having those ideas, making those associations, creates an environment in which having further ideas is easier. And more ideas means, on average, more good ideas. And I think having a local workshop was a good idea. Let's try and do it again.
Categories: Blogs

Trying to be CEWT

Sun, 07/05/2015 - 08:03

I attend, enjoy, hopefully contribute to, and get a lot from, the local tester meetups and Lean Coffee in Cambridge. But I'd had the thought kicking around for a long time that I'd like to try a peer workshop inspired by MEWT, DEWTLEWT and the like. I finally asked a few others, including the local meetup organisers, and got mostly positive noises, so I decided to give it a go.

I wrote a short statement to frame the idea, based on LEWT's:
CEWT (Cambirdge Exploratory Workshop on Testing) is an exploratory peer workshop. We take the view that discussions are more interesting than lectures. We enjoy diverse ideas, and limit some activities in order to work with more ideas.and proposed a mission for an initial attempt to validate it locally on a small scale.

Other local testers helped to refine the details in usual the testing ways - you know: criticism, questions, thought experiments, challenges, comparisons, mockery and the rest - and a list of potential attendees was drawn up. In parallel I solicited advice from the groups that had inspired me, asking what's worked well and what what hasn't, particularly in the events and in the organisation of them.

This post aggregates and roughly sorts their responses, removing mentions of specific groups or people. I'd like to thank all of them for being so forthcoming and open with their experience and advice.

I wanted to pull two specific comments out, two that I tried to keep uppermost in my mind thoughout:
  • As you will understand: there is no best practice.
  • The thought is this: at a peer workshop, I should consider everyone my peer. For the duration of the workshop, I will attempt to listen to – and question – anyone who I share the room with, regardless of whether they have more or less experience, or whether I generally consider their work good or poor, whether I am fascinated, bored, repelled, awestruck or confused. 

I started this process at the end of April and yesterday (July 4th) we had CEWT #1. There were a few rough edges, and I learnt a thing or two, and I already know some things I would change if and when we have another, but there'll more on that later. For now, here's that aggregated advice for anyone else thinking of trying it ...

StartingWe started small: in a kitchen with only a few people.

I have no idea how many interested people you know, but it is smart to keep it either very small to start with, which you can organise by yourself. Or make it a bit bigger, but then you should have some help.

I’d thought about doing this for about 12 months before our first one, and it was only when I started to talk to the others about the idea that I found they had similar thoughts and things started to move.

SizeMy experience is that you need about 10 people to have good discussions in LAWST style. 7-8 people could be okay, although I don't think you need facilitation with such a small group. You also have the risk that if 1 or 2 do not show up, your group becomes even smaller.

We have limited it to a maximum of around 25 people. As we are always looking to improve, this all might be subject to change in the near future.

I had assumed [the sense that in a peer conference everyone is granted the status of everyone else's peer] was a central guidance to peer conferences – even if, in practice, it was occasionally hard to see such respect in action. However, I’m no longer certain of this; when I’ve shared my position with other peer conference organisers, it has been (generally) either alien or less important. I think this gets hard with >8 people, and is pretty impossible with >15. A 25-person room will naturally form groups, gurus, acolytes and pariahs – so it’s ludicrous of me to expect larger peer conferences to work this way.

Personally, I think the max size for any peer group is rather under 20.

AttendeesWe have a very simple approach to application and invitation - if someone asks if they can come, then they can. Done. I tell people that there's a cutoff, what the cutoff is, and that people who apply when numbers are under the cutoff can come, and people who are later can't come.

Currently, I ask prior participants to set the theme and the date, so they know before anyone else. This gives them precedence, but if they don't take the opportunity, they don't get to go.

Wrong people: who am I to judge? However, if someone applies out of the blue, I'll talk with them so that they can judge if they're the right person. Usually their judgement is sound.

If someone's interested enough to ask to come and to give up their time to be part of it, then they're in - whether they 'fit' the group, or not. We have had people who didn't fit, and sometimes they've been wonderful contributors, sometimes they've triggered good conversations and interesting realignments. No one has walked out yet. A few participants have complained about others, and I can deal with that as facilitator if something is said early enough. I sometimes find my own comfort challenged – but I don't think it's my role to exclude someone, and I'm sure that the group is muscular enough to chew someone up and spit them out if it absolutely has to.

We are thinking of adding the possibility to choose one speaker chosen by the participants.

All organisers can introduce one (sometimes two) others to the peer conference. We often try to invite somebody outside of the testing circle to add some other views to our conferences.

If you are inviting people, then invite people you think will have something interesting to say on the topic rather than people you know or feel you need to invite out of loyalty – remember it’s a firstly a learning opportunity not a social gathering.

Even if you don’t know someone well but want to invite them, don’t be afraid to reach out and ask them – most people like to be invited to these things.

I find that the more diverse the group, the more it offers guarded respect to each individual: our two-people-with-less-than-two-years-experience thing helps with the diversity.

The Organising TeamWe are organized as a small core group with assigned roles - which rotate per event - to some of us to organize the peer conference.

A small team will help give the idea some momentum, generate more interesting ideas and share the effort of creating the event.

Play to people's strengths - we are all very different with unique skills and personalities, but we each bring something to the table.

If you have a team then agree roles (we change roles each time) to ensure things get done. Generally you will need:

  • 1 x Content Owner – responsible for describing the theme, reviewing and feedback on abstracts, ensuring all attendees have an abstract.
  • 1 x Facilitator – responsible for managing the flow of the discussions on the day (doesn’t need to speak)
  • 1-2 x Organisers – responsible for logistics (venue arrangements, ensuring costs are covered either by sponsorship or attendees, providing travel and hotel information, keeping in touch with attendees etc.).
We have introduced the formal role of 'content owner' in the conferences to keep us from going all over the place. He/she chooses the speakers. The conferences are centred around experience reports and discussions are facilitated by a facilitator.
Find some awesome people to work with, it's a lot of work for one person!
Logistics: BeforeChoose relevant and open topics that encourage a wider range of views and discussions.

Find a good venue.

Food is important - quality grub adds to the vibe.

All participants are obliged to send in a proposal for a small presentation (organisers too).

Asking for abstracts (and receiving them) helps to focus people's minds ahead of the day.

Chase people for abstracts, review and feedback on the abstracts. In my opinion, if you don’t have abstracts then some attendees will forget to prepare and attempt to wing-it resulting in less interesting talks and discussions. However that does depend upon who the attendees are.

Don't underestimate the effort required to invite people or encourage people to attend (if you have an open attendance). You will have people who drop out in the lead up to the event so be prepared.

Plan ahead, we have started planning 3-4 months ahead to give people time to commit and provide abstracts. When you invite or accept people to attend, ensure they know the outline plan with milestones such as confirming attendance, when initial and final abstracts are due etc.

Keep regular contact with those who are attending to keep them informed of plans, reminders of upcoming milestones, hotel and travel arrangements etc.

Logistics: On the DayIf you can, find someone to do the distracting mid-workshop logistics (i.e. who’s eating what, taking calls from late people).

Trying to get through all of the talks works well - fast paced and high energy.

Not worrying about getting through all the talks works well too - slower and deeper.

Breaks: as long as possible without losing momentum and direction. Proper, multithreaded conversation happens in the breaks. The “talks” are a primer for the discussions, the discussions a primer for conversations – and connections and ideas grow from those conversations.

Set-up: everyone should be able to see everyone else’s face, all the time. Other than that, don’t be precious about room layout, drinks, stationery, power supplies, matching tables or any other fripperies. Indeed, the more informal, the better. Help participants to feel comfortable, not coddled, and certainly not privileged.

Visuals: I strongly discourage slides, and encourage flip charts. They’re more immediate, more interactive, and less goes wrong. I prefer flipcharts to whiteboards, as they’re more permanent and one can flip back.

Dot voting lean coffee style gets everyone involved.

Keep presentations nice and short; 15 minutes max.

Ordering: the room gets to decide what goes early (the facilitator gets a deciding vote) – so topics at the end usually get less time. This can make them more focussed, and the speaker will often be able to tune what they have so that it suits the attention of the room.

We don't have a content owner deciding what gets attention or priority, we don't have a scribe making public notes, we don't have a mission. We all agree at the outset to be facilitated, which helps - but we don't necessarily decide what 'facilitation' is.

The relatively-fast turnover of topics helps, a lot.

FacilitationAsk the room to accept you as someone who will regulate the ebb and flow. Don’t direct (or dictate) the content.

Accept that, as facilitator, you’re not really at the workshop, and give the primary part of your attention to emotions of the people in the room, not to what is being said.

Monitoring people's energy and staying fluid with structure and content helps keep things moving.

When I'm facilitating, I try to do the job with as light a touch as possible - basically I keep a queue, keep my eye on time, and try to help the group stay within the discipline of conversing in a way that lets everyone talk, and everyone listen. Even that, however, requires my complete attention on the room - which means I don't make many notes for myself or contribute much to the conversation.

The facilitator is not a peer. The participants give the facilitator their attention, and their permission to stop and start them, in pursuit of a greater goal then their own individual airtime. The facilitator accepts their temporary status, and returns the favour by serving the group and putting his or her own needs aside.

Name cards can help your own flow.

Getting everyone’s attention focussed from chat to the group: There are clutch of approaches. Most work, most use sound or visual cues. I pick up whatever (physical) sound effect I’ve not used recently. Singing bowls, thundersticks, jingle toys. It gets to a point where, when everyone’s concentrating, one has only to pick the thing up to make people switch focus. My favourite was the vuvuzela – a disgustingly loud football horn. I don’t remember blowing it at all (except to try it out).

For each new topic, I try to remember to announce the topic and speaker, ask how much time they want to talk, support them no more than they want, and to ask the room to thank them at the end.

As someone starts their topic, I split the audio recording and also write down the start time, the time the speaker’s asked me to give them, and the time we’ve all agreed to spend on the topic. I write those as absolutes, not relatives, because calculation takes your attention – (ie 10:03:15, 10:13, 10:33). My laptop clock is always in view.

I record audio, and this also keeps track of elapsed relative time (i.e. 0:17:30 since the topic started).

I keep track of the timing info and the current queue on the same topic card that I’ve pulled off the wall – the card that started with a topic title and ended up covered in sticky dots. Keeping track of the question stack/queue is easy – it’s a list, sometimes with indents and squiggles. If sub-topics are spawning more sub-topics, do ask the room if they want to go deep or wide.

Allow the clock to rule, allow the room to override the clock. Don’t worry about going short. The room will need to regularly be reminded of the time available as the stack builds up and time burns down.

Every few questions, I’ll tell the room who the next 2-4 people on the stack. If we’re in open discussion, and I feel the room needs to move on, I’ll catch the eye of whoever is speaking, breathe in as they finish a point, and indicate the next question by pointing to someone and saying their name.

Don’t fear dropping a person from the queue – it’s your job. But don’t drop them slyly, either.

I bite my tongue (metaphorically, mostly) to stop (my) witty interjections; they’re not usually that great, and it’s an abuse of the role the room has allowed me to take. For the same reason, I don’t usually ask many questions – but I don’t absolutely exclude myself, either.

If, as time runs out on a topic, you give participants the chance to pull their questions or comments to let other questions be asked, they might just do it.

As a facilitator, the people who give me problems are those who assume their contribution is more important than the person who currently has the room's attention, the people with one thing to say and a big personal stake in having it heard, and people who stop listening after someone uses a word that is hot (or dull) for them.

I'm sometimes a problem if I get involved, and I'm lucky that people help me rein myself in if I get out of hand. But problems are few and often easy to deal with if one has a feel for the tolerance and firmness that suits the mood of the room (the whole group, not just the loud participants).

If everyone speaks at once, I need to decide when and how and whether to stop them – and if people only speak when their feel they have permission to speak, I’ve done it all wrong and need to shake up the room. Stay between these extremes, let people (including yourself) be human, aim for fine chat, and you’ll have done a job that anyone should be satisfied with.

I find that expression and body position will tell you whether someone has a new point or a follow-on (and if not, just ask), so I think that k-cards in something with <20 people are a constraining gadget.

I don’t tend to give much leeway to an extended back-and-forth between speaker and a single interlocutor.

Discourage bad behaviour more than the person who is behaving badly: Firmly and clearly block people who are being bullies, then swiftly forgive them and allow them a chance to redeem themselves in the eyes of their peers.

See for general ideas: Paul Holland on facilitation.

Success or failure (pick your own definition) is mostly down to the group, not the facilitator – but you are, as Jerry Weinberg might say, responsible for your reactions to the group.
Categories: Blogs

Pizza Chants

Sun, 06/28/2015 - 06:41

So my wife caught me giggling to myself in the kitchen. Why? I'd just seen a really corny pun on peace and peas. It wasn't the "classic" above but it was the same kind of thing. In fact, it wasn't the joke itself that had caused me to crack my face at all, but the thoughts spurred from the desire to make a better one from the same phonetic premise.

The first thing I come up with is:
Give Pisa chanceThis slight variation on the well-known punchline is a plausible sentence but to make it work as a joke I need a context that can produce it. I'm working backwards from a result to look for some setup in which it is coherent:
Did you know that casinos are illegal in some parts of Italy? Apparently a bunch of gamblers held a candlelit protest overnight. They were singing "All we are saying is give Pisa chance."This is also a testing pattern. When you're looking at responses from a system, a useful approach to finding potential issues can be:
  • I've got X. 
  • By changing X a little I can get Y. 
  • Y is plausible. 
  • Y would be bad. 
  • What context could give me Y?
The comedian Milton Jones has a beautiful gag which is a series of one-liners with the punch line "Your house stinks" spread out in his set:
 Anyone here own a cat?
 Any students in tonight?
So, you know what the punch line is going to be, what context might give a laugh here?  He goes for:
Is anyone in the audience an aromatherapist?Which is not only funny, but also a (comedy) rule of three.

Meanwhile, back in the kitchen, I am busy applying another pattern - I think of it loosely as the Spooner - where you can look for the funny by permuting some aspect(s) of multiple elements. For example switching the initial sounds of peace and chance:
Give cheeser pants
Give cheetahs pants
Give cheaters pants
Give cheetah's pants
Small beer, perhaps. No obviously gut-busting laughs here, I'll grant you. But you could imagine contexts in which you could set these some of these up as jokes, although I will say that if you search for "cheetahs pants" as I did, looking for clues to such a context, you get a lot of photos of leopard skin leggings. Which - fashion naif that I am - violated both my expectations and my eyes.

But that's testing too: generate ideas and choosing to use them or not (at the moment). Sometimes rote generation  by some formula like this is productive and sometimes not so much. As it happens, I decide to try to stretch this line further (like some of those leggings) and end up with:
Give peaches pantsWhich I found an amusing idea (this was the point my wife came to ask what had happened to her coffee) although probably a step too far in terms of plausibility... but I later found this picture:

To relate this back to testing with a specific example: imagine you have some functionality that accepts a couple of arguments. You might ask yourself questions like these:
  • what happens if the arguments are given in the wrong position?
  • does the structure, naming, usage etc of this functionality make it likely that users will mix up the arguments?
  • how would someone spot that they had made this kind of mistake?
I find an interesting overlap in techniques and skills I use for joking and testing and I use one to keep in trim for the other. I'll be talking about it at the Cambridge Tester Meetup and the UKTMF next month and then EuroSTAR in November.
Images:, The Crunchy Carrot
Categories: Blogs

Screen Test

Wed, 06/24/2015 - 21:47
Screen is a godsend for those of us who work at the command line on Linux servers. It provides a way to persist session state across connections to the server and to run multiple terminals in parallel from within in one application.

If you've ever suffered from dropped connections to the server killing your session and all of your running applications, or ever wanted to kick off long-running test code at work before you leave, peek at it when you get home and then pick it up again in the morning as if you'd never been away, or run multiple PuTTY sessions on the same server and got lost in a whirlwind of windows, screen is for you.

I use it pretty much every day.

Useful links:
Image: Wikipedia
Categories: Blogs

UI Testing Excellence

Fri, 06/12/2015 - 06:18
So I had a character picker to test across three releases, call them A, B and C.

Between A and B new characters had been added and there was a requirement that a certain range of the Latin-1 character set must be fully-populated by these additions. Between B and C there should be no differences.

Putting functional testing of the component aside, I wondered how I could efficiently compare two versions of a graphical interface component (a) for the characters on their labels and (b) for the codes of the characters represented by them. And then do it again for another pair of versions.

On a hunch, I tried selecting all of the characters in the component from release A and copy-pasting them into Excel. A little inspection convinced me that the characters had been copied reliably. So I did the same for release B. And then I simply used Excel to compare cell values for corresponding cells for the two releases. Something as naive as this does the trick, showing y where they are the same and n where not:
=IF(K3=B3,"y","n")And then I wondered if I could extract the underlying character codes using the CODE function. For the characters I needed to care about in this work, I could do it trivially:
=CODE(B3)Again, inspections convinced me that the codes were the ones I was expecting where I needed them and so I extracted the codes and diffed them with the same kind of comparison. The patterns of y and n was the same in both cases. (In fact, I used a little conditional formatting to make this very easy as you can see in the screenshot at the top which shows a simplified version of the work.)

Then, for each number in the required range, I used a conditional count like this to check that the value occurred only once in the set of codes:
=IF(COUNTIF($B$26:$I$46,B50)=1,"y","n")And then it was easy again to eyeball for the differences I wanted.  Better yet, to do the second comparison all I needed to do was copy paste C's picker characters over the top of A's and reinspect the differences.

We might regard this work as automation. Or not. For me, for practical purposes, it doesn't matter so much: I tried a tool and on this occasion it gave me the value I wanted almost immediately. It's not always like that. It also wasn't the only tool I used in this work; for example, I inspected code changes and differences across the releases using Mercurial too. This helped confirm that my Excel approach to codes was a reasonable one.
Categories: Blogs

Stretching a Pint

Tue, 06/09/2015 - 08:28
At last night's Cambridge Tester Meetup, Karo talked about heuristics (slides here). After a brief introduction to the topic, she walked us through a couple of testing mnemonics:

We then split into two groups for an exercise. While the other group applied FCC CUTS VIDS to testing Karo's kitchen - in fact, a schematic and floor plan of it  - the group I was in took FEW HICCUPPS and a beer glass used at the 42nd Cambridge Beer Festival.

There's plenty of pictures of the glass at #cbf42 but to give a quick description: it's a pint glass with a loosely-themed Hitchiker's Guide to the Galaxy/Beer Festival mashup logo (because it's the 42nd festival, we assume) on one side and a Campaign for Real Ale (Camra) logo along with the festival name and dates on the other. It has calibrations for different amounts of beer, apparently in accordance with some kind of volume marking regulations which have a CE logo.

HICCUPPS is a set of consistency oracles and we agreed to use each of them as springboards for test ideas rather than receptacles of them, to avoid being constrained by whether an idea was "in" the category we happened to be discussing and risk losing it.

Here's a selection of the ideas we came up with. I haven't edited much, only to combine some overlapping items and lose some repetition and the notes I can't understand this morning. We didn't use the internet for the exercise, but I've looked up some references while writing this post and we could certainly use it for evidence and to inspire more questions.

  • is the glass supplied at the festival always a pint glass, this shape, this size, of this manufacturing quality? 
  • is the logo in the same position, in the same proportions, in the same style across festivals? (e.g. compare images in the festival's Flickr account)
  • are the same volume measures always printed on it (pint, half-pint, third-of-a-pint)?
  • does the glass always show the certification of volume using the CE volume mark
  • is there always a theme to the beer festival? Does it need to be reflected on the glass?
  • is it important to the festival that there is continuity or consistency across festivals, glasses etc?
  • is the festival logo on the glass intended to look this amateurish? (First impression: it's like a student rag picture) 
  • There's plenty of space on the glass, why squash the logo up in the way that has been done? (The mice have detail that's hard to see)
  • Would a simpler graphic design have been more striking? 
  • is it important that the measurements are accurate? (To what degree?)
  • are the fonts chosen appropriate for the audience? (No comic sans!)
  • is the use of colour appropriate? (The logo has to sit in front of many different colours of liquid)
  • is there any relationship between Hitchikers Guide to the Galaxy and Camra? Are there any potential negative connotations that could be made? 
  • is the glass consistent with the festival beer listing booklet, posters, staff uniforms etc?
Comparable Products
  • is there a standard shape, size, material etc for beer glasses at festivals? How about at Camra festivals? Cambridge festivals?
  • what about non-UK drinkers, what would they expect from a glass? In Britain, we still use imperial measurements but Cambridge is multicultural
  • pubs often don't have oversized beer glasses (as this one is, where the pint mark is below the top of the glass)
  • how easy is it to clean vs similar products?
  • what do similar kinds of events do about glasses? e.g. do wine festivals expect drinkers to use the same glass for red and white? Do they provide cleaning facilities for glasses? Is that part of this product?
  • is the glass solid? Will it break easily if dropped? Is the flooring chosen to be gentle on dropped glasses?
  • do Camra members have any expectations about the glass based on Camra conventions?
  • we observed what we thought were injection moulding marks on our glass - would hand-made glass be expected by any attendees? (They are already connoisseurs to some extent by going to the festival.)
  • are the volume markings correct?
  • is the time and date information printed on the glass correct?
  • is the vessel suitable for drinking beer from? Is it optimal? (What is the optimal glass for beer? cider? perry? soft drinks? Does it differ across beers?)
  • is the glass dishwasher safe?
  • would a glass from earlier beer festivals be honoured at this festival?
  • what does other festival material say, show, suggest about this glass?
  • does the glass alter the taste of its contents?
  • does the logo imply some endorsement from Douglas Adams' estate? (Particularly since Adams was from Cambridge)
  • is the glass built to last? (If so, last for what duration? The festival, life?)
  • is this really the 42nd festival? (according to who?)

User Desires
  • is it easy to drink from? to hold? to pass between people (e.g. friends for trying a taste, to the bar staff?)
  • is it stable when put down?
  • should it be more tactile, e.g. with 3D logo on it?
  • can this design of glass be stacked? is it stable when stacked?
  • is it easy to fill, can the measures be seen by the bar staff?
  • is it easy to carry multiple glasses (e.g. three in a triangle)
  • is it unique (e.g. for collectors)
  • it is robust?
  • does it have appropriate thermal properties (e.g. help to keep cold beer cold?)
  • is it safe (e.g. will it break into sharp shards when dropped?)
  • do customers desire gender-specific glasses? ("Do you want that in a lady's glass?" )
  • do customers want a glass that signifies no alcohol is in the drink? How about other kinds of specialist desires e.g. markings for gluten-free or vegetarian beer (is there such a thing, we asked? Yes.)
  • how are the glasses packaged for transport? Are they space-efficient?
  • are the production costs reasonable? affordable? 

  • what is the product here? (we have permitted ourselves to switch between the glass, use of the glass, the festival ...)
  • is the thickness of the glass appropriate, comfortable to drink from?
  • do all of the instances of this glass at the festival look the same? Should they? To what tolerance?
  • should there be half-pint glasses too?
  • is the glass consistent with other aspects of branding?
  • what is the Camra logo about? It's looks like it has a lid. Is that intentional?

  • do I want to drink out of it?
  • is it obvious that it's a receptable for liquids? For drinking from?
  • is it suitable for display?
  • does it look good in a collection of such glasses?
  • can it be easily, safely, efficiently transported and stored?
  • will the colours and other markings fade?
  • what else could it be used for? (e.g. holding coins or pens, as a vase, watering flowers, magnifying glass ... but this is a different testing exercise)
  • does it chip easily?
  • could you hurt people with it? (deliberately or not?)
  • is the glass inert?
  • can you stick it in your pocket when you need your hands free?
  • is it compatible with devices for holding glasses (e.g. deckchairs, belts)

  • what is the CE volume marking? Would we need to test it in some way (e.g. check that the manufacturer is licensed to use it?)
  • are there hygiene standards for drinking vessels (e.g. certain grade, thickness, transparency of glass?)
  • are there conventions, contractual agreements, regulations about using the name of Cambridge or Camra in association with events?
  • does the festival have a license to sell beer?
  • do the bar staff need licenses or training to serve beer?
  • some brands of beer might require their product to be served in glasses branded for them?
  • does the logo conform to copyright law (e.g. with Hitchikers Guide to the Galaxy images)
  • does the glass fulfil the needs of the beer? (e.g. to have its head displayed, show bubbles, permit its colour to be appreciated, compared with others etc)
  • are glasses required to be round? (if so, how round? Could it be square? elliptical?)

  • what are common problems of any kind of branded product? branding wearing off, typos, correct copy etc
  • glasses with handles are often a pain to fit into a cupboard

  • the logo might not be obvious to people not in the intersection of Hitchikers and Camra  fans
  • explainability is a kind of testability heuristic
  • the precise location of the festival isn't given, only the town. Should it be precise?

  • this is a pint glass to most Brits at least. To others it might just be a glass
  • is it obvious a beer glass? Probably it is to those familiar with the conventions of such glasses
  • does it obey the laws of physics?
  • is it a practical object, or a collectors item?
  • why does it have a wooden barrel in the logo when the beer at the festival no longer uses them?
  • does the audience expect nostalgia?
  • is it quintessentially English?

Images: Twitter, Wikipedia, Amazon 
Categories: Blogs