Friday, September 19, 2014

The Let's Test Oz experience ...

The Personal Journey

There were a lot of nerves on my behalf leading up to the Let's Test Oz conference this week.  My wife was diagnosed with severe anxiety much earlier in the year, made much worse by the death of her mother.

Leaving her whilst I went to the neighbouring country for the best part of a week was worrying, but I also knew it had potential to be good for her to know that she could cope and was more capable than she knew.  It could be a great point on her recovery, or a point where she badly relapsed.  Thankfully we had a backup plan in our number one son, who is a very capable and level-headed man at sixteen now.

It was great that at Let's Test I had a couple of people who knew this backstory - I was checking into the hotel with physical and emotional baggage - and were there to check on me and understand if I didn't want to be super-social all the time.  I think at times this support net is what the online testing community has learned to weave best.

What I'm going to do is pick up on a few of the best moments and revelations from the conference, to condence my experience ...

The Train Ride

I had a two hour train ride in the morning with tester Kim Engel, fresh from our conversation about flat earth's.  It was a great journey, and a great conversation to get me in the mood - I almost talked myself hoarse.  It reminded me one of my favourite pastimes with my son is travelling, because whether hiking or driving, it gives you opportunity to talk and explore.  With my son it's about exploring history and ideas around it.  With Kim it was exploring aspects of testing and mental health that we've both had personal journeys with.

With such conversations it's actually often a disappointment when you reach your destination, because you've enjoyed the journey too much.  Yup - I was actually slightly sad to arrive (but not for too long).

Coaching Testers Workshop

I've spoken previously about James Bach and Anne-Marie's workshop on coaching testers.  It had some great ideas circulating, and James had found some examples from movies of people coaching others.  I had to say I feel slightly ashamed that I have not yet watched the Magnificent Seven, despite liking Westerns - I need to remedy this at some point!

Great focus was given to understanding yourself as a coach, who you are and what your behaviour is.  How do you interact with people?  What's important to you in others?  Then to look at the person who is looking for coaching, and asking what they need in a coach.  Sometimes that's not you.  James and Anne-Marie talked frankly about their coaching, and that occasionally they will recommend an individual goes to the other for coaching.

As always with a good workshop, this included reams of hands-on.  We logged in anonymously to Skype and we got to work with other people in the room, taking turns to be the coach and to be the student.

I'll be absolutely blunt, I thought the person I was coaching knew me, and was playing a game.  I kept getting incredibly frustrated but trying to be calm with them.  When it came to debrief it turned out that the person I paired with had never used Skype before and was a slow typist.  I learned a valuable lesson there that especially online you need to get some form of check-in about the student, how they feel and what their backstory is, rather than "leap into" the coaching.

It's a lesson I really should know - but it's amazing how the session helped to reinforce that, and instead I made my own assumptions that it was someone trolling me.

So you wanna be a boxer?

SoftEd ran a couple of boxing training sessions which were absolutely superb.  I occasionally do something similar in Wellington (I've talked about Josette, one of our instructors here).

Boxing training is a very intimate kind of training.  You pair up with someone, and take turns doing exercises,with one person using gloves to hit, and another holding the pads being hit (you hit the pads, not the person!).  You have to both be mentally in a similar zone, and develop a kind of rhythm with each other.

That makes it oddly quite a social activity.  The best kind of pairing is when you're both supporting the other with "try moving your stance", "we're half way, don't give up", "c'mon, keep going, nice" - it was some of Bach and Charrett's coaching tips in miniature.

Half the success of any conference like this is being able to mingle with people who you don't know.  Meet new people, make new allies.  The boxing sessions proved to be a great "meet and greet" event, with conversations with other boxers strung out through the rest of the conference.

Which brings me to this tweet ...

The boxing was one method (I'll talk about the other in a moment), but there were the experienced keynote speakers touring the conference, and there were other speakers such as myself.  But mingling and listening I realised something important, "everyone has an experience report and a story inside them, just keep your ears open".

As I said in that tweet, in some ways listening to the raw stories from others was a great opportunity to really spread my net over the conference.

Put yourself around people with passion

The other method of putting myself around a bit came at lunchtime.  It all felt like being a bit at High School with "who shall I sit with?".  Occasionally I just really needed to eat and dash, and I'd just sit alone.  But I tried to use it as an opportunity to sit with new people, and introduce myself.

I however earned myself the "special little snowflake" achievement on Monday lunchtime though - finding a table of people I didn't know, and asking if it was okay to sit with them.  They were from a completely different conference!

But this actually led to a bit of a revelation.  They were a conference of ultrasound operators and were curious about who we all were.  So I got to tell them a little about software testing, as well as ask them a bit about their conference.

My interest was hightened a bit.  Recently my brother and his wife had a daughter.  One thing that surprised me was the ultrasound. When my son was "under development", we had a couple of ultrasound pictures of him, and they were a bit like one of those 3d puzzles.  If you stared at really hard, you couple perhaps make out a skull.  Ish.

But for Thea, her pictures were strikingly clear, the technology had really come along leaps and bounds.  So surely being an ultrasound operator was a lot easier now?  Wrong.

Turns out they do more and more checking with ultrasound as it's such a non-invasive proceedure.  They were spending a conference looking at example images of different ailments such as a damaged appendix.  In their normal routine they might not see some of these examples, so it was all about improving their ability to look and and recognise issues.  They were using the conference to broaden their experience from other operators, so when they went back to work on the Thursday they were just that bit sharper.

The improvements in technology made some things a lot easier.  But at the end of the day it required a human eye and human judgement.  And damn it - wasn't the same true about testing's relationship to technology over the same timeframe?  Some things had got easier, but at the end of the day, it's about a human eye and human decision making.

This led me to an important understanding - I learn a lot about testing, but not always from testers.  I'm quite a talker, but I'm a good listener too.  In fact when we go touring around New Zealand, my wife despairs of me as I really enjoy going into quiet shops and talking at length to the owners, where we come from and finding out some of their history.

I am attracted to spending time with people with a passion, and an ability to animatedly talk about that passion.  It doesn't have to be anything I'm interested in - in fact often it helps if it's not.  Just this year I've spun pieces off from conversations with Josette my boxing instructor and Lotz my musician friend.

Testing has a good few parallels, and if you listen out, there is knowledge out there from people going through similar experiences which you can Shanghai, and add to yours!

An interesting chat with Erik Peterson leads to some self-reflection ...

I had an interesting lunchtime chat with Erik Peterson where we talked about heuristic models for testing.  I came to the realisation that I'm heavily dependant on an "experiental model" (although I do use others) - basically "when I used to program, I once saw this happen" or "I've seen a bug like this in a similar system".

That's of course okay, as long as you realise it's fallible.  And it's greatest fallibility is you aren't looking for a bug you've never experienced before - you have a blind spot to anything you've never seen or heard.  It also made me look at some of my writing - overall my writing heavily leans towards a "series of experience reports", occasionally postulating a model from this experience.

It's an interesting look in the mirror at my way of thinking - also tying into my previous post about trying and failing.  It's like I'm drawn to having a pool of experiences to base judgements on.

My talk

Yup - I'm not being egotistical here, but not only did I enjoy giving my talk on "deprogramming the cargo cult of testing", but to my shock, I walked out with an expanded take on it.  Some of the questions asked allowed me to think and explore the subject in ways I'd not expected.

The topic was really talking about the system of testing we've put into place over the last 12 months, and I talked about it back in my piece on exploratory testing.  We put together a new way of testing when we moved to being agile, but we engaged with our customer to talk to them about what they felt they got out of our old methodology?  What did they feel they get from a test plan, a test script or a test report?

The point was this was to form a matrix of values from our customer - this meant whatever approach we took for testing it needed to address these values in some manner.  If it didn't then we weren't done with our approach, it wasn't hitting the needs, and we needed to rethink.  But not only that, we had to make sure we were making "how our testing worked" visible to the customer.

An example of this would be how the customer saw test scripts both as "proof of testing" and "training material".  We ended up using qTrace to record our sessions as "proof of testing" and for "training material" sharing an internal testing handbook we already had, and making sure we kept it up to date sprint on sprint.

Someone noted the piece tied in a bit with Keith Klain's keynote where he talked about avoiding being overly whiny or self-centred about testing's problems, but understand the person you report to "has problems and needs" that you don't know of.  To try and go to them, not with more problems, but trying to help and aid them.

The bottom line to this approach was that we made sure we had an evangelical fervour to delivering real value to our customers, in a testing approach that we felt accurately addressed their needs.  In this I really was pleased we seemed to carry on the spirit of Alessandra Moreiraa's talk about engaging and influencing people.  In fact the conclusion from our talk was it would be a mistake to wait for a major shift from waterfall to agile before engaging with a client to ask if the testing you're performing is really "ticking the boxes" from both the client and the test team point of view.

Final thoughts ...

These are the things that really stuck with me - a very interesting conference, with a lot to take home.  I sadly missed the Fiona Charles keynote at the end of the Wednesday, which I was looking forward to.

There was a lot to take in, but also fun to be had along the way - one of the funnest activities being Joanne Perold and Carsten Feilberg's workshop where we replicated problems in communication by using a Lego building exercise to replicate the the software building process.  This was an exercise I would love to try again with different rules to see if it causes some of the outcomes I expect.  Likewise the boxing and the coaching activities were nicely hands on.

The team behind Let's Test Oz really did an excellent job in making this happen - the venue and food was amazing, everything ran well, and everyone seemed to come to the conference ready to really share and engage.  I made sure before writing this, I sent an email to the key players, asking them to circulate to all who needed to read it.

Thanks guys!

A tale of two tutors ...

Several of my recent blog posts have talked back to my experiences at University, and I'm somewhat shocked to find out that 25 years have since passed, and yet I still feel I'm out on an active path of learning.

University was a formative time for me in a lot of ways, as it is for many people. Apart from the independence aspect, it's also a time where you learn there isn't a textbook syllabus anymore.  Instead there's a whole world of different ideas and theories in your chosen discipline, many of which contrast, and education at this point becomes trying to reason between this knowledge.

An important role in shaping me as a human being through these years came from two very different individuals who were my tutors in physics and astronomy at the University of Sheffield, and this post is about who they were and where that influence came from ...

Professor Fred Combley

Fred was my tutor for physics.  It's hard to think of him as "Prof Combley", he was Fred, a very warm and gentle person, whose office was almost always open to any student.

His approach for our third year course in cosmology was really unique.  He didn't teach a single lecture. We did!

Now that might seem like a very lazy lecturer, but it was both inspiring and revolutionary.  Each week, three students were given an area to prepare a lecture for, and we'd present the next week.  We'd have to go to the library to research, and we had a session booked with Fred to discuss the topic.  We were in fact given the keys to our own learning.

I loved doing it, and loved sitting and exploring the topic with him.  Versus the many other lectures we attended that seemed to get droned out by writing out mathematical formulae by rote, this was us, the empowered student, picking out what was important and what to say.  And the information from those lectures stuck.  I loved it so much that it led me to trying teaching, and is one of those reasons why I'll almost always put my hand up to speak or present when given an opportunity.

But what stuck with me most of all was as I said was his warmth.  I wasn't a model student at all.  At school I was the top of the class in physics, but at University within a group of my peers, I was decidedly below average at the subject, and that caused me a lot of stress.  I felt almost robbed of my identity.  In fact I failed exams in my first two years and had to do resits (I did say that failure was my natural style).  But with perseverance I got through - although despite not being a star student, as I said Fred always had a time for me, and more respect than I felt I deserved at the time.  I always felt he believed in me, even when I didn't believe in myself, and that was important getting through some tough times.

I bumped into him again three years later - I was popping back to the University of Sheffield to have some lasers etched for my project at the University of Essex.  [Essex was partnered with Sheffield, who had superior clean room facilities].  He saw me in the corridor, and to my amazement remembered me (as a physics tutor I imagined I was one in a sea of faces), and dragged me into his office to catch up - he was very pleased to hear I was still in physics and enduring (although I was still failing exams).

Thanks to the wonder of the internet (Fred was involved with CERN, so got it first), we kept in touch afterwards, with an email a year to catch up and tell him about my new career in software, and the interesting things I was learning.

Then one day in 2001, I realised he was no longer on the staff at Sheffield University, and smiled as I realised he must have finally retired.  Sadly no, as another of the teachers at Sheffield, Dr Susan Cartwright had to inform me that he was due to retire, but alas an aggressive cancer had killed him before that could happen.

I was gutted.  Anyone who knew him couldn't help but be gutted ...

Prof David Hughes

David Hughes was my tutor for astronomy and in many ways the polar opposite to Fred.  I mentioned him of course in previous blog posts ...

David was a hero of mine, I'd seen him speak on astronomy on The Sky At Night, and he was simply brilliant in the televisation of the Giotto probe intercept with Haley's Comet.  I've put up links before (such as this) of him speaking - he is without doubt the greatest speakers I've ever heard, able to convey incredible concepts in a simple and entertaining form.

But there was a problem - we really didn't get on well, at all.  I talked in my previous post that when we were asked to plot out a graph of 200 points, I asked why we weren't using a computer for the task, to which he replied ...

"You can feed a computer a string of numbers, and it can add them and divide them, and multiply them faster and more accurately than any human will.  It will even draw a mean graph line through them.  But it will never go "uh-oh, that piece of data looks out of place".  Only human beings can look at, and if need be, ignore data that could be erroneous.  If you just feed numbers into a computer without an intrinsic understanding of the data and the measurements you're using, you're essentially cutting out human judgement and intuition.  You're not here to learn how to enter numbers into a machine, but how to see those patterns for yourself, and trust to your own judgement over that of a computer."

In that blog, I mentioned how inspirational that quote is, and how I return to it again and again.  But what I didn't tell you was how I responded to it at the time.  I hated it, and I didn't like him for what he said.  I felt belittled over my idea to use a computer program (I was good at using computers for such tasks) and annoyed at him over it.

It didn't help that unlike Fred, David seemed to have his favourite students to whom he was always joking and telling funny stories to.  However when I had to spend time with him, he always seemed to be being on my case and being incredibly pushy.

It took me years though to realise that compared to those student who got the funny tales, that I got the best from David Hughes.  I had excuses galore to hide behind, but he had a habit of blasting them away, and not letting me settle for "not giving it a go", frequently pushing me with "you can do better than this".

Another conversation I remember having was of me whining about a 1000 word essay we had to complete in a week, and how that was almost impossible.  He told me bluntly that if I did my research and got the ideas in my head, I could easily sit down an knock out that length (as he often did for astronomy magazines) in just a couple of hours.

And damn, he was right!  I've been working on this blog post for about an hour, and Microsoft Word tells me I'm over 1200 words.  Every time I do a word count on a blog or magazine piece, I just remember that conversation, and go "damn, you're right again!".  So it's fair to say he has been a major influence in my writing on testing.

But he wasn't always so gruff, when I started to appreciate how he'd not let me sabotage myself with ideas that "it can't be done", I also remembered him talking to me when I failed an astronomy paper, telling me not to let it worry me too much, and it was a mark that we could work on and "make alright" next time.

Unlike the tragic story of Fred, Prof David Hughes is retired, but still talking about the subject with which he has boundless passion, and has a nice gig doing astronomy lectures on cruise ships ...

Models of tutoring and coaching

These two characters were going through my head for the last week, and I've realised how much in any mentor or coach we need elements of both of them.  David's pushiness to not allow me to settle for second best or to sabotage myself with negativity before I'd begun and Fred's gentle nature to help pick you up and gently say "well, what do you think you can do better?".  A mixture of hardness and softness dictated on which is needed to coax out the best from an individual.

Everyone probably need a bit of pushing and a bit of picking up, but the focus for different individuals is completely different, because those people, how they view themselves is different - making learning and coaching itself a contextual activity. It makes me think of Shifu, the master and trainer in Kung Fu Panda.  He originally tries to teach Po how to do Kung Fu in the exact same manner he's trained all his other students.  But the training is disastrous, and Po eventually gets really dejected.  However when Shifu sees Po in the kitchen climbing shelves to find cookies, he realises that Po has more raw ability than he originally thought.  He comes up with a training scheme different to the others, and which focuses on what motivates Po.

In many ways, all these elements turned out of be part of the model that James Bach and Anne-Marie Charrett presented at Let's Test Oz 2014 as part of their workshop on coaching testers.  However instead of the two data points in my life called "Fred" and "David", they've been researching, trying out models, and looking for fundamental truths about what makes good coaching.  I'm going to resist the urge to paraphrase them but it was without doubt the session at Let's Test that I found myself taking the most away from.  If you have the chance to ever attend their workshop in future, I recommend you get yourself along to it!

[And if you can't manage that, hound them to write a book about it]

Saturday, September 13, 2014

Reality, models, and all that jazz ...

My last few posts, on flat earths and my experiences at University, have brought me very close to a topic I've really wanted to cover this year - how do you work out what reality really is?

One of the reasons I wanted to go to University was to study the latest knowledge about physics and astronomy.  But to my initial surprise that wasn't completely the case.  In most subjects, but especially the history of astronomy, we studied how past civilisations viewed the universe.  

So we were presented with ancient models of the universe, why they thought like that, and about the critical observations which ended up leading to new a understanding (and occasionally the controversy that produced).  Prof David Hughes encouraged us to look out through old papers, for instance where astronomers were convinced that the craters we see on the Moon were actually volcanoes.  The point was not to laugh at these people from history and feel smugly superior because "we know better than you", but to understand how our ideas evolve as we find out things from observation to develop a sense of intuition around science.

Having thought about this a bit, this is my view around what scientists such as Stephen Hawking call "model dependant realism" - which is a kind of scientific version of Plato's allegory of the cave (the one that says we're often trying to understand reality not from direct observations, but from watching shadows on a cave wall).


Much like in Plato's cave, "stuff" is going on.  And actually here "stuff" can be the science of the universe, a historical event, even a crime.  It's something that's going on - and it happens whether or not it's being observed, and whether or not people have an adequate model to understand it.  I mean it's not like a tree in a forest starts to fall over and goes "wait, should I hold on until someone is around to hear this?".

Now here's the important thing, we might think we know what reality is, but we don't all we can say is we think we have a good working model of it (you'll get this as we go on).


As we watch "stuff happening" we start to gather information or evidence from this observation, and build up more and more of a record.  Just because we have a pool of information though doesn't mean we know what's going on - we often will witness things which at the time seem contrary to what we understand (heck, that's the basis of most scientific experimentation).

"The Model"

The model is basically a set of rules or an understanding taken from evidence which we use to piece together an idea of "what reality might be".  

If you like, it's pretty much the best fit of how reality might work based on the evidence we have.  To do this we have to sift through our evidence, and try and work out which pieces seem the most accurate, and try and work with them.  Whatever model we have has to fit the evidence we have as best we can.  If we're using an existing model, and a lot of the evidence doesn't fit the model, we have to think about building a new model.  

We also have to be careful (as with our flat-earther yesterday) about using the model we have to eliminate any evidence we see which doesn't fit our model (also see; climate change denial and Creationism), and I'll talk about that later.

An example of this process would be for instance Tycho Brahe recorded astronomical information religiously for years, but it was only when mathematician Johannes Kepler started to work on this that he found several mathematical patterns, developing astronomical models to cover them.

But you'll also see it in court - when someone is under trial, a jury is presented with two sets of evidence "for" and "against" including eye witness statements, forensic information, records.  Some of this information will be contrary.  They're then put into a room to sift through everything they've been exposed to, they will deliberate and choose some evidence over others, to attempt to determine a model (guilty or not guilty) that is their best determination of what "reality" probably was (ideally beyond reasonable doubt).

And again in history - as a historian you may find the letter written from a leader after a great battle going, "the enemy surrendered in droves at the end, and we took their arms, and after they pledged to go home and fight no more, they were free to go".  However an archaeologist finds a mass graveyard nearby.  Is that the burial site of the dead from the battle, or did some great atrocity happen?  [A good example is King Richard and his nephews]

So we ignore some evidence?

Unfortunately, sometimes we do have to, but we have to think very carefully when we do.  If you're timing how long it takes raindrops to fall down a window and you get a set of times like "11.1 seconds", "10.4 seconds", "110.9 seconds", "17.2 seconds", "9.2 seconds".  One of those times just stands out as questionable.  Is is possible it really took 110.9 seconds?  Did maybe you leave the stopwatch on without realising?  Or did you do a transposition error when recording?  That piece of data is highly suspect as it doesn't fit in.  You carefully repeat and repeat, but never get anywhere close, and so you do somewhat suspect that data to not be reliable.

In a trial, someone might say they've seen someone at a certain place and time - could it be they're mistaken?  Or that they just saw someone who looked like them?  Or they are flat out lying?  All are possibilities.

I myself suffer occasionally from a very weird condition called sleep paralysis.  It means sometimes I come out of sleep in a very confusing half-awake, half-asleep manner.  You are aware of being awake, but your body is still paralysed as if asleep.

It's a very confusing state to be in.  For instance, this year I've woken up to find my dead father-in-law sitting on the bed, and he had a decent chat to me about some guy he knew who used to go fishing.  The scary thing is it feels incredibly real, and it registered in my brain is as real as any waking conversation I've had.  That said, I don't find myself believing in ghosts from it, despite having witnessed this.  The clues are (a) I suffer from sleep paralysis and (b) I couldn't move or speak.  This sadly causes me to have to disregard the experience, although of course I do want to believe I've spoken with him, it's hard not to.  As with our flat-earther, there are things we find ourselves emotionally attached to, and you don't get any more emotionally attached than to someone who died who you loved.

Where it all goes wrong

The models we have of course are only as good as how they help us to make sense of this thing out there called "reality" - to understand our observations, and indeed to predict what may happen.  Science, understanding of some historical event and indeed religion are all models.

Sadly humanity goes wrong though when we start to hold a model sacred, especially if there's a vested interest in the continuance of that model.  Invariably then that happens is the model is used to filter out observation and evidence that contradicts it, instead of using this evidence to drive towards a better understanding of reality.  Our flat-earther yesterday pretty much dismissed all evidence provided because it contradicted his model.  If he could find the smallest area of doubt, he dismissed the whole piece of evidence entirely.

You see a similar phenomenon when fundamentalist religions (of all denominations) will dismiss anything which seems contrary to their teachings and understanding of the universe.  It happened to Galileo when he suggested the world might orbit the Sun, instead of the Sun orbiting the Earth, and he wasn't alone.

Sadly this is a course of action which can keep people in ignorance, and leave us "blaming the observers" over seeking better models.  I actually don't think religion needs to be all bad - many of them have been around for hundreds of years.  I fundamentally (but not too fundamentally you understand) believe that religions in a similar manner to Shakespeare's old plays, endure because they speak to an inner questioning voice we all have.  [Shakespeare's plays though based hundreds of years ago are still fundamentally about love in the face of obstacles, revenge, jealousy, betrayal, joy - human conditions which continue to endure in modern life]

As a scientist when I used to read Genesis's version of the creation, I didn't take it literally, but liked the idea of how God related to each step of creation, and it all formed a logical sequence.  Following our current scientific models of the evolution of the universe, we do believe still that it did begin with light, created land then the sea, then animals and finally humans.  Ish.

But it's the ideas of how we should treat each other that endure the most and from that we hold most dear and relevant from religion.  Ideas like "thou shalt not kill", "thou shalt not steal", "thou shalt not commit adultery" and "thou shalt not covet thy neighours ass" do make a kind of sense, and have been the fundamental basis of most human laws (although sometimes badly followed according to if you're the person in society "with the power).

In general, I as a human am going to find myself a lot happier if I don't go around killing others, and likewise it causes others around me to reciprocally be happier.  Okay there might be special cases, for instance if someone is intent on committing harm to myself or my family, but generally it holds.  A similar logic goes for stealing and those other items.

Even the most ardent atheist I know has a strong sense of justice, which in many ways is a distillation into our society of those originally religious laws (just leaving behind some of the God bits).  But even so, it's another example of people finding some things from a model useful and applicable, so using them to move forward, but leaving those elements which can't be supported behind as superstition and "past their sell by date".

We can do that without leaving the religion behind altogether if we still find it has value - not everyone who wants to revisit their models has to be an atheist.  Indeed, I'm not.  As a young man in my early 20s I found the teachings of Christianity really important, especially the redemptive message of forgiveness for ourselves and others.  We don't have to be perfect, we just need to strive to be our best (sadly not all Christians take it that way).  And hence my faith allowed me to strive to be the best person I could be (over sadly how others will use it as a metre to measure others as unworthy).  I took the pieces from the Bible that inspired that, but with the case of pieces from Genesis I took them more as a "metaphorical explanation" over a "nuts and bolts description of how the universe was created".  After all it was a model, and we'd happened to find better models ...

The bottom line is though, don't be a slave to your models!  And yes, that includes certain models of testing ... cough ISO 29119 cough ...

And finally

Since writing about Prof David Hughes and my experience with him, I've found a couple of presentations by him.  He's without doubt one of the most inspirational speakers and thinkers I've met, and even if you've got a casual interest in astronomy, you may find a lot to take away from his talks ...

Friday, September 12, 2014

Do we live on a flat earth? A mental exercise ...

Yesterday I was exchanging some travel information with fellow Kiwi tester Kim Engel (btw, she does contract testing, and she's currently free) ahead of Let's Test Oz.

During the course of the day we were talking about opinion vs fact, and she came out on Twitter with this,

This really interested me, and we ended up continuing our conversation on email afterwards.  I've spoken before about asking "how do you know if something is true?".

I remember in a book I read at school about someone scoffing at some tribe they met "because they probably think the Earth is flat".  We though can usually feel smug and secure we know better, "because someone told us it's round".

But typically that's all it is, someone telling us something, and we take it as gospel.  In truth one of the huge problems is we're bombarded with so much misinformation, we often have to relearn some of the garbage we learn.  When I was five, we were watching John Craven's Newsround and I tried to understand what the big deal about the Voyager probe launches were - I mean we were sending manned spacecraft to other stars and planets, what's the big deal.  My parents asked me what I meant, and I said we saw it all the time on Star Trek (they were avid fans).  Somehow I'd thought we really had ships like the starship Enterprise.

And through school I learned so much misinformation through friends.  Somehow in my mind I had it that the Earth spun to create gravity.  I took as gospel that there was a witches house in the trees on the school grounds (this was infant school folks, not senior school) and that kinds that went into the woods there never came out.  Heck even Santa was a difficult thing for me to shake, I'd seen him at department stores, and my parents had even faked a phone call from him once to get me to go to bed (and muddled my concept of reality in the process).  I will not even begin to tell you all the misinformation I had about sex from my peers at high school!

Okay - so believing everything you hear can, and in my case did, get you into difficulties.  But what of the other polar opposite?  And this is where my conversation with Kim went.

Is the Earth flat?

Looking at the Earth and is it spherical or flat from first principles.

Using my own eyes and experience.  Certainly I've looked at the Earth every day in my life, and from my perspective it certainly does look more like it's flat than spherical.

But haven't you seen picture of the Earth from space, it's a sphere!

That's true, I've seen pictures of the Earth where it does look spherical.  But (a) I've never seen the Earth beneath me like that with my own eyes and (b) I've also seen similar pictures of Tatooine in Star Wars, so I know the technology to fake that view is there.

What about the fact that you can fly from the UK to NZ via America OR via Asia.  Getting to the same point via two opposite directions.  It's a sphere!

I've only ever travelled here via America.  And even then for starters I didn't really know where we were at any given point, and half the time it was in darkness.  My wife once travelled there and back via Asia, but having used her to read maps when we're driving you'll forgive me for not putting much faith her word.

If you've ever seen a partial lunar eclipse, you can see the Earth's shadow, and it's round!

You see, this is where a lot of people get confused.  Of course the Earth is round.  It's round, but flat.  Just like a plate.

But the Earth is a sphere.  Different cities are in different timezones.  When London is in daylight, Auckland is in darkness.  You wouldn't get that with a flat earth.

I only know if it's daylight or darkness where I am.  I can't be in two places at the same time.  I might Skype someone in London, but how do I know what I'm seeing isn't just a studio set somewhere?


And on and on it goes.  This is actually a peek into what my grandfather is like.  He's one of the smartest men I know, has studied all his life, and someone I admire a lot.  But he's of the opinion the Apollo Program was faked, and try as we might ...

Kim had noted she'd seen this phenomenon herself especially when people try and engage about religion or climate change.  "Correlation vs. causation comes up a lot with the remaining sceptics out there.  The difference between opinion and fact is empirical evidence. But empirical evidence has a way of changing, and the facts change with it."

As with my flat-earther or my Moon conspiracy grandfather, people have an odd way of filtering out evidence that doesn't fit their model.  With such people, will you ever be able to provide sufficient information to their satisfaction of how their model is wrong?  No, because invariably they'll alway find some area of doubt in something you've provided to undermine it.  Their model is right, not because they can prove it, but because there is a 1% area of inaccuracy or doubt in yours.

Carl Jung noted this has, and always will be a problem.  There are a lot of people like us who are used to dealing with things from a pure logical point of view.  And therefore when you're presented with a disagreement will get as much information and logic to sway the other person.  But not all people are wired that way, with some people having a much more emotional over logical response to things, and it being their core way of understanding and responding to the world.  And it's hard (if not damn near impossible) for a person using logic to sway someone who is used to making decisions in this manner.

And even people who think they're super-logical can have their off days!  I mean the kind where you pass by an electronics shop and go "oh, they've released a new phone ... it's only slightly bigger than the one I have, and not much faster, and it's so expensive, it makes sense to keep to my current phone".  And as you take the bus home surfing on the new phone you bought regardless, asking "how did this happen again?".

But the bottom line is you can't sway or convince anyone of anything if they're not coming to the conversation with an open mind.  If their mind is already made up, good luck!

The Razor

How do you keep an open mind though, without being duped by everyone who had a tale?  This is probably where we need to reach for the razor blades, namely Occam's Razor.

Occam's Razor accepts that basically you can't witness and observe everything (like our flat earther wants to).  But neither should you take everything on faith either.  When empirical evidence is provided to you, it's either going to be true or false, but which is more likely?  Typically the one with the least number of assumptions.  The universe after all tends toward simplicity over elaborate and needless complexity.

So if the spherical Earth is being propagated as "a big lie", then ...
  • Someone has been paying off every space launch since 1955 to fake evidence of it being flat.
  • This conspiracy would include both the USA and the USSR, bitter enemies at the height of the Cold War, but united to suppress the truth about the flat Earth
  • When someone flies from London to Auckland via America, it's okay, because that route exists.  But if they go via Asia, then the pilot flies around Africa for a bit until the passengers get disorientated and it gets dark.  He then heads via America, to New Zealand, stopping off at a secret replica of Singapore they've build deep in the Andes.
  • When it's morning in New Zealand and I ring my mother in London, and she says it's evening, she's lying.  As is my brother when I ring him.
  • You're all lying to me, because you're in on it!

However if the spherical Earth was true, then maybe it's just that some people are convinced that Governments will lie to us about just about everything (although really ... that's a good point as well).

Is this a testing skill?

Being able to navigate bias is a key skill for a tester.  In many ways testing is about like Kim mentioned, finding empirical evidence to support or challenge someone's opinion.  Typically for us testers that "opinion" is that "we've done coding this software, and it's ready for release".  That can be an opinion as emotional and occasionally irrational as our flat-earther (let's face it, no-one likes to think they've done a bad job on something), especially if timescales are getting tight.  Great we have a new user registration page, and I can create A user - but does that mean I can create ANY user?  And what do I mean by ANY?

Right now there are all forms of different approaches to testing out there, but which are the ones worth trying and putting your neck and reputation on the line for?  Maybe you want to just stick with the method you've always done, because it's always worked (doesn't that feel a bit like you're being a flat-earther there?).  Then again, maybe something really does sound too good to be true, and you feel right to be sceptical without more information?

No-one can make these decisions for you, and ideally no-one should make those decisions for you.  However to really attain our potential as smart testers we need to understand our own thought processes, and challenge ourselves over the filters we use regarding new ideas and information.

Tuesday, September 9, 2014

Alice and the Cheshire Cat talk ISO 29119

Alice took a quizzical look at the Cheshire Cat, “so you’re a tester?  But you’re a cat!”.

“Oh,” the Cheshire Cat beamed, his almost rictus grin fixed permanently and inflexibly across his face, “in your world that might be a problem, but not in this one.  Did you have a better candidate in mind?  Would YOU want to sit in a meeting with the Mad Hatter, changing his seat every few minutes, or tell the Queen of Hearts that the project might have to be delayed?”  He purred a very self-satisfied purr to himself.  “No … I think you’ll find in reality, or as close as it gets in this place, I’m the perfect person for the job.”

Alice looked at the heap of worn leather bound books that the cat had seemed, almost impossibly, to leasurely recline on top of.  They came up to Alice's shoulders, and she ran her eyes down the spines ...  “What are all those books?”, she inquired.

“Oh THESE?  This is our documentation … we’re all ISO 29119 compliant over here you know.  You try telling the Queen of Hearts it's not needed.  Actually my last few predecessors did just that, and that’s how I came into the role, if you get my drift,” at which the cat extended his claws and made a mimed guillotine motion against his own throat, “but then of course business can be so cut throat, especially,” and he giggled to himself, "when it comes to the subject of test execution."

Alice took a book at random from the pile that said “test plan”, and started to read it.  “But this is all a nonsense, you’ve just written ‘all work and no play makes Jack a dull boy’ repeatedly with a few headings!”

“And yet it is to the standard”, beamed the Cheshire Cat ever smugly, “the standard says we must have a test plan … so we got a book and called it the test plan.  It mandates we should have certain titles within that book as well, and being compliant, we have those as well.  But then …”, and the cat began a very self-satisfied purr, “nowhere does it say that it has to be based on any kind of reality, which as you know in this place is the hardest type to find.  And as the regulation says, we are allowed to tailor the actual contents as we most see fit ... as long as it follows a template.”

The cat stood up, gave a little stretch, and started to do that annoying vanish from the tip of his tail, until all that remains was his smile.  That smile still beamed as he said, “and do you know, I think this template suits me just fine …”  And with that he was gone.

Monday, September 8, 2014

Pieces of paper don't change reality ... Stop 29119

My family are a fairly spiritual one - my parents were after all hippies when I was born, and hence an awareness of ourselves, and our role within the world to make it a better place was an important part of our education.

My parents tried out a few churches as they grew older to find one which felt right - spiritual, and yet we were an engineering family.  Somehow religion and science really needed to be good bedfellows - so we ended up joining Winshill Methodist Church.

The Reverend there were a very passionate man named WH Pittam, who just had a great way with the congregation.  There are few men of the cloth who'd so relish playing Barabbas (the man who's freed instead of Jesus) in the Easter Story drama.

He could also do a mean sermon, and aim it at children.  One of the ones I remember being a favourite was the one about a lamb who'd had enough of being a sheep, and decided it wanted to be a lion.  And so it went through a transformation to make itself more of a lion - it got a wig to make a shiny mane of hair, it got a pair of vampire teeth to look more fierce, if tried to practice it's roar, and even went to far as to put a sign around it's neck to say "LION".  But it didn't really work.  It could come close to making itself appear more like a lion, but in truth it was no closer to being an actual lion. 

Certifications for all!

There is something quite close to this in the Wizard Of Oz - the Scarecrow, Tin Man and Lion go to the Wizard to get some trait they think is missing.  But their encounter and trials with Dorothy brings these abilities out of themselves naturally. In the end, the Wizard is exposed to be a snake oil salesman/con man.  And he still has a trick up his sleeve - yes the trio have found their own brains, heart and courage, but he gives them trinkets - the Scarecrow a paper certification (to show he has knowledge), the Tin Man a token of appreciation (to show he's a philanthropist) and the Lion a medal (to show he has courage).  The odd thing is the Wizard has done nothing really, all he's given each individual is a token to show others they have a trait that they already know they had.  Yet in the movie, everyone's most thankful (even though the Wizards real plan was to send them on a suicide mission).

So why does any of this matter?

Because it's happening right now, in testing.  I've spoken before about the potential pitfalls with ISTQB qualifications being a way to measure if someone is a "tester".  In the Wizard Of Oz, when the Wizard produce a paper certification, did the Scarecrow really get any smarted from it?  The Scarecrow showed his smarts through his trial with the Wicked Witch Of The West.  The Wizard didn't help or aid or mentor, be just pulled out a certificate.

The Wizard though said the certificate was important because it would show other people that the Scarecrow was smart.  But the truth is the Scarecrow had proved that smarts himself, and he owned that intelligence as his own - why should someone unconnected and unqualified essentially steal that with a certificate?  Someone really should have taken the Wizard to task.

But now we have a new issue on the testing horizon - ISO 29119, the standard on "software testing", and it has a lot of people in an uproar.  It's been doing the rounds for a while, and supposedly "been out for feedback".  People who have traced the standard have noticed that in all that time there doesn't seem to be much change occurred at all, although there has certainly been no shortage of comment.

The issue is it attempts to implement an idealised "best practice" to software testing as a mandatory standard, and encourages customers to make sure their vendors are compliant.  This is a great idea - we can follow an idealised model if the whole project delivers to testing following the strict idealised model for requirements and developments.  The harsh fact is though that projects are tailored around objectives, constraints and contexts which will take them in different ways off that "idealised path", and testing needs to flex with the rest of the team to find the best pragmatic method to match the project as it's being delivered.  To take on a rigid standard and sit in our ivory tower complaining that the project hasn't been delivered correctly for testing (as the ISO standard seems to encourage) reeks of unprofessionalism, and it really does not serve our customers.

It has potential to be detrimental to our industry, to the minds who make up it, and as said to the teams and customers we have skin in the game with.  The only people who seem to benefit from this are those who will be in charge of selling and auditing the standard.

There is a petition to force a rethink of the ISO 29119 standard, and I really encourage you to read through and make your own mind about this.  As before all I can encourage you to do is read the facts and make up your own mind.  As a tester, this standard will impact you.  Much like our lamb, having a sign saying "lion" won't make it a lion.  Having a certificate with a tick won't ensure your testing is really the "best practice" being followed for your customers.

The petition is here, and yes, I signed.

Some useful articles,

Friday, September 5, 2014

Reputation and engagement

I have a friend Lotz who owns and runs a record company and band in Wellington.  I've managed to catch his group, HMR Records a few times.

Lotz is a proper professional musician, and has built up HMR over several years.  They specialise is reggae, but have a stable of musicians which include a lot of genres.  As always I there's a lot you can learn in life by having around you people who have a genuine passion in life, and just listening.

A few months ago I ended up getting a lift back with him from a gig, and he told me a bit about this mixer he uses.  Sound checks for gigs are always a bit of a nightmare (heck, they're a kind of test, aren't they?).  Typically most time goes for the headline act, and that can leave very little for the other acts to tune up, especially if there are problems, when things tend to overrun.  A core part of his business model wasn't just the diversity of his group (something for everyone, and no two songs too much the same), but also that he could pre-program his mixer, so they needed minimal sound check time.

This was really important because he wanted to build up the reputation that HMR was "easy to work with" and no egos.  They wanted to build up that kind of reputation because it was important to get new work to be seen as someone that was no hassle, and could "play well with others".  No one wants to be an act after all that people go "uh, not them".  And Lotz is about one of the most relaxed people to deal with you could imagine.

Today I ended up meeting with several senior managers, and found that conversation with Lotz came flooding back.  Of course in our own way in the IT vendor industry, we're striving as individuals to get the same reputation of "easy to work with" for our companys' sake.

But what does that mean?  Does"easy to work with" mean we obey every customer request?  Even the ones which we know will cause problems down the line?  Of course not.

The trick of course is this mysterious thing called "engagement".  And there's no real absolute recipe - different people do this differently according to their personality.  Engaging means listening to what the customer wants, and potentially asking for clarification.  It's then guiding them through your experience on other similar projects, highlighting pitfalls you've experienced before, to attempt not only to mentor and enlighten them, but also to try and give counteroffers of what you can do (if you can't just say yes to their requests).

Engagement with a customer is a complex thing, because in essence it's a relationship.  And any relationship is a tightrope.  You are trying to give them the benefit of your experience, hopefully without belittling them or playing the "I'm right, you're wrong".  I know with the relationship with my wife, I've occasionally tried to use my advanced experience in science to tell her she's wrong, and as I settled down to bed on the couch that night, wondered to myself "well, that could have gone better".  No-one likes to be made to feel stupid, no matter your qualifications - and we tend to shut such people out.  The greatest teachers we know are not the ones who made us feel ignorant, but the ones who made us go "ah!", as we came to a realisation.

A comment I loved from KWST3 is a great relationship with a customer is like the movie Inception, where you're planting the seeds of powerful ideas about testing in the minds of your clients.  But the clients need to feel these ideas are their own, not ones "mandated" and forced upon them.  As I've often said, we tend to value ideas we understand more than ideas we're forced to take up, and trying to make our actions and activities understood (with clients and team) is a big part of my role.

If you're in New Zealand and reading this, I really recommend if you ever get the chance to go to a HMR Records gig, you go along - they're really great guys, and well worth seeing live!

By the way the Time Will Tell YOLO - before you think the YOLO is too gangster - it was actually a charity event run by a Les Mills PT to thank the hospice who'd looked after her father.