Have We Lowered Our Standards On Quality?

This post was inspired by two comments I read online. The comments were left under a video related to the recent problem present on the latest release of the Kindle Color Soft. Thousands of customers have been reporting a strange yellow discoloration on the screen of their new devices, so I headed to Youtube to check if anyone had reported these findings in any product reviews or unboxing videos. The entire first page of search results made some mention of this yellow band defect and the first comment that struck me was "How did they get this through QA?" and under another video review a comment read "Yeah, that's why I never buy products when they first release." 

The comments above made me reflect upon the current state of products and software in the marketplace. It seems to me that companies are not too bothered about releasing their products into the market without addressing problems that are often so apparent, customers like myself wonder how they ever got the green light for release in the first place. Unfortunately for customers, things have been this way for a while.

The first case of botched first releases I thought of was the infamous 'Bendgate' incident following the release iPhone 6 and 6 Plus release in 2014. Customers at the time were reporting that the phone was bending under pressure - sadly for Apple, the skinny jean fashion trend was in full force, which meant skin-tight 'pressure' around the pocket region where the phone would be kept. iPhones bent, so your legs didn't have to (not that they could in those jeans any way). After investigation, in September 2018 it was revealed that Apple knew about the structural flaw in the device all along.

Skipping forward to 2016, Samsung released the Note 7. By the end of that same year they recalling all devices after reports of batteries in the devices overheating and even exploding. Not only was this a huge safety concern for customers, the recall itself also cost the business a reported $5.3 billion! As well as being hugely damaging their reputation. 

And what about software? I've been so kind as to spare you, the reader, of having to read through a long list of all the computer games and upgrades in the past few years that have been released with countless bugs in them, requiring many companies to make public apologies and even pay out refunds. To add, I've also excluded the numerous problems I, and others have reported (and keep finding!) when attempting to perform basic tasks in the day-to-day applications. 

So here we are approaching 2025 and it seems we haven't moved forward much, which brings me to the recent Kindle Color Soft. Released on the 30th October it was a promising move for Amazon. There are very few companies that have released color e-ink reading devices and I must say, as an avid reader they have done great things in pushing the technology and market forward. Customers were now waiting in anticipation for their Kindle devices to arrive - finally, everything we love about Kindles (simple UI, great library, usability etc) would now have color?! Count me in..

Let's just ignore the price of the device for now, that can be argued at another time - or perhaps in the comments? .. When the Kindle Color Soft was first announced, I already knew that I would not get the first batch of the release. The tester in me said let's observe some behaviour and outcomes and time box it at 4 days tops, if all was good I'd go purchase. Why did I have this approach? Subconsciously I expected some problems in the initial release - and just totally accepted it as normal. And it appears it has indeed become the norm! 

Customers that were brave enough (or just, expecting a working product as normal) to purchase, started reporting a problem with their new devices. A yellow band at the bottom of the reader - and within a few days customers had shared their updates on Youtube, Reddit and other forums mentioning that the customer service teams had instructed them to send them back and they'd immediately get a replacement sent out to them once fixed and back in stock, or a refund transferred. 

I then went to the Amazon product page for the Kindle colorsoft, only to find that yes, you still can buy it but don't expect to receive it for another 3-6 months?! (now in the UK that time has reduced to 6-8 weeks)

So going back to those questions that inspired this post. 

Was this issue captured in QA/testing? Was the product under test at any point at all? 

I Googled 'Does Kindle have a QA department?' - the results were old job Vacancies for QA Engineer and QA manager roles. So we'll assume in this case that they do. 
On that basis, I will assume my fellow testers did indeed capture such an obvious flaw, which opens the door to other questions like:

Do these QA Engineers actually test? or are they expected to be 'Quality Coaches', 'QA Motivators', 'QA Coordinators' etc, the list goes on - basically focusing so much on quality (an endless path) that no actual testing gets done. 
Or perhaps these issues were indeed reported, but not communicated effectively? Or they were communicated effectively but to a Manager that was up against some tight deadlines that viewed testing as the final chunk of development, with no problem at all slicing that chunk smaller and smaller if development/coding takes a little longer. 

There are so many layers to problems that are found in products but a few things are for certain, according to me:

1. Customers are increasingly accepting the fact that first releases of products are going to be bug-ridden, what this means for businesses is, you actually might have a great product but due to lack of care for customers, your product will be affected by this and may even see lower volumes of sales for your first release, until at least the good reviews start coming in.

2. The doors of criticism for your product are many and they hold a lot of weight. Product reviews really matter - and nowadays they come with pictures attached or even video reviews containing evidence of their claims too so as to dispel any doubts about the customers' critique. As of this afternoon 11th Nov 2024, the Kindle Color Soft is rated 2.4 out of 5 stars, with 131 ratings.
Pay attention to product reviews. If you are a tester or a QA professional, treat it as test data. 

3. Businesses need to listen to their testers. Go ahead and hire a Quality Coach, Quality Engineer, Quality Assurance team - they can do great things in teams and facilitate some great discussions but one thing is for sure and that is, you need somebody on your team, skilled enough, actively looking for problems in your product and you want to know what those problems are before the customers and market do.

4. There is an environmental concern around product releases and that is, where will these thousands, if not millions of product recalls end up? Recycled we hope. Not stuck in the teeth of some poor whale in the ocean. 

5. As customers we should simply not expect this and in fact, I have vowed to make a conscience effort in actually purchasing the next product I want upon it's initial release. And if riddled with obvious, apparent, easily found bugs or defects - I will be publicly reviewing /even blogging about this as I think this could be a means of change - though a tiny spec in the grand scheme of things. We should keep our expectations high and not simply expect things to be faulty upon first release.

PS. This was part random rant, part random thought dump. Thank you for taking your time to read.

Meanwhile if you have an e-reader to recommend, I am indeed hoping to buy one - birthday is fast approaching people! *nudge *nudge *wink *wink

Read More

Book Review Part 4 (FINAL): The 2 Minute Tester - Chapters 22-33

 

The final part of our book review for 'The 2 Minute Tester" - I'm sure you haven't been as excited for a finale since Breaking Bad! ...right? ... tough crowd..😝

Similar to the three previous review posts, I will briefly summarise the chapters and share my thoughts on each one separately. For a summary of my entire review, you can shortcut and just scroll right to the end of this post.


Chapter 22 - Management: Problem-Solving in QA Management - The Three Ps in QA

This chapter introduces the ‘PPT’ approach to problem solving (is it People, Processes or Tools at the cause of the problem?)

Generally I find this approach to identifying root causes of problems a good starting point. The author admits that there are of course other things that can cause problems with project delivery that fall beyond the 3 mentioned above but it would be a good shortcut for getting started.

In my opinion I feel this could be a good approach to projects in general - not just when there are problems that arise. We should always be assessing our progress and performance not just addressing these things after the problem occurs. 


Chapter 23 - Continuous Improvement (Kaizen)

This chapter briefly summarises the Kaizen philosophy, which is a philosophy of making small steps that can eventually accumulate and lead to big improvements. 

Overall a good chapter with what I think is a great philosophy for change. Again, I feel this can be applied throughout life in general with whatever one wishes to pursue.


Chapter 24 - Risk Mitigation: Stop it Going Wrong Before it Does

This chapter focusses on how one can address potential risks and resolve them before they become real problems for us.

I noticed that the ‘PPT’ problem-solving method from the earlier chapter is referenced, but this time the ’T’ stands for ‘Technology’ as opposed to ‘Tools’ - are these interchangeable? That’s debatable I guess.

The key takeaways of this chapter tell us to ‘1. Risk assess your people, 2. Risk assess your processes, 3. Risk assess your technologies’

I’m not sure these needed to be 3 separate takeaways. The author could have just had one takeaway, “risk assess you people, processes and technologies”.

That aside though, I’m not too sure how I feel about risk assessing ‘people’. For example, in the example the author uses in the chapter, we are told that a contractor was the most senior Automation Engineer on the QAT team but has been talking about leaving for another contract. We could have risk assessed the skills required on the team instead of assessing the person.

To further add, let’s say you required a Senior Automation Engineer on your project, if the best option for hiring was a contractor, then this risk should have been assessed at that stage instead of hiring and leaving the issue to become more important later - this would be truly stopping it going wrong before it does (as the chapter title reads)


Chapter 25 - Defects Adding to The Product Backlog and The Risk to Your Project

This chapter addresses how to handle projects that have a large backlog of bugs/defects building up.

We are told about the complexities that present themselves to the Product Manager when trying to arrange work from the project backlog, juggling between bug fixes and building new features.

In my experience, on agile projects if a defect is found during work on a ticket that is to be delivered in a sprint or by a specific date, then that ticket itself is not moved forward until the defect is fixed within the same pull request/ticket and therefore affects delivery of the project/feature in its entirety so is not added as a separate ticket and put on the backlog.

Alternatively, if or when there are defects found outside of the feature/epic being worked on, then those are the defects that tend to get added to the backlog because they are deemed beyond scope of the current work. In such a case, if we have bugs building up, we’d have to start thinking about making our case to the Product Owner in relation to getting some of the bugs addressed - as also suggested in this chapter.

During this chapter, I found there was emphasis put on ‘defect count’. To me, the count of defects on a  backlog doesn’t really tell us much. The defects there may not have been assessed or refined to determine whether they are even valid bugs yet.

One of the key takeaways of this chapter mentions “look at root causes, any clustering?” - in my experience, the majority of bugs do not contain their root cause so I am not sure how beneficial this could be. As Testers we must be detailed and clear in our bug reporting. We can suggest possible causes which could be very helpful to the Engineer, but the Engineer in most cases would be better suited to identify a root cause, as they would have more technical understanding of the code and how it all works together. Though I do agree, when observing the defect count we should look for any clustering of tickets to help us group them and better manage the workload.


Chapter 26 - Keeping it All On Track: Daily, Weekly and Monthly Tasks

This chapter basically gives us an example of a task list, containing daily, weekly and monthly tasks.

This was quite a disappointing chapter for me because five pages used up on an example of a task list, made up of vague examples of different tasks. Eg, ‘deep dive into automation’


Chapter 27 - Tracking QA Project Progress (Waterfall) 

We are advised in this chapter, to specifically monitor the schedule, costs and defect rate when tracking progress on a project.

These can be applied to any project in my opinion but the author chooses to specifically mention 'the QA Project'. I am left wondering what exactly a QA Project is? Is it something that only the QAs are doing, as part of their own team? For example, having the team look at different options for some tooling that would help them? Or does the author refer to the testing phase of a software project, the ‘QA project?’ - this isn’t clear to me in the chapter.

A negative for me in this chapter is we are told to monitor the defect ‘expected vs actual’ number. I would have liked to understand the author’s angle here a bit more. To me, it leaves me with questions like how does one decide their ‘expected’ amount of defects on a given project? One of our book club attendees suggested this could perhaps be a measurement against previous projects/features, but we all felt that to monitor this, isn't a great idea.

Additionally, this chapter has advises us to create data points when monitoring and make them colour coded. Then we are shown some examples on how this could look. Unfortunately, it doesn’t help us much because the book is printed in black and white.

One of the key takeaways is just ‘use colours’ - I had to go back and re-read the chapter to remind myself why the author has 'use colours' as a key takeaway. It would have been nice to have summarised that key takeaway with some more details. Something like 'use colours when creating data points'

Lastly, the final key takeaway is “Look at trends with defects over the week…” this key takeaway and the chapter implies, that if there is a high defect count, this is a bad thing. For me, a high defect count could be a sign of good progress and that our testers are actually finding bugs before other people do.


Chapter 28 - Stand-Ups: The Power of Introverts

This chapter briefly summarises what daily stand ups are and how they should be used. 

When I finished reading this chapter, I wondered why ‘the power of introverts’ was mentioned in the title. There’s no mention of introverts, or dealing with such people, experiencing a standup as an introvert, how stand ups can help introverts. So to me, it was pointless mentioning 'the power of introverts' in the title.

Scattered throughout this book are quotes, at the beginning of chapters that I feel are totally misplaced. This chapter is one of those that starts with a misplaced quote.

“Not all those who are silent do not want to talk.” - Debasish Mridha

Does this chapter even mention anything about how to help others be more vocal, or in any way relate to the quote above?...you guessed it, NO.

The author suggests we take the chairs away during stand ups and make people stand. This could be a little harsh perhaps, especially nowadays where many are working from home since COVID. Not sure I agree with making everyone stand. Worth noting also that this book was published during COVID so the author would have been aware that offices were closed. Are we expected to stand up even from home? 

Other than the negatives above, the chapter does a decent job of giving us some tips to improve our daily stand ups - but I'm still not sure how this is related to making me a better QAT professional as the title of the book claims?


Chapter 29 - Effective Communication

Generally in this chapter we are advised to communicate in person where possible instead of via emails and other chat applications. The problem I found was that despite that, the chapter focuses more on ways to communicate via emails and apps. It doesn’t give us a method of how to effectively communicate.

This is demonstrated in the 3 key takeaways at the end of the chapter: “1.explore alternatives to email for effective communication, 2. Schedule regular catchups, 3. Think of the rule of 5 and the rule of 3 when emailing”

2 of the 3 key takeaways are about emailing..

Additionally, none of the above takeaways have told us anything about how to communicate effectively.
What if we already do the above 3 in our daily work but still struggle with communicating effectively? Such a scenario is possible in my opinion.


Chapter 30 - Finance: Budgets and QA

In this chapter it is implied that managing budgets isn’t exciting "but like flossing and eating veggies, you gotta do it".. that quote is from this chapter and hurt me just to type it. Apologies.

'People' in this chapter are referred to as 'resources', something I’m not a fan of personally. We are also told about tools that can help when budgeting your QA project - as opposed to what kind of things to budget for.

Essentially this chapter is lost on me, because I am not told why a QA needs to budget for a project.

In my testing career, even as a Test Lead I never had to budget a project myself. When I needed to hire, I’d give some context as to why we need to hire, what role we need to hire for and what the market rate is. I'd then get hiring once management above agreed it was something we could do.

This chapter seems to be more relevant for a QA Manager, so like other chapters perhaps it should have made mention of this in the title.

We are also told to factor in overtime costs when budgeting. Personally, I’m not a fan of people being made to work overtime - but being told to factor this in, feels like we are already accepting that it will definitely happen.

One of the key takeaways of this chapter is: “Get involved early and use Excel” - Why Excel? Since we are talking about budgeting, how about a free alternative? Humour aside, I think the author should have mentioned something general like "use spreadsheets to help with budgeting"


Chapter 31 - Productivity: The Power of Next Action Steps - Keeping Momentum

This chapter suggests some tips that can potentially help us manage our tasks throughout the day, in busy environments where there are constant interruptions that can disturb our momentum.

The tips are summarised as, having no more than 5 items on your action list at any one time and time-blocking each action.

I find myself agreeing with time-blocking actions to help us keep focused on one thing. however the author suggests that when deciding what five actions to work on, we are advised to not care too much about the priority order of the actions, which is also one of the chapter’s key takeaways.

I’m not sure I agree with this, when drafting out ones actions for the day, arranging them according to the priority seems like something we would naturally do. We shouldn’t bother ourselves with other tasks if we have someone or something waiting on one of the tasks we have to complete. The author does however say that this is not a technique that should restrict us so is open to that adjustment.

I like the last paragraph making the distinction between being busy and productive - "The aim is to maintain momentum and not lose focus on what needs to get done as opposed to what actually gets done. This is part of the difference between being busy and being productive"


Chapter 32 - Management: Delegation

The chapter suggests tips on how a manager can delegate tasks to their team in a more effective manner.

This chapter felt a little contradictory to me because there’s mention of “spend time with your QA professional..” as opposed to just delegating tasks to them without setting out the context and goals - but then advises to set expectation boundaries - is this the manager’s expectations? If so, that doesn’t sound like a nice experience for the QA in my opinion because it seems like the manager here is only spending time with the QA so that they can dictate some expectation boundaries.

We are then advised to work up a list with them and set our YOUR thoughts for tactical and strategic success. What about listening to the QA’s thoughts?

It then mentions “…You are empowering your people and giving them clear information on what success looks like.” - this to me sounds like another contradiction.

Giving people information on what success looks like, without first hearing their feedback or ideas is not empowering them.

The end of the chapter mentions “you are delegating but not leaving people exposed and unsure of what they need to do” - the language used here implies that all of the tasks have been dictated to the QA.


Chapter 33 - Management: The Feedback Loop 3 To 1 Rule

This chapter advises how to give feedback in a more positive way.

Again, at the beginning of this chapter we find another misplaced quote: “In a growth mindset, challenges are exciting rather than threatening. So rather than thinking, oh, I’m going to reveal my weaknesses, you say, wow, here’s a chance to grow.”

Yet, we are told nothing in this chapter about how revealing our weaknesses can help us grow. There isn't really anything mentioned about growth - it is assumed that growth comes via the way the feedback is given.

The chapter just mentions that when giving feedback, do not focus mainly on negatives. These negatives may not be weaknesses though - there could be some external factors causing a delay on the project, lack of clarity from business team, conflicts in the team causing problems etc. Though are negatives, these are not weaknesses, they are just factors causing a negative impact on the project.

I do however like that there is a focus in this chapter on positive reinforcement and how beneficial it can be. Too often in QA we focus on negatives - since we are always curious about what things could possibly go wrong, where problems are - sometimes we look at all things in such a way.

Overall, some really good points about focussing on the positives over the negatives when managing a team and some good key takeaways for this chapter.


My Summary of The Book: 

The chapter above causes me to reflect on my feedback sections for each chapter. Perhaps I was reading and focussing too much on the negatives?

Overall this book has some decent advice for those starting out as Testers/QA Engineers, but personally I didn’t take too much benefit from it. A lot of the book did not reflect the industry as a whole and seemed to be focussed on a particular method of QA - having QAs completely separate from the engineering/development team. 

This book would be a good read for a junior QA and will give them a broad range of the different aspects of the testing role.

2/5 stars from me. 

Read More

Book Review Part 3: The 2 Minute Tester - Chapters 15-21

 

We're almost done with our book review series for 'The 2 Minute Tester'! We discussed maybe completing the book throughout the next batch of reading, as opposed to reading only another 7 chapters and then part 5 being just covering the remaining 5 chapters. As the chapters are short and concise, this may just be possible. Look out for part 4 coming next week! 

The chapters we discussed this week were a cause for more questions than discussions. Most of which we have mentioned below.


Chapter 15 - Running a Defect Board/Triage Meeting

This chapter summarises defect review boards (DRBs) and how they are used. 

Mentions: “…are used in both agile and waterfall projects.”

- In my experience as a tester, I have only used a defect board in waterfall projects after a test phase. In agile projects, I haven’t used a defect-specific board as a mid and senior level tester. I think in agile projects, this is something more likely to be done by a product owner or management to track open bugs and get them prioritised. The process I am familiar with in agile projects is, when a bug is found, report it and notify the product owner so that they can prioritise the fix vs their other deadlines/projects. We can of course advise on impact when reporting the bug too but the reviewing of the defects etc isn’t something I have experienced in the testing role throughout agile projects - but this book is aimed at testers.

- The key takeaways from this chapter are “rules for running an effective defect board…” many of which I agree with but one of them is “don’t have your DRB over 4 hours long”, for me even 1 hour in one of these things seem like a drag.

- I cannot say I am in favour of this approach, and prefer the agile approach I mentioned above. For this reason, some of the key takeaways/rules do not work for me, for example “Own them: make sure defects are assigned, either to the dev team or even better an individual.” I prefer to have the product owner prioritise the work. The defect should be reported well enough that any dev that has capacity can work on it, rather than dictating who should fix it.

Chapter 16 - Machine Learning in Testing: The Cyborgs are Coming…Trends

This chapter briefly describes what machine learning can be used for however, I would have liked to have read a general summary/introduction as to what machine learning is. Perhaps the author has assumed the reader already knows what this is, as he refers to it as “one of the buzzwords in testing”

- The bulk of the chapter is about 2 specific tools (Appliance and Functionize). It’s good in a sense that I have never heard of these tools nor have I used them but as mentioned above I’d like to have a brief introduction to what it is. Too much focus is put on the tools and it spills over into the key takeaways too (2 of the 3 being the websites for the tools)

- I agree with the author in that AI and machine learning are the buzzwords at the moment, and there are differences of opinion emerging in relation to how reliable it can be as a tool for testers. 

Chapter 17 - Soft Skills

This chapter introduces the reader to some soft skills and briefly explains how they can help one improve as a tester. Though I do feel these transfer over into any role or aspect of life in my opinion.

The author writes: “You can be the best tester in the world but if you can’t communicate in a clear and effective manner, you are not going to progress in the world.”

- I’m not sure how I feel about this statement. It seems like a contradiction - you cannot be a good tester, let alone the best in the world if you can’t communicate, period. Secondly, the author only lists clear and effective as being the measurement for progressing. In my experience, people can be clear with you but still rude. I don’t believe it’s such a black and white, clear subject.

- I like that the author emphasises the importance of soft skills and how they can enable one to work effectively and more importantly harmoniously 

Chapter 18 - Writing a Great Bug Report: Test Techniques

This chapter as you’d expect, provides tips on how to write a bug report.

- I agree that a tester must be able to write a good bug report. In my opinion, a bug report can range in length and detail, similar to what the author mentions also “A great bug report can be one paragraph.”

- I also agree with the 3 of the numerous principles listed for a good bug report; 1. Keep it neutral in tone, 2. Define the defect clearly and 3. Clear steps to reproduce the issue

- I like the principle “Don’t lead the witness’. I have never heard this term, but agreed that we should only suspect or suggest solutions and underlying causes when drafting bug reports.

- I would have liked to have seen something around communication listed. I’m not a fan of creating bugs and dropping them over to devs without you having at least spoke to them and confirmed it together. I feel it can be a bit cold, when devs just received tickets assigned to them without first ensuring you are on the right track. Of course, this doesn’t always happen but in my ideal working environment, it would.


Chapter 19 - Processes to Improve QA Processes

This chapter briefly mentions some processes that can help other processes, such as strategic approaches vs tactical approaches.

- The chapter also starts with, what I feel is an out-of-place Aristotle quote: “We are what we repeatedly do. Excellence then is not an act, but a habit”

- I can see how this applies to life and gaining new skills, but not work processes. In my opinion, habits can be a hindrance when working (ways of working that no longer help us but are still habits)

- The author references a book titled ‘Checklist Manifesto’ by Arul Gawande. I haven’t read this and will now be checking it out in the hope that it will help me manage my productivity.

- Moving on through the chapter, there is mention of the direction of QAT moving towards automation and DevOps - and the checklist manifesto is referenced in relation to how it can help us achieve this. Are we to accept and not challenge this direction? I would have liked to read the Author’s thoughts on this

- There is also reference to the failures in the aircraft industry during the 1940s and 50s to demonstrate the impact of using checklists. I have seen a few times now, that the tech world use the factories/auto mobile industries a lot when bringing in processes. Safety checklists for pilots are very different to checklists for daily/testing tasks in my book. The cause for the checklists being used by pilots has a specific reasoning and context behind it, and it seems that because they saw improvements in pilot safety, someone decided this will mean improvement in getting tasks completed. As mentioned, it seems that way to me, but could not be the case.

- Time-boxing each checklist item, as suggested in this chapter seems like an interesting approach and is not something I have tried.


Chapter 20 - Joining it Up and Looking for The Breaks: Integration

This chapter briefly discusses integration testing i.e testing to see how everything comes together.

- Again, in this chapter we are referencing back to the 1950s and rockets exploding. We are told that after a deep dive investigation they found the root cause of the events were because all of the components were tested individually and not altogether in the final build. I’d have to look into this. I don’t see how this information benefits the reader in relation to this topic at all. Again, a completely different time, reason and context for testing the way they did.

- The chapter mentions: “Just like massive engineering projects, in software development and QA, we don’t have to wait until all pieces are integrated before we test them”

- For me this statement has a few problems. Firstly, we’ve just read before this about how dangerous it can be when approaching things that way, but then we are told in this paragraph about the benefit of testing all things individually ie, the same way. Also, there’s clear separation between software development and QA in the quote above, but QA is part of and embedded into software development.

- I feel it’s always a good thing to try to remember how your piece of work you are testing, fits and works when it interacts with other parts of a system or application


Chapter 21 - Company Culture: Or Avoiding Rejection

This chapter seems to advise us on how to identify a good company culture.

- The chapter opens with what I feel, another misplaced quote: “Company cultures are like country cultures, Never try to change one. Try, instead, to work with what you’ve got.” - Peter Drucker

- When framing this quote within a work environment, I totally disagree with this. I believe that if a company’s culture is toxic for you, then you should not “work with what you’ve got” - sorry Peter.

- I like that assessing company culture has been given a chapter dedicated to the importance of it. Company culture can make or break someone in a role, in my experience.

- Again, as mentioned in a previous chapter, we are made to think the way someone dresses is in someway important at all

Mentions:
“One example I had was a QA engineer who was technically brilliant but dressed like he was going to the beach..”

- Why is the way he was dressed mentioned at all?.. The Author should at least tell us why he thinks this was a negative thing worthy of mention.

- It is also mentioned that this QA did not fit their tightly-controlled, agile way of working and should have adapted the way he worked and communicated in order to fit within it. Again, I strongly disagree with this approach. No one should adapt to fit into a culture that they feel is not good for them.


And with that, we end part 3 of our book review. Don't hesitate to share your thoughts with us! 

Until next time! 

Read More

Book Review Part 2: The 2 Minute Tester - Chapters 8-14

 

Part 2 of our book review series for 'The 2 Minute Tester' and this time around I am sharing my feedback and thoughts on chapters 8 through 14. Some of these chapters were a cause of some good discussions (thank you fellow book club members!) 

Chapter 8 - User Acceptance Testing (UAT): Test Techniques

- This chapter briefly defines UAT as ‘real users using the real system to run through real business processes’

Mentions: “..create UAT test cases..” And “…act as an enabler. You are enabling the business users to access the system” but then later in the key takeaways section mentions “be very clear in how you categorise any issues you find..”

- I’m confused - I have assumed based on the chapter that the end users do all the testing not us. This chapter implies that we just enable the UAT via setting up the system, providing logins etc. But then it specifically mentions we are the ones that found the issue and categorise, in the key takeaway.. Is this because the UAT team are reporting these directly to us? In my experience this hasn’t happened. They may report to us if they need help with certain aspects of their testing, but not for us to collate all their bugs, observations and enhancement requests. However, my experience with UAT is quite limited in this regard (reporting etc)

- The chapter also appears to contradict itself by defining UAT as having real users run through real business processes, but it’s the tester that is defining all of the business processes via test cases - in my experience end users always know their business processes and do not require test cases for those. Perhaps some high level charters will be shared with them, and then agreed - we can then help with the steps/scenarios

- The first page and half of the next is specifically for waterfall environments. Only a few lines mention how things might be different in an Agile environment - but could be applied to waterfall projects too so not to clear on the difference here other than having it included as part of definition of ‘done’ in a sprint.

- I agree with one of the main takeaways being 'act as an enabler/facilitator'

Chapter 9 - Test Team Leadership: Management

A chapter on leadership and what it requires.

Mentions: “Your role is to describe what good looks like, for example, ‘Defect-free release of system A in 8 weeks with 35 specific user flows fully functional and able to cope with 10,000 concurrent users’.”

- regarding this, if we never test it at all, technically it’s defect-free (because there'd be no defects raised)..Also 'defect-free' in my opinion is just not a measurement

- all of the above can be achieved by automation. Do we really need testers? Just automation engineers it seems

- that aside, I like that the example used for defining the description of good is specific, and avoids ambiguity (as this can be quite vague in teams at times)

Mentions: “In both agile and waterfall you need to resist the urge to micromanage..” but the example used when suggesting an alternative approach is quite patronising and sounds does very micro-managing-like “you have an awful lot of negative tests..”

- this also contradicts another one of the key takeaways listed “ask questions; don’t offer what you think the answer is immediately”. In the example above the author has already offered the answer of looking at commonality between the tests to reduce the number.

- I like that the chapter emphasises coaching and enabling over micro management. 


Chapter 10 - Dealing with Management or ‘Wait, what do you guys do again?’

This chapter does a pretty good job of offering ways testers (not just managers) can demonstrate the value they bring to a team and a project.

- I like that one of the key takeaways specifically mentions ‘..demonstrate, and not with powerpoint..’

I also liked that the author reminds us that “this is a problem you need to own and resolve” - but I’m not sure on the 'resolve' part of that. It can happen that testers demonstrate their value, but are still not valued by their team or specific team members (in my experience)

There are some good examples of ways to collaborate like mentioning 3-amigos sessions etc. There was mention of using the ABC macro which is ambiguity checking in requirements but I found this could be used in refinement sessions too and also we could look for not only the presence of words like 'may', 'might', 'could' and 'should' but we should also be aware of the absence of them too.

For example an absence of the word ’should’ could imply that we have either mistakenly assumed something ‘will’ do something or we haven’t explained how something ’should’ work (what we expect to happen)


Chapter 11 - SWOT and How it Can Help Your QA Delivery: Test Techniques

This chapter summarises the SWOT approach which is a 2 by 2 table showing strengths, weaknesses, opportunities and threats.

Generally I like that this chapter emphasises on talking to members of your team to get the full picture. The example SWOT table shown in the chapter was limited to only a QA team, but I think it could apply well to cross-functional teams too with input from each role.


Chapter 12 - Becoming a Test Manager (1): Management

After the first paragraph I feel I somewhat start to know the author as opposed to previous chapters which were your typical tips. I’d like to see more of the authors experiences shared. This was a good touch to the chapter for me. 

One of the key take aways listed is ‘Empathise, don’t tell.’ I enjoyed this short chapter. The author mentioned 3 references for working in management ‘communicate, set and see’. The small criticism I have is this is not really specific to management, and can apply in general for all levels of testers as we are often the bearers of bad news, it’s important we share this information in the correct way. 


Chapter 13 - Becoming a Test Manager (2): Management

This chapter should have been before the previous one in my opinion because unlike the previous, it touches on making the move into management and what you can do to get there.

In general I like this chapter too, there are some practical bits of information I can take with me, some books referenced and I generally agree with all of the key takeaways from the chapter.


Chapter 14 - GDPR: Data

This chapter highlights the importance of the GDPR data legislation and how we can go about navigating this in our roles. 

In my experience, companies usually have specific teams that handle this, and also generate some policies that employees have to abide by. Training is also provided but with that said, I’d have liked this chapter to have focussed more on the importance of data and anonymisation.

Data legislation and policies around this are very important for testers - so I am glad the topic of data has a dedicated chapter in this book. There are some good points raised around using production data in testing and how potential breaches lay in wait.

I did however disagree with one of the key takeaways which is simply “Do a data audit”. This could be something a tester could initiate or facilitate but doing the audit themselves - I’m not sure that is best/good practice as in my experience, as mentioned above it's always been a third party doing the auditing.

Read More

Book Review: The 2 Minute Tester - Chapters 1-7

 

The 2 Minute Tester by David Bruce is a blend of soft and hard skills, advice and tips spread throughout 33 chapters. The review for this book will be split in a series of posts, as and when the specific chapters are covered in our book club. The questions and comments I have on each chapter are cause for some great discussions and my comments are not always correct. In fact, I was reading back through, and it seems in the beginning of the comment I appear to be unsure of a thing, but then later convince myself that I actually agree. Take my review as you will - and I certainly hope it doesn't discourage you from reading.

Introduction pages:

At the beginning, in the “Who this book is for” section, it mentions:

“..where assurance is key..” And “…assurance requires a deep understanding of techniques…”

• I disagree that assurance is solely the tester’s responsibility, despite the common title of “quality assurance engineer”

Chapter 1 - Exploratory Testing

This chapter generally summarises exploratory testing and the benefits of using the approach

mentions:

“you don’t have much time..”
And “exploratory testing is a useful technique when there isn’t much time available, you need the system tested quickly”

• At first I didn’t think the time issue was relevant to exploratory testing, but upon reflection if I didn’t know a system and someone wanted their product shipped asap, I’d first ask - why am I testing this now? at the point where I have not much time and more importantly, why do I not know the system? (if working on a project)
  
  Questions aside, if you want to get something tested quickly and you do not know the system/application of how it works - exploratory is the natural approach to take. So agree here.

• I found it interesting that point 3, mentions “pre-investigation” and speaking with the team, under the chapter of exploratory testing. This is often overlooked.
  
  

Mentions:
“is it often most effective either at the beginning or end of a specific test phase..” And “…if operating in an agile manner, at the end of each sprint”

• I’m not sure I agree with this. Why would exploratory testing be most effective at the end of a sprint or test phase? Exploratory testing is about learning, ideally we’d want to carry this out as soon as the product is ready to test (and also via discussions before it is built)

Chapter 2 - Effective Team Working in QAT

This chapter is a brief summary of how to effectively work as a QA team.

Mentions:

“effective team working” and “..what makes an effective QAT team?..” 

• Seems to imply the QA team are separate from the rest of the dev/engineering team. Personally I believe when you make QA a separate team, this is quite ineffective. I prefer the Agile approach where you have cross functional teams (QA, dev, product owner etc all in one team together collaborating) 

Mentions:

“stop thinking of devs as people whose work you are trying to find fault with and start with the view that you are helping them deliver the best solution possible”

- how about both? I am aiming to find problems in the work - and help them deliver the best solution possible. Just be kind and compassionate as you should be - don’t treat people bad, period. 

Chapter 3 - The Tester Mindset: 80/20

This chapter explains that 80% of our efforts should be focussed on edge cases and 20% of our efforts on "happy paths"

Mentions:

“As your suite of tests increase (if you are testing in an iterative, agile manner) more and more of your negative tests should be automated”

- 1. Why is this restricted to agile? Waterfall projects also have test suites build up over time, from which you can automate the negative tests

- 2. I don’t have experience of working with test suites on agile projects, but we did trial X-ray and the test case approach didn’t work for us

Chapter 4 - Risk-based Testing: Test Techniques


This chapter briefly explains what risk-based testing is, how and when it is used and the benefits of the technique.

- The author tells us that risk-based testing is when “…you need to test the most-important, or riskiest parts of the solution”

- Personally I’m not sure where I stand in relation to assuming here that ‘most-important’ and ‘riskiest’ are the same thing. On a pension application for example, the most important thing could be simply seeing what the current valuations of their savings are, a value feeding through to the front end - but most riskiest could be performance. What happens when hundreds of thousands of customers do this at the beginning of the month, after they’ve been paid by their employers and they login to see the state of their pension?

I incline to believe they are the same, because important can be the riskiest, depending on who you are referring to - your customer? Or your in-house architecture team?

- I like the 3 parts mentioned for defining most-important, or riskiest:

- key functionalities
- Likelihood of failure
- Impact of failure

- I also like that we are reminded a lot of the information above can be found through discussions with subject matter experts. I’d have liked to have seen this mentioned as a key takeaway at the end of the chapter

- In the section for “Impact of failure” I liked the example used for purchasing something as a gift on an online store and hiding the price of the gift for the receiver

- One of the key takeaways from this chapter that are mentioned is “focus on key functionality, less so on edge cases.” Usually in projects, a lot of the key functionality is usually automated and monitored. But even if they are not, I believe when looking at the most-important parts of a system, spending a good amount of time thinking and exploring edge cases is required, and in fact often naturally performed by the tester due to the importance/risk of the part being tested

Chapter 5 - Static Testing: Test Techniques


This chapter appears to assume that static testing is performed on requirements documents.

- We are told that great benefit of this technique is that it is done very early in the development cycle, but then in the next paragraph tells us in relation to tools to use for this technique, “In terms of tools it is hard to beat the human eye for static testing.” The author seems to assume Testers cannot get involved even earlier, before any documentation is drafted throughout this chapter.

- I have found working with Product Owners that present their ideas/intentions for building something before drafting documents up as gospel, to be of great benefit.

- Again the bulk of this chapter is advising us how to perform static testing on documents using search functionalities however I liked that the author suggests we look out for terms such as should, may, could, possibly - the author reduces this to searching for these in documents, but we should always take note when such terms are used in discussions around the product in general too.

- I’m left pondering on whether I need to go back to my “ISTQB Foundations” material, to verify whether static testing, is only to be done on documentation? (As the chapter suggests)

Chapter 6 - Building Your Own Avengers Team: Recruitment


This chapter is a brief summary of how to build a diverse team.

- First immediate thought was, is the author testing us Marvel fans by telling us Thanos is part of Avengers “team”?! Using an example to demonstrate the strength of diversity, the author writes: “..it’s hard to think of a more diverse team than Black Widow, Thanos and Thor, for example” 

- On how to build a great diverse team, the author mentions: “You hire different from yourself…if they look like you, then you don’t have a diverse team.” That could be true, but in Agile environments there is typically no separate QA team. So if you look around, and everyone doesn’t look like you, but they have the same role and perform the same job as you - is that diversity? In my opinion, the cross-functional team approach is a great example of diversity - you have many diverse opinions on one subject all viewing it from the lens of their role in the team. This can strengthen our testing, and help us consider more variables when testing too.

- It seems the author isn’t sure if by diverse he means the way people look, or diverse in relation to skillsets and approaches. The author mentions how he had an interviewed someone wearing a flowery Hawaiian shirt, shorts and sandals - but ended up being hired, based on his passion and background working in startups.. no mention of skillset or presenting a diverse opinion or approach here. 

- In my humble opinion, a diverse team is more about the shared ideas, approaches and opinions and consideration for others when sharing those. For example, we could hire an automation engineer that believes all the testers are beneath them because they are doing the work of monkeys manually tapping and clicking away. So here, we have some diversity in opinions/skills and approach, but does that now mean we have a great team? 

Chapter 7 - Specification by Example, or True Living Documentation: Test Techniques


This chapter refers to a test technique called 'specification by example' but doesn't do a great job of explaining what it is. The lack clarity here though is my problem and I'll be reading up on this matter.

- Admittedly I don’t know much about Gojko Adjic or his concept of SBE - and that’s a good thing because now I will go and read up and learn more (one of the key takeaways listed at the end of the chapter too)

- UPDATE: after reading up about it, I initially thought this was the gherkin/BDD approach to things (which is how I interpreted it from the book) - but there does seem to be a subtle difference. Still slightly confused. With that said though I’m all for removing ambiguities and having some clear (as possible) requirements.

- The chapter is a little confusing to me, but that’s my problem

- One of the other key takeaways at the end of the chapter mentions: “Remember it isn’t just QA, it covers development, business analysis and project management as well” - but in my experience this is the same for your typical set of requirements also. We all interpret the document in our own way, and frequently revisit (sometimes with customers) to ensure we are on the right track. Not sure why this is mentioned as if it is specific to the ‘specification by example’ approach

To be continued...

Read More

My Testing Journey Presentation - Hosted by QA Beginners Club

 

The amazing team over at QA Beginners Club were kind enough to let me share my testing journey. 

I spoke about what I was doing before testing, making the transition into it and where I am today.

I also touched on the highs, lows and the many lessons learned along the way. 

You can check it out via the Youtube link below! They also have more great content too, so be sure to subscribe.

Thank you Chris and everyone else at QA Beginners Club. 

https://www.youtube.com/watch?v=iKRu6pB_TC8

Read More

Testing - A Jackie Chan Approach

 
“What do you mean you do not accept my bug?!” *Flying kick*

No - not that!

In this post, I will highlight some of the important, sometimes forgotten aspects of testing via some observations I made from watching possibly every Jackie Chan movie ever made over the years (kung fu movie fan here) - hence the ‘Jackie Chan Approach.’

Jackie Chan is an absolute master at martial arts performance and arguably the greatest stuntman to appear on film. Known for his unorthodox approach to martial arts action, I believe we can take some important reminders from Sensei Jackie.

It is my belief, that Jackie Chan would be the greatest tester ever, had he ever tested software for a living. I know, I know, but hear me out…

Firstly, **Disclaimer - this post is not to condone any acts of violence**

And so we begin. Martial arts movies typically follow the standard plot formula. Something like:

if bad.thing happens to main.character
    and main.character seeks revenge

        using.KungFuPowers
then movie.genre = 'martial arts'
end

Putting my poor attempt at coding aside, the Jackie Chan movies follow the same structure but there are two things that make the Jackie Chan movies distinctly relatable to our job as Testers.


Establishing the Mission

In our role, like the movie genre above, we want to start out by establishing our mission.
Has this product upset us in some way, or someone of value to us? A client, or end user perhaps? ‘Upset’ here, meaning we have experienced a problem of some kind.

In this case, we would probably be addressing a particular bug or pain point of the product found by somebody else. Not a great place to be in, as ideally we would have found this problem during testing, before our person of value did, however it is a very real possibility and a most certain reality that will occur at some point in our careers.

When addressing problems found by others, our mission will consider these factors and how we embark on that mission will be a great learning experience. Our mission in this case could be:

Verify the conditions of the problem and try to ensure that under the same circumstances, this bug no longer occurs.

Taking a more general approach, as Testers our mission should be:

Find potential and actual problems

This general approach can encompass risk-storming at the beginning of a project, all through the development cycle up to and after delivery of the project. All of our testing activities can happen in the context of this mission. For example, When refining or building requirements for it, how can we find potential problems?, what are the risks?, in what ways is it testable? Are we sure we have understood our clients? How would things look and playout in the worst case scenario, if everything goes wrong?

In trying to get some answers for those questions, we are achieving our mission.

Unfamiliar Places

In ‘Rush Hour’ and ‘Rumble in the Bronx’, Jacke Chan plays a detective arriving in the USA from his home country China. Whilst in pursuit of his mission, he is forced to get familiar with the unfamiliar very quickly, adapting to new places, new relationships and new situations.

This is very similar in testing. We are regularly faced with the unfamiliar and are forced to work in these new contexts, otherwise our job, and the quality of which it is performed suffers greatly. We are met with new features and enhancements, and to help us make sense of these, we have to build relationships with new people and teams, often speaking with people we haven’t met before, again - with the aim of achieving our mission.

Naturally, in unfamiliar situations there is a lot of exploring and there are many questions asked. The answers we get for those questions help us ask more specific questions until we start to get a more clear picture and we can then begin to build a story around our mission. Much like Jackie did in those movies.

Challenges Along the Way

Here is where I find one of the two things that make Jackie Chan unique in his movies, which I believe can relate to testing.

Typically in a Jackie Chan flick, the obstacles that stand in the way of him and his mission take the form of ‘bad guys’. You know the type, angry facial expressions, speaking in grunts and not the most friendly of people. Combat ensues and this is where Jackie becomes the ‘greatest tester that never tested software’ in my books.

The obstacle has presented itself, in the form of a group of bad guys but lo’ and behold, there’s a ladder in the room. We know what ladders are for, but being the creative person Jackie is, he decides to use this ladder as a weapon! And ladders aren’t the only victim of his innovations. For example, one viewer online noted, that in just one movie there was use of:

- A belt

- A pool cleaning net

- A bicycle

- Some chairs

- A pot of spaghetti (flung by his feet onto head of assailant holding him from behind)

- Some plates (flung Frisbee style)

Now I haven’t ever used any of the above tools to test a product but I thought to myself, here is someone that will use every possible thing available in order to address any challenges - utilising tools that were not intended for use in a martial arts encounter.

I relate this to testing because I believe we should have the same approach in our roles. I want to find and use any tool possible that will help me succeed in my mission. We should be testing beyond requirements and acceptance criterias. Communication is a great tool. I once met with a team to get some valuable data regarding a project I was working on. The team never had a tester speak to them prior to that. Like Jackie, I want to take a look around and see whatever is there that can be utilised. This can span from meeting with other colleagues and teams to get valuable information, reading through documentation to using tools that perform specific functions on our product under test (and much more depending on your context, and more importantly whether you care to look!)

No Replacements

It is well known that Jackie Chan performs his own stunts in his movies and certainly pays the price for doing so. Over the years he has broken many bones and his body has taken a real beating. This brings me on to the second thing I believe we can apply to testing.

Jackie Chan, like every other actor has the option of brining a stunt man in to perform the stunt on his behalf - but that’s not Jackie. You watch a Jackie Chan movie, in anticipation of what great stunts will perform, and then for the 20 minutes of bloopers that playout at the end as the credits roll.

As the tech world rapidly evolves, there are now many tools on the market that we can use as testers - but these are exactly that, ‘tools.’ The tools help us as we conduct our testing. We use tools that can automate things, generate data for us, extract, upload, report and much more but we should never want to replace the importance of a skilled tester actually testing. Jackie Chan doesn’t want a stunt double, why should we want someone to ‘act’ as a tester on our part?.. If Jackie Chan needed to fly over a castle wall, then perhaps he might need some CGI tool to help with that. Likewise if we needed a hundred thousand-odd records of enduser data created in our database, we too might look for a tool to help us do that - but that tool isn’t the one testing - just like that wouldn’t be Jackie flying over that castle wall!

Mission Accomplished

And so, those are my observations. What I call the ‘Jackie Chan Approach’ - establish your mission, utilise all things possible that could (even unexpectedly) aid you in achieving your mission and replace your efforts with tools when reasonable, never seek to replace testing itself.

Roll credits! 

I hope you enjoyed this post, and if not at least benefitted somewhat. Please do check out my other posts on the topic of Testing. Regards, James

Read More