Have We Lowered Our Standards On Quality?

This post was inspired by two comments I read online. The comments were left under a video related to the recent problem present on the latest release of the Kindle Color Soft. Thousands of customers have been reporting a strange yellow discoloration on the screen of their new devices, so I headed to Youtube to check if anyone had reported these findings in any product reviews or unboxing videos. The entire first page of search results made some mention of this yellow band defect and the first comment that struck me was "How did they get this through QA?" and under another video review a comment read "Yeah, that's why I never buy products when they first release." 

The comments above made me reflect upon the current state of products and software in the marketplace. It seems to me that companies are not too bothered about releasing their products into the market without addressing problems that are often so apparent, customers like myself wonder how they ever got the green light for release in the first place. Unfortunately for customers, things have been this way for a while.

The first case of botched first releases I thought of was the infamous 'Bendgate' incident following the release iPhone 6 and 6 Plus release in 2014. Customers at the time were reporting that the phone was bending under pressure - sadly for Apple, the skinny jean fashion trend was in full force, which meant skin-tight 'pressure' around the pocket region where the phone would be kept. iPhones bent, so your legs didn't have to (not that they could in those jeans any way). After investigation, in September 2018 it was revealed that Apple knew about the structural flaw in the device all along.

Skipping forward to 2016, Samsung released the Note 7. By the end of that same year they recalling all devices after reports of batteries in the devices overheating and even exploding. Not only was this a huge safety concern for customers, the recall itself also cost the business a reported $5.3 billion! As well as being hugely damaging their reputation. 

And what about software? I've been so kind as to spare you, the reader, of having to read through a long list of all the computer games and upgrades in the past few years that have been released with countless bugs in them, requiring many companies to make public apologies and even pay out refunds. To add, I've also excluded the numerous problems I, and others have reported (and keep finding!) when attempting to perform basic tasks in the day-to-day applications. 

So here we are approaching 2025 and it seems we haven't moved forward much, which brings me to the recent Kindle Color Soft. Released on the 30th October it was a promising move for Amazon. There are very few companies that have released color e-ink reading devices and I must say, as an avid reader they have done great things in pushing the technology and market forward. Customers were now waiting in anticipation for their Kindle devices to arrive - finally, everything we love about Kindles (simple UI, great library, usability etc) would now have color?! Count me in..

Let's just ignore the price of the device for now, that can be argued at another time - or perhaps in the comments? .. When the Kindle Color Soft was first announced, I already knew that I would not get the first batch of the release. The tester in me said let's observe some behaviour and outcomes and time box it at 4 days tops, if all was good I'd go purchase. Why did I have this approach? Subconsciously I expected some problems in the initial release - and just totally accepted it as normal. And it appears it has indeed become the norm! 

Customers that were brave enough (or just, expecting a working product as normal) to purchase, started reporting a problem with their new devices. A yellow band at the bottom of the reader - and within a few days customers had shared their updates on Youtube, Reddit and other forums mentioning that the customer service teams had instructed them to send them back and they'd immediately get a replacement sent out to them once fixed and back in stock, or a refund transferred. 

I then went to the Amazon product page for the Kindle colorsoft, only to find that yes, you still can buy it but don't expect to receive it for another 3-6 months?! (now in the UK that time has reduced to 6-8 weeks)

So going back to those questions that inspired this post. 

Was this issue captured in QA/testing? Was the product under test at any point at all? 

I Googled 'Does Kindle have a QA department?' - the results were old job Vacancies for QA Engineer and QA manager roles. So we'll assume in this case that they do. 
On that basis, I will assume my fellow testers did indeed capture such an obvious flaw, which opens the door to other questions like:

Do these QA Engineers actually test? or are they expected to be 'Quality Coaches', 'QA Motivators', 'QA Coordinators' etc, the list goes on - basically focusing so much on quality (an endless path) that no actual testing gets done. 
Or perhaps these issues were indeed reported, but not communicated effectively? Or they were communicated effectively but to a Manager that was up against some tight deadlines that viewed testing as the final chunk of development, with no problem at all slicing that chunk smaller and smaller if development/coding takes a little longer. 

There are so many layers to problems that are found in products but a few things are for certain, according to me:

1. Customers are increasingly accepting the fact that first releases of products are going to be bug-ridden, what this means for businesses is, you actually might have a great product but due to lack of care for customers, your product will be affected by this and may even see lower volumes of sales for your first release, until at least the good reviews start coming in.

2. The doors of criticism for your product are many and they hold a lot of weight. Product reviews really matter - and nowadays they come with pictures attached or even video reviews containing evidence of their claims too so as to dispel any doubts about the customers' critique. As of this afternoon 11th Nov 2024, the Kindle Color Soft is rated 2.4 out of 5 stars, with 131 ratings.
Pay attention to product reviews. If you are a tester or a QA professional, treat it as test data. 

3. Businesses need to listen to their testers. Go ahead and hire a Quality Coach, Quality Engineer, Quality Assurance team - they can do great things in teams and facilitate some great discussions but one thing is for sure and that is, you need somebody on your team, skilled enough, actively looking for problems in your product and you want to know what those problems are before the customers and market do.

4. There is an environmental concern around product releases and that is, where will these thousands, if not millions of product recalls end up? Recycled we hope. Not stuck in the teeth of some poor whale in the ocean. 

5. As customers we should simply not expect this and in fact, I have vowed to make a conscience effort in actually purchasing the next product I want upon it's initial release. And if riddled with obvious, apparent, easily found bugs or defects - I will be publicly reviewing /even blogging about this as I think this could be a means of change - though a tiny spec in the grand scheme of things. We should keep our expectations high and not simply expect things to be faulty upon first release.

PS. This was part random rant, part random thought dump. Thank you for taking your time to read.

Meanwhile if you have an e-reader to recommend, I am indeed hoping to buy one - birthday is fast approaching people! *nudge *nudge *wink *wink

Read More

My Testing Journey Presentation - Hosted by QA Beginners Club

 

The amazing team over at QA Beginners Club were kind enough to let me share my testing journey. 

I spoke about what I was doing before testing, making the transition into it and where I am today.

I also touched on the highs, lows and the many lessons learned along the way. 

You can check it out via the Youtube link below! They also have more great content too, so be sure to subscribe.

Thank you Chris and everyone else at QA Beginners Club. 

https://www.youtube.com/watch?v=iKRu6pB_TC8

Read More

Testing - A Jackie Chan Approach

 
“What do you mean you do not accept my bug?!” *Flying kick*

No - not that!

In this post, I will highlight some of the important, sometimes forgotten aspects of testing via some observations I made from watching possibly every Jackie Chan movie ever made over the years (kung fu movie fan here) - hence the ‘Jackie Chan Approach.’

Jackie Chan is an absolute master at martial arts performance and arguably the greatest stuntman to appear on film. Known for his unorthodox approach to martial arts action, I believe we can take some important reminders from Sensei Jackie.

It is my belief, that Jackie Chan would be the greatest tester ever, had he ever tested software for a living. I know, I know, but hear me out…

Firstly, **Disclaimer - this post is not to condone any acts of violence**

And so we begin. Martial arts movies typically follow the standard plot formula. Something like:

if bad.thing happens to main.character
    and main.character seeks revenge

        using.KungFuPowers
then movie.genre = 'martial arts'
end

Putting my poor attempt at coding aside, the Jackie Chan movies follow the same structure but there are two things that make the Jackie Chan movies distinctly relatable to our job as Testers.


Establishing the Mission

In our role, like the movie genre above, we want to start out by establishing our mission.
Has this product upset us in some way, or someone of value to us? A client, or end user perhaps? ‘Upset’ here, meaning we have experienced a problem of some kind.

In this case, we would probably be addressing a particular bug or pain point of the product found by somebody else. Not a great place to be in, as ideally we would have found this problem during testing, before our person of value did, however it is a very real possibility and a most certain reality that will occur at some point in our careers.

When addressing problems found by others, our mission will consider these factors and how we embark on that mission will be a great learning experience. Our mission in this case could be:

Verify the conditions of the problem and try to ensure that under the same circumstances, this bug no longer occurs.

Taking a more general approach, as Testers our mission should be:

Find potential and actual problems

This general approach can encompass risk-storming at the beginning of a project, all through the development cycle up to and after delivery of the project. All of our testing activities can happen in the context of this mission. For example, When refining or building requirements for it, how can we find potential problems?, what are the risks?, in what ways is it testable? Are we sure we have understood our clients? How would things look and playout in the worst case scenario, if everything goes wrong?

In trying to get some answers for those questions, we are achieving our mission.

Unfamiliar Places

In ‘Rush Hour’ and ‘Rumble in the Bronx’, Jacke Chan plays a detective arriving in the USA from his home country China. Whilst in pursuit of his mission, he is forced to get familiar with the unfamiliar very quickly, adapting to new places, new relationships and new situations.

This is very similar in testing. We are regularly faced with the unfamiliar and are forced to work in these new contexts, otherwise our job, and the quality of which it is performed suffers greatly. We are met with new features and enhancements, and to help us make sense of these, we have to build relationships with new people and teams, often speaking with people we haven’t met before, again - with the aim of achieving our mission.

Naturally, in unfamiliar situations there is a lot of exploring and there are many questions asked. The answers we get for those questions help us ask more specific questions until we start to get a more clear picture and we can then begin to build a story around our mission. Much like Jackie did in those movies.

Challenges Along the Way

Here is where I find one of the two things that make Jackie Chan unique in his movies, which I believe can relate to testing.

Typically in a Jackie Chan flick, the obstacles that stand in the way of him and his mission take the form of ‘bad guys’. You know the type, angry facial expressions, speaking in grunts and not the most friendly of people. Combat ensues and this is where Jackie becomes the ‘greatest tester that never tested software’ in my books.

The obstacle has presented itself, in the form of a group of bad guys but lo’ and behold, there’s a ladder in the room. We know what ladders are for, but being the creative person Jackie is, he decides to use this ladder as a weapon! And ladders aren’t the only victim of his innovations. For example, one viewer online noted, that in just one movie there was use of:

- A belt

- A pool cleaning net

- A bicycle

- Some chairs

- A pot of spaghetti (flung by his feet onto head of assailant holding him from behind)

- Some plates (flung Frisbee style)

Now I haven’t ever used any of the above tools to test a product but I thought to myself, here is someone that will use every possible thing available in order to address any challenges - utilising tools that were not intended for use in a martial arts encounter.

I relate this to testing because I believe we should have the same approach in our roles. I want to find and use any tool possible that will help me succeed in my mission. We should be testing beyond requirements and acceptance criterias. Communication is a great tool. I once met with a team to get some valuable data regarding a project I was working on. The team never had a tester speak to them prior to that. Like Jackie, I want to take a look around and see whatever is there that can be utilised. This can span from meeting with other colleagues and teams to get valuable information, reading through documentation to using tools that perform specific functions on our product under test (and much more depending on your context, and more importantly whether you care to look!)

No Replacements

It is well known that Jackie Chan performs his own stunts in his movies and certainly pays the price for doing so. Over the years he has broken many bones and his body has taken a real beating. This brings me on to the second thing I believe we can apply to testing.

Jackie Chan, like every other actor has the option of brining a stunt man in to perform the stunt on his behalf - but that’s not Jackie. You watch a Jackie Chan movie, in anticipation of what great stunts will perform, and then for the 20 minutes of bloopers that playout at the end as the credits roll.

As the tech world rapidly evolves, there are now many tools on the market that we can use as testers - but these are exactly that, ‘tools.’ The tools help us as we conduct our testing. We use tools that can automate things, generate data for us, extract, upload, report and much more but we should never want to replace the importance of a skilled tester actually testing. Jackie Chan doesn’t want a stunt double, why should we want someone to ‘act’ as a tester on our part?.. If Jackie Chan needed to fly over a castle wall, then perhaps he might need some CGI tool to help with that. Likewise if we needed a hundred thousand-odd records of enduser data created in our database, we too might look for a tool to help us do that - but that tool isn’t the one testing - just like that wouldn’t be Jackie flying over that castle wall!

Mission Accomplished

And so, those are my observations. What I call the ‘Jackie Chan Approach’ - establish your mission, utilise all things possible that could (even unexpectedly) aid you in achieving your mission and replace your efforts with tools when reasonable, never seek to replace testing itself.

Roll credits! 

I hope you enjoyed this post, and if not at least benefitted somewhat. Please do check out my other posts on the topic of Testing. Regards, James

Read More

Retro Notes: Reflections on 2021

 

In this post I will be reflecting on the previous year and in typical 'retro' fashion I will explore what went well, what didn't go so well and will also address some actions I have decided to take going into the new year.

What went well? 

Towards the end of 2021 I was promoted to a Senior position and immediately following my promotion I noticed that there are many skills I need to work on in order to do the best job I can in such a position. My time management skills for example had a big shake up as the work load increased. So any event/situation that sparks some reflection I do view as important. 

Secondly, 2021 was a year in which I had conducted many interviews with various candidates and got to interact with many ideas. I have changed some of my stances of various activities in the workplace including some prominent testing practices. For example, I used to favour certain opinions such as 'no need for test cases' as a best practice. But I now think it depends on the context. If you are in a company which operates in an industry with many legal aspects and regulations, you might be required to sign those off before even testing - so in examples like this, having test cases is probably a better idea. 

Another opinion I held which I challenged, was Testers decide when things are ready to ship. I used to think that though we do get involved from the very beginning, we as Testers are the ones who test the feature against the requirements and then decide whether it should be shipped/merged. However, upon reflection and a few discussions with other testers and colleagues, this should be something done by the person who owns the project, either Project Manager or Product Owner. My view became more apparent with tickets that work as they should when measuring against the acceptance criteria but certain pathways that were not covered, were not working. so in such cases we have tickets where the acceptance criteria is met and working fine, core flows are working fine but funky edge cases after some exploration are causing errors. In my opinion these tickets were not good to go, but our Product Owner disagreed, accepted the risk and asked us to create additional tickets to fix those other errors. At first it was quite a shock, but I now agree as Testers we do not hold such authority - which is actually a good thing in my opinion. 

Lastly, I also realised the true importance of context both in testing and with anything in general, before populating opinions or drafting actions. To summarise, 2021 was a year of true reflection in regards to my career, what I am doing and where I want to go. Though these are questions I often ask myself I feel like last year things began to get a lot more clearer for me.

What didn't go well?

As you can probably tell by the nature of the content in the cards above, all of those cards could be down to one factor which is time management. I've didn't mind working from home when the pandemic first happened. It's definitely the more safer thing to do in my opinion but after having my second child it can get quite difficult just juggling everything in general. I'm still adapting. I noticed my passions and activities shifted and my life revolves around my kids now. Parents will probably relate to this more but it seems like my life has taken a big shift away from what I once felt were REALLY important. Now they are just about important. I think due to this shift I haven't put as much effort into self-learning, and when I have done I just get side tracked with more life stuff. I set myself quarterly objectives all last year but didn't achieve a single one and I didn't want to beat myself up about it either. I do however think that I really need to develop my task/time management as well as develop habits as opposed to doing things in the spur of the moment when some inspiration comes. All work in progress. 

Actions

So far the only action I have come up with, which is realistic and achievable is to draft my blog posts before writing them up in the hope that this blog will at some point benefit as I really want to give back. I have taken on an 'apprentice' outside of work, who wants a change of career and I think rehashing a lot of the stuff they have to learn will serve as motivation for some new posts. 

So I go into 2022 not on a high, but feeling pretty good about the year to come and I'm glad I was able to reflect on where I can make improvements moving forward. 

I'd like to just use the rest of this post to thank the amazing testing community. Though I may not speak with you all regularly I get immense inspiration when I go through my LinkedIn and Twitter feeds. 

Thank you all, and I wish you all the success in 2022 and the coming years! 

James

Read More