Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll

Introduction to QA Engineering

Well done!

You have completed Introduction to QA Engineering!

Sign up for Treehouse Back to Library
Preview
Video Player
00:00
00:00
00:00
  1. 2x 2x
  2. 1.75x 1.75x
  3. 1.5x 1.5x
  4. 1.25x 1.25x
  5. 1.1x 1.1x
  6. 1x 1x
  7. 0.75x 0.75x
  8. 0.5x 0.5x
Use Up/Down Arrow keys to increase or decrease volume.

Reporting Quality

5:20 with Ryan Saul

One of the most important parts of quality assurance is collecting lots of data and reporting that back to your company for decision making.

One of the most important parts of quality assurance is collecting lots of data, and 0:00
reporting that back to your company for decision making. 0:05
QA is a very data-driven role. 0:09
Ask yourself the following questions about where your product is right now. 0:12
[SOUND] Do we need more tests? 0:16
[SOUND] Are we catching most of the bugs that we could be? 0:19
[SOUND] What do our customers think of our product? 0:22
Are they happy with it? 0:25
These are important things to answer. 0:27
You need to let your company know how the software is performing 0:29
out in the real world, or if there's too many bugs. 0:32
Or if something isn't totally ready for release, so 0:36
that the company can make decisions based around that. 0:38
In this video we'll go over some easy reports that can help you do that and 0:42
answer those questions. 0:46
So the first report we'll cover is the coverage report. 0:48
This is the simple overview of your testing efforts. 0:51
The idea here is to show what tests are covering what parts of your application, 0:55
and how much more testing is needed to cover that gap. 1:00
So let's say we have an application that has three main parts. 1:03
Login, or authentication, creating cards on a board, and printing out those cards. 1:06
Currently, we have two tests for authentication, another four tests for 1:12
creating cards on the board, and we don't have any tests for printing out the cards. 1:16
This tells us that there's a lack of testing happening in one part of the app 1:21
and that we need to write some new tests before we can feel confident in it. 1:25
Another thing that the coverage report can show is where you're automation efforts 1:29
should be going. 1:32
If you track how long tests take to run, 1:34
then this will be a very easy decision to make. 1:36
The longest tests should probably be automated first 1:38
because they save you a lot of time. 1:42
But if you don't have that kind of data, 1:44
look at automating areas that require a lot of testing to complete. 1:46
This is a simple way to illustrate that data can be very powerful in helping 1:50
to make decisions. 1:55
Both for your business and for your own QA efforts. 1:56
You always have limited time. 2:00
So it's good to know where to spend it. 2:01
So now that the software has been released some customers are reporting bugs. 2:04
This is bad. 2:09
And we need to know about these bugs as part of the QA team. 2:10
The difference between you finding a bug before the software is released and 2:14
a customer finding a bug is huge. 2:18
Keep a tally of how many bugs are being found by the customers each month. 2:21
Log them just like bugs that are found in development. 2:26
These are escaped bugs that have found their way into the released product. 2:30
Fixing these is much more costly than fixing them in the key way process. 2:34
We want to put these bugs into an escaped bugs report 2:39
to show to the rest of the team. 2:42
This report basically tells you if you have some areas in the development or 2:45
testing process that need to be worked on. 2:49
This report will also help justify putting more investment into that. 2:52
We want to make customers feel more confident in their product 2:57
each time that there is a new release. 3:01
Further more, every time a bug gets released or 3:03
out into the wild, it costs much more than it would to fix it before it was released. 3:06
Finally, another simple report is gathering user feedback. 3:12
If you're unfamiliar with the Agile manifesto, 3:16
I recommend checking out the Treehouse course on Scrum basics which goes over it. 3:19
But a very important aspect of Agile is user feedback. 3:25
How does customer feedback fit into QA, though? 3:29
Basically this will inform if you are looking for 3:33
the right things in the software when it is being tested. 3:35
How simple are the workflows? 3:39
Are the buttons confusing? 3:42
Is the documentation helpful? 3:43
These are the types of things you can get from user feedback. 3:46
This is just a simple report that I put together with all 3:50
the feedback that we've gotten back from our customers on the RSVP app so far. 3:53
We aggregate the positive and negative feedback as well as other feedback for 3:58
those that didn't specify. 4:03
We also take in all of the feedback and 4:05
calculate the satisfied percentage by the amount of users giving 4:07
positive feedback divided by everyone else who has given feedback. 4:11
Your way of calculating this number may be a little different, and 4:15
you may not even really need this number for your purposes. 4:19
But I feel like it's a really nice way to show 4:22
how many users are satisfied by your product. 4:24
You can solicit customer feedback from a support or sales team. 4:28
What are your customers saying about the software? 4:33
This can be tricky. 4:36
When generating a user feedback report, 4:38
you wanna gather all the feedback you can get. 4:40
And look for patterns in what the customers are saying. 4:43
This isn't an easy task though. 4:47
It's often done by a product team, so 4:50
you might be able to get that sorta data from them. 4:52
In QA, we wanna know if our tests are looking for the right things and 4:56
if we need to revisit old tests. 4:59
This is why user feedback is great. 5:02
I really just want to go over some simple reports that you can keep 5:05
in your back pocket. 5:08
In reality, just about any piece of data you're collecting, 5:10
could probably find it's way into a useful report. 5:13
So, be on look out for things your company may find useful. 5:17