Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
One of the most important parts of quality assurance is collecting lots of data and reporting that back to your company for decision making.
Questions:
- Do we need more tests?
- Are we catching most of the bugs that we could be?
- What do our customers think of our product? Are they happy with it?
Reports:
- Coverage report: How many tests belong to different parts of the software?
- Escaped bug report: A report of all bugs that have been reported by customers and not internally.
- User feedback report: What are customers saying about the software? Look for trends in feedback.
Related Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign upRelated Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign up
One of the most important parts of quality
assurance is collecting lots of data, and
0:00
reporting that back to your company for
decision making.
0:05
QA is a very data-driven role.
0:09
Ask yourself the following questions
about where your product is right now.
0:12
[SOUND] Do we need more tests?
0:16
[SOUND] Are we catching most
of the bugs that we could be?
0:19
[SOUND] What do our customers
think of our product?
0:22
Are they happy with it?
0:25
These are important things to answer.
0:27
You need to let your company know
how the software is performing
0:29
out in the real world, or
if there's too many bugs.
0:32
Or if something isn't totally ready for
release, so
0:36
that the company can make
decisions based around that.
0:38
In this video we'll go over some easy
reports that can help you do that and
0:42
answer those questions.
0:46
So the first report we'll
cover is the coverage report.
0:48
This is the simple overview
of your testing efforts.
0:51
The idea here is to show what tests are
covering what parts of your application,
0:55
and how much more testing is
needed to cover that gap.
1:00
So let's say we have an application
that has three main parts.
1:03
Login, or authentication, creating cards
on a board, and printing out those cards.
1:06
Currently, we have two tests for
authentication, another four tests for
1:12
creating cards on the board, and we don't
have any tests for printing out the cards.
1:16
This tells us that there's a lack of
testing happening in one part of the app
1:21
and that we need to write some new tests
before we can feel confident in it.
1:25
Another thing that the coverage report can
show is where you're automation efforts
1:29
should be going.
1:32
If you track how long tests take to run,
1:34
then this will be a very
easy decision to make.
1:36
The longest tests should
probably be automated first
1:38
because they save you a lot of time.
1:42
But if you don't have that kind of data,
1:44
look at automating areas that require
a lot of testing to complete.
1:46
This is a simple way to illustrate that
data can be very powerful in helping
1:50
to make decisions.
1:55
Both for your business and
for your own QA efforts.
1:56
You always have limited time.
2:00
So it's good to know where to spend it.
2:01
So now that the software has been released
some customers are reporting bugs.
2:04
This is bad.
2:09
And we need to know about these
bugs as part of the QA team.
2:10
The difference between you finding a bug
before the software is released and
2:14
a customer finding a bug is huge.
2:18
Keep a tally of how many bugs are being
found by the customers each month.
2:21
Log them just like bugs that
are found in development.
2:26
These are escaped bugs that have found
their way into the released product.
2:30
Fixing these is much more costly than
fixing them in the key way process.
2:34
We want to put these bugs
into an escaped bugs report
2:39
to show to the rest of the team.
2:42
This report basically tells you if you
have some areas in the development or
2:45
testing process that need to be worked on.
2:49
This report will also help justify
putting more investment into that.
2:52
We want to make customers feel
more confident in their product
2:57
each time that there is a new release.
3:01
Further more,
every time a bug gets released or
3:03
out into the wild, it costs much more than
it would to fix it before it was released.
3:06
Finally, another simple report
is gathering user feedback.
3:12
If you're unfamiliar
with the Agile manifesto,
3:16
I recommend checking out the Treehouse
course on Scrum basics which goes over it.
3:19
But a very important aspect
of Agile is user feedback.
3:25
How does customer feedback fit into QA,
though?
3:29
Basically this will inform
if you are looking for
3:33
the right things in the software
when it is being tested.
3:35
How simple are the workflows?
3:39
Are the buttons confusing?
3:42
Is the documentation helpful?
3:43
These are the types of things
you can get from user feedback.
3:46
This is just a simple report
that I put together with all
3:50
the feedback that we've gotten back from
our customers on the RSVP app so far.
3:53
We aggregate the positive and negative
feedback as well as other feedback for
3:58
those that didn't specify.
4:03
We also take in all of the feedback and
4:05
calculate the satisfied percentage
by the amount of users giving
4:07
positive feedback divided by everyone
else who has given feedback.
4:11
Your way of calculating this number
may be a little different, and
4:15
you may not even really need
this number for your purposes.
4:19
But I feel like it's
a really nice way to show
4:22
how many users are satisfied
by your product.
4:24
You can solicit customer feedback
from a support or sales team.
4:28
What are your customers
saying about the software?
4:33
This can be tricky.
4:36
When generating a user feedback report,
4:38
you wanna gather all
the feedback you can get.
4:40
And look for
patterns in what the customers are saying.
4:43
This isn't an easy task though.
4:47
It's often done by a product team, so
4:50
you might be able to get
that sorta data from them.
4:52
In QA, we wanna know if our tests
are looking for the right things and
4:56
if we need to revisit old tests.
4:59
This is why user feedback is great.
5:02
I really just want to go over some
simple reports that you can keep
5:05
in your back pocket.
5:08
In reality, just about any piece
of data you're collecting,
5:10
could probably find it's
way into a useful report.
5:13
So, be on look out for
things your company may find useful.
5:17
You need to sign up for Treehouse in order to download course files.
Sign upYou need to sign up for Treehouse in order to set up Workspace
Sign up