1
00:00:01,020 --> 00:00:05,840
So one thing I hear a lot these days
is to work smarter, not harder.

2
00:00:05,840 --> 00:00:09,140
This applies to QA
engineering really well.

3
00:00:09,140 --> 00:00:13,940
I'm going to talk about two concepts
that are all about working smarter but

4
00:00:13,940 --> 00:00:16,680
are also polar opposites of each other.

5
00:00:16,680 --> 00:00:19,410
Exploration testing and test automation.

6
00:00:20,820 --> 00:00:25,550
So exploration is all about discovery and
insight.

7
00:00:25,550 --> 00:00:27,130
When you explore a city,

8
00:00:27,130 --> 00:00:30,910
you probably want to discover
things you've never seen before.

9
00:00:30,910 --> 00:00:33,780
So maybe you throw the map away, but

10
00:00:33,780 --> 00:00:38,160
you also probably have some
insight into what to look for.

11
00:00:38,160 --> 00:00:42,060
Like knowing there's a block with some
great restaurants or art galleries.

12
00:00:43,450 --> 00:00:47,160
I feel like exploration
testing is a lot like this.

13
00:00:47,160 --> 00:00:50,310
You probably know
the application pretty well, but

14
00:00:50,310 --> 00:00:55,190
you wanna find some potential
problems by trying out new things.

15
00:00:55,190 --> 00:00:59,240
Instead of executing the exact
steps in a test case,

16
00:00:59,240 --> 00:01:01,390
play around with
the settings a little bit.

17
00:01:01,390 --> 00:01:03,240
And try things out in new ways.

18
00:01:04,340 --> 00:01:06,060
More often than not,

19
00:01:06,060 --> 00:01:09,670
there's part of the app that you
suspect might have some problems.

20
00:01:10,840 --> 00:01:13,560
Now, this probably sounds
pretty informal and

21
00:01:13,560 --> 00:01:17,900
maybe just something you do outside
of the normal test plan but in fact,

22
00:01:17,900 --> 00:01:20,990
this can, and should be added
to your test plan document.

23
00:01:22,100 --> 00:01:25,040
What you want to do with exploration tests

24
00:01:25,040 --> 00:01:28,740
is define what part of
the application you're going to test.

25
00:01:28,740 --> 00:01:32,300
Put some conditions around what
you aim to do in that test, but

26
00:01:32,300 --> 00:01:36,840
don't put any of the formal testing steps
like you would in a typical test case.

27
00:01:38,120 --> 00:01:41,360
Exploration tests should be time boxed, or

28
00:01:41,360 --> 00:01:45,110
in other words, only be done
within a certain period of time.

29
00:01:46,130 --> 00:01:49,890
An hour to two hours for
a certain part of the application

30
00:01:49,890 --> 00:01:54,840
may be enough time to let testers try
out a few things and see what they find.

31
00:01:56,350 --> 00:01:59,700
These exploratory tests
are important because

32
00:01:59,700 --> 00:02:04,170
going back to the edge case versus
happy path definitions from earlier,

33
00:02:04,170 --> 00:02:08,160
we need to try things that our
users might do before they try it.

34
00:02:09,280 --> 00:02:13,830
Now if your assumptions are correct and
you find a good use case from exploratory

35
00:02:13,830 --> 00:02:18,880
testing or maybe even a bug, then you
should probably write that down as a new

36
00:02:18,880 --> 00:02:22,290
test case with reproducible
steps in the future.

37
00:02:23,450 --> 00:02:26,920
Okay, now let's talk about what
is probably the complete opposite

38
00:02:26,920 --> 00:02:29,330
of exploratory testing, automation.

39
00:02:30,350 --> 00:02:35,310
Automated tests, as the name suggests,
is about codifying your tests so

40
00:02:35,310 --> 00:02:37,430
that a machine can run it.

41
00:02:37,430 --> 00:02:41,500
Going back to our regression tests
from our previous test plan we

42
00:02:41,500 --> 00:02:46,620
probably have a huge library of tests that
we've written for our software already.

43
00:02:46,620 --> 00:02:49,220
And we keep adding new
features all the time.

44
00:02:49,220 --> 00:02:52,920
Our regression test suite,
as we like to call it, keeps growing and

45
00:02:52,920 --> 00:02:56,620
growing, until running all
of the regression tests for

46
00:02:56,620 --> 00:03:01,590
each new version of the software
takes days, weeks, or even months.

47
00:03:02,590 --> 00:03:06,540
We can stay on top of this by
putting our test into code

48
00:03:06,540 --> 00:03:07,750
that can be run by a machine.

49
00:03:08,910 --> 00:03:11,870
Tests that would probably
take a human hours to run

50
00:03:11,870 --> 00:03:13,970
now can be run in minutes by the machine.

51
00:03:15,050 --> 00:03:18,970
Remember how we also talked
about configuration testing and

52
00:03:18,970 --> 00:03:22,690
running our tests on many
different machines and browsers?

53
00:03:22,690 --> 00:03:24,350
Automation helps here a ton.

54
00:03:25,350 --> 00:03:30,090
We can now run the same test across all
of our different-supported browsers and

55
00:03:30,090 --> 00:03:33,600
versions with ease on each
new version of the software.

56
00:03:35,090 --> 00:03:37,410
Our tests are also very exact and

57
00:03:37,410 --> 00:03:40,280
not prone to human error
like they were before.

58
00:03:40,280 --> 00:03:44,230
And we don't need human testers to run
them in the first place, which frees up

59
00:03:44,230 --> 00:03:48,878
our testers to do more exploration
tests and other more valuable tasks.

60
00:03:48,878 --> 00:03:53,920
There are drawbacks to automated
testing however, the biggest problem is

61
00:03:53,920 --> 00:03:59,630
that it's not so much automated
tests as it is automated checks.

62
00:03:59,630 --> 00:04:02,510
We can only write tests for
things that we already know.

63
00:04:03,540 --> 00:04:09,260
It's very difficult to automate
exploration tests but not impossible.

64
00:04:09,260 --> 00:04:13,160
And so the machine can probably only
test things that we already discovered.

65
00:04:14,890 --> 00:04:19,750
Another big drawback is that the amount
of time it takes to write automated tests

66
00:04:19,750 --> 00:04:24,240
is usually much longer than
it is to write a manual test.

67
00:04:24,240 --> 00:04:27,790
I don't recommend automating
feature tests for this reason.

68
00:04:27,790 --> 00:04:32,880
Because feature tests are usually only
run once or twice and then tossed away.

69
00:04:33,960 --> 00:04:37,300
We want to focus on
automating regression tests

70
00:04:37,300 --> 00:04:40,220
that we know we will need to run over and
over again.

71
00:04:41,240 --> 00:04:46,180
If you know a test will be run many
times then the amount of time it takes

72
00:04:46,180 --> 00:04:50,650
to automate it will probably be worth
the investment in the long run.

73
00:04:50,650 --> 00:04:54,720
For web application testing I
recommend looking in to Selenium,

74
00:04:54,720 --> 00:04:57,718
which is like coding how a person
would interact with a web browser.

75
00:04:57,718 --> 00:05:02,590
See the teacher's notes for a link to our
Selenium course right here on Treehouse.

76
00:05:03,780 --> 00:05:06,900
Okay, that's it for test execution.

77
00:05:06,900 --> 00:05:11,760
In this section we've learned how to
create a test plan, what regression and

78
00:05:11,760 --> 00:05:15,200
configuration tests are,
what exploration and

79
00:05:15,200 --> 00:05:20,110
automation tests are and how all of
these fit into your test plan strategy.

80
00:05:21,230 --> 00:05:24,590
Remember, we want to test smarter,
not harder.

81
00:05:25,600 --> 00:05:28,539
Try to remember this when
creating a test plan,

82
00:05:28,539 --> 00:05:31,960
we don't want to test
everything all the time.

83
00:05:31,960 --> 00:05:34,980
Because it's expensive and
probably a waste of time.

84
00:05:36,050 --> 00:05:40,660
We want to put together a plan
that satisfies the company's needs

85
00:05:40,660 --> 00:05:42,630
within our budget and on time.

86
00:05:44,100 --> 00:05:45,280
Thanks for watching.

87
00:05:45,280 --> 00:05:50,330
In our next section, we'll get into how
to report failures in our tests as bugs.

88
00:05:50,330 --> 00:05:50,880
I'll see you there.