Bummer! This is just a preview. You need to be signed in with a Basic account to view the entire video.
Exploration Testing and Automation5:51 with Ryan Saul
In this video, we will discuss exploration testing, and some of the basics of automation.
Introduction to Selenium Course
Test Plan: RSVP App - Edit Invitee Feature Release
Verify edit invitee feature for RSVP app.
Regression test previous features.
Test on all supported browsers - Chrome, Firefox and Edge on Windows 10.
Run automated tests - ½ Day.
New Feature testing - 1 Day.
Regression testing - 1 Day.
Supported browser testing - 1 Day.
Total: 3 ½ days.
- Test results.
- Bug report.
- Configuration test matrix
- Release must contain edit feature.
- All critical defects must be fixed.
- 100% of regression tests pass.
- 100% automated tests pass.
- All supported browsers must pass.
Risks and Contingencies:
- Previous client data may not load.
- Rollback to previous version.
- TEST1: [AUTOMATED] - Invite Someone new
- TEST2: [AUTOMATED] - Confirm someone
- TEST3: [AUTOMATED] - Remove someone
- TEST4: Edit someone
- TEST5: [EXPLORATORY] - RSVP workflow
So one thing I hear a lot these days is to work smarter, not harder.
This applies to QA engineering really well.
I'm going to talk about two concepts that are all about working smarter but
are also polar opposites of each other.
Exploration testing and test automation.
So exploration is all about discovery and insight.
When you explore a city,
you probably want to discover things you've never seen before.
So maybe you throw the map away, but
you also probably have some insight into what to look for.
Like knowing there's a block with some great restaurants or art galleries.
I feel like exploration testing is a lot like this.
You probably know the application pretty well, but
you wanna find some potential problems by trying out new things.
Instead of executing the exact steps in a test case,
play around with the settings a little bit.
And try things out in new ways.
More often than not,
there's part of the app that you suspect might have some problems.
Now, this probably sounds pretty informal and
maybe just something you do outside of the normal test plan but in fact,
this can, and should be added to your test plan document.
What you want to do with exploration tests
is define what part of the application you're going to test.
Put some conditions around what you aim to do in that test, but
don't put any of the formal testing steps like you would in a typical test case.
Exploration tests should be time boxed, or
in other words, only be done within a certain period of time.
An hour to two hours for a certain part of the application
may be enough time to let testers try out a few things and see what they find.
These exploratory tests are important because
going back to the edge case versus happy path definitions from earlier,
we need to try things that our users might do before they try it.
Now if your assumptions are correct and you find a good use case from exploratory
testing or maybe even a bug, then you should probably write that down as a new
test case with reproducible steps in the future.
Okay, now let's talk about what is probably the complete opposite
of exploratory testing, automation.
Automated tests, as the name suggests, is about codifying your tests so
that a machine can run it.
Going back to our regression tests from our previous test plan we
probably have a huge library of tests that we've written for our software already.
And we keep adding new features all the time.
Our regression test suite, as we like to call it, keeps growing and
growing, until running all of the regression tests for
each new version of the software takes days, weeks, or even months.
We can stay on top of this by putting our test into code
that can be run by a machine.
Tests that would probably take a human hours to run
now can be run in minutes by the machine.
Remember how we also talked about configuration testing and
running our tests on many different machines and browsers?
Automation helps here a ton.
We can now run the same test across all of our different-supported browsers and
versions with ease on each new version of the software.
Our tests are also very exact and
not prone to human error like they were before.
And we don't need human testers to run them in the first place, which frees up
our testers to do more exploration tests and other more valuable tasks.
There are drawbacks to automated testing however, the biggest problem is
that it's not so much automated tests as it is automated checks.
We can only write tests for things that we already know.
It's very difficult to automate exploration tests but not impossible.
And so the machine can probably only test things that we already discovered.
Another big drawback is that the amount of time it takes to write automated tests
is usually much longer than it is to write a manual test.
I don't recommend automating feature tests for this reason.
Because feature tests are usually only run once or twice and then tossed away.
We want to focus on automating regression tests
that we know we will need to run over and over again.
If you know a test will be run many times then the amount of time it takes
to automate it will probably be worth the investment in the long run.
For web application testing I recommend looking in to Selenium,
which is like coding how a person would interact with a web browser.
See the teacher's notes for a link to our Selenium course right here on Treehouse.
Okay, that's it for test execution.
In this section we've learned how to create a test plan, what regression and
configuration tests are, what exploration and
automation tests are and how all of these fit into your test plan strategy.
Remember, we want to test smarter, not harder.
Try to remember this when creating a test plan,
we don't want to test everything all the time.
Because it's expensive and probably a waste of time.
We want to put together a plan that satisfies the company's needs
within our budget and on time.
Thanks for watching.
In our next section, we'll get into how to report failures in our tests as bugs.
I'll see you there.
You need to sign up for Treehouse in order to download course files.Sign up