Exploration Testing and Automation5:51 with Ryan Saul
In this video, we will discuss exploration testing, and some of the basics of automation.
Introduction to Selenium Course
Test Plan: RSVP App - Edit Invitee Feature Release
Verify edit invitee feature for RSVP app.
Regression test previous features.
Test on all supported browsers - Chrome, Firefox and Edge on Windows 10.
Run automated tests - ½ Day.
New Feature testing - 1 Day.
Regression testing - 1 Day.
Supported browser testing - 1 Day.
Total: 3 ½ days.
- Test results.
- Bug report.
- Configuration test matrix
- Release must contain edit feature.
- All critical defects must be fixed.
- 100% of regression tests pass.
- 100% automated tests pass.
- All supported browsers must pass.
Risks and Contingencies:
- Previous client data may not load.
- Rollback to previous version.
- TEST1: [AUTOMATED] - Invite Someone new
- TEST2: [AUTOMATED] - Confirm someone
- TEST3: [AUTOMATED] - Remove someone
- TEST4: Edit someone
- TEST5: [EXPLORATORY] - RSVP workflow
So one thing I hear a lot these days is to work smarter, not harder. 0:01 This applies to QA engineering really well. 0:05 I'm going to talk about two concepts that are all about working smarter but 0:09 are also polar opposites of each other. 0:13 Exploration testing and test automation. 0:16 So exploration is all about discovery and insight. 0:20 When you explore a city, 0:25 you probably want to discover things you've never seen before. 0:27 So maybe you throw the map away, but 0:30 you also probably have some insight into what to look for. 0:33 Like knowing there's a block with some great restaurants or art galleries. 0:38 I feel like exploration testing is a lot like this. 0:43 You probably know the application pretty well, but 0:47 you wanna find some potential problems by trying out new things. 0:50 Instead of executing the exact steps in a test case, 0:55 play around with the settings a little bit. 0:59 And try things out in new ways. 1:01 More often than not, 1:04 there's part of the app that you suspect might have some problems. 1:06 Now, this probably sounds pretty informal and 1:10 maybe just something you do outside of the normal test plan but in fact, 1:13 this can, and should be added to your test plan document. 1:17 What you want to do with exploration tests 1:22 is define what part of the application you're going to test. 1:25 Put some conditions around what you aim to do in that test, but 1:28 don't put any of the formal testing steps like you would in a typical test case. 1:32 Exploration tests should be time boxed, or 1:38 in other words, only be done within a certain period of time. 1:41 An hour to two hours for a certain part of the application 1:46 may be enough time to let testers try out a few things and see what they find. 1:49 These exploratory tests are important because 1:56 going back to the edge case versus happy path definitions from earlier, 1:59 we need to try things that our users might do before they try it. 2:04 Now if your assumptions are correct and you find a good use case from exploratory 2:09 testing or maybe even a bug, then you should probably write that down as a new 2:13 test case with reproducible steps in the future. 2:18 Okay, now let's talk about what is probably the complete opposite 2:23 of exploratory testing, automation. 2:26 Automated tests, as the name suggests, is about codifying your tests so 2:30 that a machine can run it. 2:35 Going back to our regression tests from our previous test plan we 2:37 probably have a huge library of tests that we've written for our software already. 2:41 And we keep adding new features all the time. 2:46 Our regression test suite, as we like to call it, keeps growing and 2:49 growing, until running all of the regression tests for 2:52 each new version of the software takes days, weeks, or even months. 2:56 We can stay on top of this by putting our test into code 3:02 that can be run by a machine. 3:06 Tests that would probably take a human hours to run 3:08 now can be run in minutes by the machine. 3:11 Remember how we also talked about configuration testing and 3:15 running our tests on many different machines and browsers? 3:18 Automation helps here a ton. 3:22 We can now run the same test across all of our different-supported browsers and 3:25 versions with ease on each new version of the software. 3:30 Our tests are also very exact and 3:35 not prone to human error like they were before. 3:37 And we don't need human testers to run them in the first place, which frees up 3:40 our testers to do more exploration tests and other more valuable tasks. 3:44 There are drawbacks to automated testing however, the biggest problem is 3:48 that it's not so much automated tests as it is automated checks. 3:53 We can only write tests for things that we already know. 3:59 It's very difficult to automate exploration tests but not impossible. 4:03 And so the machine can probably only test things that we already discovered. 4:09 Another big drawback is that the amount of time it takes to write automated tests 4:14 is usually much longer than it is to write a manual test. 4:19 I don't recommend automating feature tests for this reason. 4:24 Because feature tests are usually only run once or twice and then tossed away. 4:27 We want to focus on automating regression tests 4:33 that we know we will need to run over and over again. 4:37 If you know a test will be run many times then the amount of time it takes 4:41 to automate it will probably be worth the investment in the long run. 4:46 For web application testing I recommend looking in to Selenium, 4:50 which is like coding how a person would interact with a web browser. 4:54 See the teacher's notes for a link to our Selenium course right here on Treehouse. 4:57 Okay, that's it for test execution. 5:03 In this section we've learned how to create a test plan, what regression and 5:06 configuration tests are, what exploration and 5:11 automation tests are and how all of these fit into your test plan strategy. 5:15 Remember, we want to test smarter, not harder. 5:21 Try to remember this when creating a test plan, 5:25 we don't want to test everything all the time. 5:28 Because it's expensive and probably a waste of time. 5:31 We want to put together a plan that satisfies the company's needs 5:36 within our budget and on time. 5:40 Thanks for watching. 5:44 In our next section, we'll get into how to report failures in our tests as bugs. 5:45 I'll see you there. 5:50
You need to sign up for Treehouse in order to download course files.Sign up