Automation Assist, Part 1: Showcase
The benefits of UI Test Automation extend beyond verification. In this article series, I will show you how to use automated test scripts to assist:
- Training staff to use the app (coming soon)
- Test Data Generation (coming soon)
In the article “Use UI Automation to Assist Agile Showcases”, I introduced the idea and my experience. Now, I will show you how to do it.
- Reliable automated test scripts for newly implemented user stories.
For every user story, in the spirit of true Agile, there must be at least one automated end-to-end test for it. Showcase scripts will be based on them. Some might say: “we just want to use automation for showcases, like a use-once style”. However, it does not work this way. I understand the reasoning behind the above statement: the team has no confidence in maintaining automated test scripts. In that case, the team should seek external help to get test automation done properly.
Developing one-time-use showcase scripts still requires effort. Without being executed many times in the CT server, the scripts themself might be wrong, out of date, or flaky.
The top tier showcase script are one-time-use, but the page objects and helper functions are fully reusable. For examples,
login('manager','pass2')may be used in two different showcase scripts, but the definition of the reusable
loginfunction is in the
test_helper.rb. If the login page of the app changed later on, only the
loginfunctions need to be updated. There will be no changes to showcase scripts.
- The browser window remains open after the execution of the automation script completes.
We must keep the browser open so that customers can see the state of the application. More importantly, the business analyst often needs to navigate the application manually (at normal speed) to add further explanations.
Practices on preparation
When presenting the product to customers, preparation is a must. However, for showcases, the preparation time is very limited, usually 30 mins tops. Why?
- This is a regular thing
The team needs to do this once every sprint (typically, every 2 weeks). Also, business analysts might use showcase scripts when meeting with customers.
- The team is still busy working on the current sprint
Especially towards the end of a sprint, the team members are quite busy. Allocating a group of people to prepare the showcase for hours (every sprint) is not practical.
All these mean that the team needs to get showcase scripts done at very high efficiency. Here are some best practices:
1. Clone from existing automated tests
Copy selected test script files (for planned user stories) from the test folder, e.g.
spec to a dedicated
showcase folder. It is a good idea that the
showcase are sibling relationships under the same parent folder. This way, any reference to a relative file would still work.
Rename the test script file to a more meaningful name for the showcase, for example,
It is OK to break an existing test script into multiple automation script files, for example,
payment_spec.rb can be split into
sc2_payment_paypal_ok_spec.rb , …, etc.
“Copy and Paste’ may be acceptable to a certain extent in showcase scripts as they will usually be discarded after the showcase. Still, good design principles shall be applied, such as refining the scripts according to Maintainable Test Design and applying Functional Test Refactorings when appropriate.
2. Remove assertion steps and add comments that might help
While you can leave the assertion steps in the showcase scripts, I would suggest that you remove them as this will make the scripts more concise and slightly faster execution.
Also, feel free to add some comments or helpful hints into the scripts that will help others to understand them better, like below.
puts("Pause here for 10 seconds ...")
# BA talk about the new option in business type list
Because the engineer/business analyst who did the preparation might be absent the next day for unknown reasons, comments and hints will make it easier for others to step in.
Some automation scripts have side effects. One successful execution does not necessarily guarantee the next one (a very experienced test automation engineer may achieve that). For example, after executing an automation script of user signup with a fixed email address, the next executions will fail.
Putting technical factors aside, this means the team must rehearse the showcase, i.e. executing the showcase scripts multiple times to avoid ‘Oops’ in the presentation.
Practices on execution
A simple rule on the automation scripts execution for showcases: we quickly pass the not-so-interesting parts (such as user login), but we will be slow (pause, or stop) on showing the interesting parts. In other words, we control the execution speed (for test automation, the faster, the better).
I will demonstrate with my TestWise tool.
- Run automation scripts to a certain step, then stop/pause
- Right-click a test step, and select ‘Run to line’
- Automation starts, drive a browser to navigate the app, until the chosen step.
- You may optionally, uncheck the pause button (where the red arrow points to) and resume the execution.
it "Pause and Continue" do
login_page = LoginPage.new(driver)
# => Right click to run to the next line
# => Press resume to continue: click login
After the test execution is paused, you can resume or stop it.
2. Pause and Continue using fixed waits
Running the scripts up to a certain step is fine for team members with good test automation knowledge. For business analysts, I would recommend using simple waits. The reasons: a wrong step might be selected, test scripts could be formatted wrongly, a selected step might not be pausable, …, etc. The solution: add fixed waits, such as
sleep 10 (means, pause here for 10 seconds).
Generally speaking, business analysts are quite comfortable with explaining things. They will work well with a known time limit that is fully tunable anyway.
- Just simply right-click any line in a test case, and select ‘Run <TEST NAME>’ .
TestWise, if the test scripts follow the convention, will keep the browser open when running an individual test case.
it "Pause and Continue: show business types" do
business_sign_up_page = BusinessSignUpPage.new(driver)
puts("Pause ~ 10 seconds for BA to explain")
"Yoga" + "']").click
The above scripts (business sign-up) click and show all business types, pause 5 seconds (for BA to explain), then highlight 4 business types one by one (for a few seconds), and finally select ‘Yoga’ business type.
3. Sidekick scripts
Sidekick scripts are a set of small utility scripts that help you to navigate the application to a certain state quickly and reliably. For example, my common sidekick scripts are:
Inside a sidekick script, there might be a few scenarios, as below (in login_spec.rb)
it "login as registered user" do
endit "login as admin user" do
We can run an individual scenario only.
4. (Desirable) Attach execution of selected test steps to the existing browser
During a showcase, customers often ask questions. To respond, the business analyst needs to show and tell. In this situation, the team is not able to prepare the complete automation scripts beforehand. However, we can utilize some existing test steps, and run them directly against the current browser.
After that (the ‘attaching’ was established), the business analyst can freely copy steps and edit test scripts, and run them quickly.
Of course, to be able to achieve this, the business analyst (or whoever does the demonstration) needs to have a very high degree of understanding of the test scripts and the test tool. What I can tell you is that the efforts were very worthwhile after seeing the impressed and happy customers.