Windows Automation 01 - Getting Start with Calculator Automation

This is the first in a series of blogs talking about Windows Automation Test. Other related blogs are:

02: Send Mail with Mail Client
Qt Automation 01: Jump Start with the First Sample
Qt Automation 02: Control Spying

In this article, we will use Windows 10 Calculator application to show how to automate test Windows native application. Node.js is used as programming language and also Cucumber.js is used as test automation framework. After walking through with us in this article, you will have some idea on how to develop Windows automation with JavaScript skill, and also how to write a behavior driven automation test.

Content

• Preparation
• Design Scenario
• Create object model
• Edit automation script
• Make it data driven
• Test report

Preparation

• Knowledge

  1. It is preferred to have some knowledge on Cucumber.js and BDD. To learn Cucumber.js and how to create a script, read the blog series BDD Web Automation
  2. Learn Node.js and JavaScript language, including the new JavaScript language feature async / await, I am sure you know where to find these materials with search engine, so won’t provide links here.

• Tools

  1. Node.js Node.js is used when install npm packages. We won't use any additional packages for this article, but you may use some external packages when you write script of your own.

  2. CukeTest: Development environment for Cucumber.js. Cucumber.js is a popular Behavior-Driven (BDD) automation framework. CukeTest can be downloaded from http://cuketest.com/download. It also has Windows Store and Mac version.

  CukeTest has full capability of automating Windows applications, it supports different types of application, including native Windows Apps, WPF, WinForm, Qt, Delphi, DevExpress etc.

Design Scenario

Open CukeTest, create a new project, with Windows template:

This template is used for creating automation scripts for Windows applications.

Then you can edit the scenario in the feature file as follows:

The following is the text version:

Feature: Windows Calculator Test
Windows Calculator automation test

  Scenario: Simple Addition
    Given click on the number 1
    And click the "plus" sign
    When click on the number 2
    And click [equals] button
    Then the result should be 3

Create object model

In the project you have created, click "model1.tmodel" file in the project tree to open Model Manager, which is a component of CukeTest. The object model file defines the test objects that maps to the controls on application under test, and the file has extension “.tmodel”. When automation testing runs, script uses the properties defined in the model file to locate the controls.

Now we use Model Manager to spy controls on the Calculator. Click “Add Object” button, Model Manager will hide itself, so you can select a control from screen. You then select a control and add it to the model by clicking “+” button on "Add Objects Window".

To do it in a faster manner, you can click “Batch Add Controls” button on the toolbar, and also select one of number buttons from Calculator app. It will then list all number buttons on the same level:

You can then batch add them to the model. Do this similarly for sign buttons ( +, -, *, /, =)，and also the result display control.

In the Model Manager, when you select an object in the model tree, the corresponding control on the Calculator will be highlighted, which tells you that the control is identifiable with the object properties in the model:

If you accept the default properties recommendation when add objects, the Name property is one of “Identification Properties”, and the value is "Displayed is 0". However, in Calculator, the value of the Name property changes when you enter digits into Calculator. As shown below:

Also, the Name value will change to some localized text if you change the locale of windows to Non-English language.

To ensure the object model you have created work on Windows with different languages settings, you can remove the Name property from this object and just use AutomationId property. It should be sufficient identify this control. Also change name “Display as 0“ to something more relevant:

"AutomationId" is a property that is designed specially for test automation, therefore use it whenever possible.

Notice that when removing this Name property, it is moved from “Identifying Properties” to “Other Properties”. In some occasions, you want to add them back as “Identifying Properties”, you can do it by clicking this problem and selecting “+” button on the property grid toolbar.

Save the changed model file.

Edit automation script

Create a new support/hooks.js file and set the step timeout value, if a step definition runs longer than this timeout value, it will stop the execution of this step, mark it as failed, and move to the next step.

let { setDefaultTimeout } = require('cucumber')

//set timeout to be 10 seconds
setDefaultTimeout(10 * 1000);

You can change the timeout value to some value you think appropriate for your application. The default value 5 seconds should be long enough for more cases, this is provide just in case some application repond longer than the default value.

Open definition1.js in editor, put it side-by-side with the feature panel, click the button next to each step, it will automatically generate code stub for the step in function as the following:

const { Given, When, Then, After } = require('cucumber');
const { TestModel } = require('leanpro.win');
const { Util } = require('leanpro.common');

let model = TestModel.loadModel(__dirname + "/model1.tmodel");

//// Your step definitions /////

Given(/^click on the number (\d+)$/, async function (digit) {
    return 'pending';
});

Given(/^click the "([^"]*)" sign$/, async function (sign) {
    return 'pending';
});

Then(/^the result should be (\d+)$/, async function (result) {
    return 'pending';
});

When(/^click \[equals\] button$/, async function () {
    return 'pending';
}); 

For more information about how to create code from the feature file, refer to Generate Cucumber Script with Right Tool.

Now you can implement automation code for these step definition function stubs. You can copy the method code from Model Manager or drag-drop it to editor:

After adding all the implementation, your definition1.js code should look like the following:

const { Given, When, Then, After } = require('cucumber');
const { TestModel } = require('leanpro.win');
const { Util } = require('leanpro.common');
const assert = require('assert');

let model = TestModel.loadModel(__dirname + "/model1.tmodel");

//// Your step definitions /////

async function clickNumber(digit) {
    switch (digit) {
        case 1:
            buttonName = 'One'
            break;
        case 2:
            buttonName = 'Two';
            break;
        case 3:
            buttonName = 'Three';
            break;
        case 4:
            buttonName = 'Four';
            break;
        case 5:
            buttonName = 'Five';
            break;
        case 6:
            buttonName = 'Six';
            break;
        case 7:
            buttonName = 'Seven';
            break;
        case 8:
            buttonName = 'Eight';
            break;
        case 9:
            buttonName = 'Nine';
            break;
        default:
            break;
    }
    await model.getButton(buttonName).click();
}

async function clickSign(sign) {
    switch (sign) {
        case 'plus':
            return model.getButton('Plus').click();
        case 'minus':
            return model.getButton('Minus').click();
        case 'multiply':
            return model.getButton('Multiply by').click();
        case 'divide':
            return model.getButton('Divide by').click();
    }
}

Given(/^click on the number (\d+)$/, async function (digit) {    
    await model.getWindow("Window").activate();
    await clickNumber(digit);
});

Given(/^click the "([^"]*)" sign$/, async function (sign) {
    clickSign(sign);
    //Calculater need some time to perform calculation
});

Then(/^the result should be (\d+)$/, async function (result) {
    let name = await model.getText("Display").name();
    console.log('result', result, name, name, name.endsWith(result.toString()))
    return assert.ok(name.endsWith(result), `expect ${result}, actual ${name}`);
});

When(/^click \[equals\] button$/, async function () {
    return model.getButton('Equals').click();
    //Calculater need some time to perform calculation
    await Util.delay(1000); 
});

After(async function() {
    //take screenshot
    let screenShot = await model.getWindow("Window").takeScreenshot();
    //insert into report
    this.attach(screenShot, 'image/png');
})

Notice that in the code, we have also added a hook function After, which runs after each scenario. In the code, you can see that it captures a screenshot of CukeTest application and inserts it into the report. For more information about Hook, you can read article Hooks and TimeOut.

Make it data driven

It is very common practice to make the test automation data driven. In CukeTest, it is easy to do. First convert the scenario type to Scenario Outline, and then add the data line. For more details on how to do data driven in Cucumber.js, you can read Create Data Driven Automation Script with Doc String & Data Table.

To do it:

1. Right click the scenario title, change the scenario type to Scenario Outline:

   

   You then get the following:

   
   notice that all parameters has been extracted out and inserted into an Example table below the scenario description, so that you can add more data to this table.

2. Add data to Example table

   

   Note: You can edit data in Example manually or imported from csv data

The result feature text is the following:

Feature: Windows Calculator Test
Windows Calculator automation test

  Scenario Outline: Simple Addition
    Given click on the number <number1>
    And click the "<operator>" sign
    When click on the number <number2>
    And click [equals] button
    Then the result should be <result>
    Examples: 
      | number1 | operator | number2 | result |
      | 1       | plus     | 2       | 3      |
      | 9       | minus    | 3       | 6      |
      | 4       | plus     | 5       | 9      |
      | 3       | multiply | 6       | 18     |
      | 8       | divide   | 2       | 4      |

You may notice that to convert the automation script to data driven, there is no need to change the code part, we just updated the feature file with more data.

Test report

Click on the “Run Project” button on CukeTest to run the whole project. After running, CukeTest automatically shows you an html test report.

Notice that the report has both summary and details, and when expand the scenario detail, you can also see the screenshot of the application.

Summary

As you can see from above the example, CukeTest allows you to quickly build BDD automated tests for Windows applications. It support Node.js development automation scripts, and also provides Windows control spying and model management capabilities and a rich set of Windows automation APIs; CukeTest helps quickly build behavior-driven test script and visual report. With this tool, you can create a Windows automation script that is easy to use and maintain.

See also:
Windows Application Automation 02: Send Mail with Mail Client