Modeling the Windows Calculator: Part 2

Posted by Ben Simo

Adding Basic Validations

In the previous post, I created a simple model for starting and stopping the Windows calculator, and for switching between standard and scientific modes. I then created the code needed to execute that test and ran a test that hit each of the defined actions once.

As the next step, I reran the test with the MBTE configured to capture GUI object information as it executes. This created a checkpoint table that I then ran through a script that removes duplicates and combines rows that are the same for multiple states. I also manually reviewed this table to verify that the reported results are as I expected. I made some tweaks to the table based on my expectations. I can then use this checkpoint table as input for the next test execution. You may view the edited checkpoint file using the link below.

The checkpoint table is one of two table formats that I use for defining test oracles. I call the other format a state table. The state tables contain one validation per row and have additional fields for creating user-friendly descriptions of the validation. The state tables can also be used to reference external code for complex results validations. The checkpoint files contain one GUI object per row and the columns define the properties to validate and the expected values. While not as user-friendly as state tables, checkpoint tables are easy to automatically generate during test execution and reuse as input for future tests.

My calculator checkpoint table currently contains only positive tests to ensure that expected objects appear as expected. It does not yet contain any validations to ensure that the unexpected does not occur. For example, it contains no check to ensure that the calculator stops when the window is closed.

I then created a state table and added two oracles stating that the calculator window should exist when running and not exist when stopped. I gave each of these a failState value of "restart" to indicate that if these checks fail, the application should be restarted to resume testing.

My model currently contains the following files:

• Action Table: calc.action.xls
• State Table: calc.state.xls
• Checkpoint Table: calc.checkpoint.xls

I then ran a test with this model. The MBTE executed a test that hit each of my test set actions once without me needing to give it a sequence of test steps. The MBTE automatically generated the test steps based on the model.

The results from this test execution may be viewed here. Some features in the results require Internet Explorer and may not function in other browsers. These results are usually placed on a file server, so there may be issues I have not yet noticed when accessing them from a web server.

There are some failures reported in the results. These appear to be tool issues rather than bugs in the Windows Calculator. I will look into these failures later. Do you have any ideas about the failures?

The color-coded HTML results make it easy to tell what happened. Each row indicates what happened, where the action or validation was defined, the code executed, and other pertinent information. Please explore the results and send me any feedback.

What would you like to add to this test next? More validations? Additional actions?

Do you have any observations or questions about this automation approach? Please add them to the comments.

---

Modeling the Windows Calculator

• Part 1: Starting actions
  
• Part 2: Basic validations