Testing your application

The functionality described in this chapter is about testing your Python or MATLAB applications. The descriptions here are based on the functionality in MATLAB. The functionality in Python is mostly the same, so only significant differences are described here.

In addition to the tests you write for your application, it is recommended to write tests for the connection between your application and the front-end. For example, you could test whether clicking a certain button updates the submission data of another component using the data that was entered in the form.

In Python a pytest fixture (with scope 'function' and autouse set to True) creates the form object tree if the form is on the path. If you are using a testing framework other than pytest, call the _initialize() method of the Testing class in your test method setup code to get the same behaviour.
The submission data is updated when one of the gestures is executed. When a button is pressed using press, the underlying event is triggered (gui_event).

In MATLAB, Simian GUI provides a testing framework that works similar to the MATLAB App Testing Framework. For Python the testing functionality can be used with the testing frameworks like unittest or pytest.

There are three gestures you can simulate during the tests:

  • choose: Choose an option of multiple possibilities. For example, selecting one of multiple radio buttons.
  • press: Pressing a button, checkbox etc.
  • type: Type text in a component, for example in a TextField or Number component.

The table below shows you what gestures are available for what components. If a component is not listed in the table, no gestures are available for it. Note: this testing framework does not create an actual instance of your application. Instead, the form object tree of the form being tested is created using the gui_init method during the setup of the test method. In MATLAB this is done in a TestMethodSetup method.

ComponentchoosepresstypeComment
Button
Checkboxchoose: Provide the target value (true/false).
press: Toggles the checkbox.
Currency
DataGridPerform a gesture on a child component by providing this component as a parent. For example: testCase.type("textfield_key", "Text to type", "Parent", "my_datagrid");
Day
EditGridPerform a gesture on a child component by providing this component as a parent. For example: testCase.type("textfield_key", "Text to type", "Parent", "my_editgrid");
Email
NumberThe value can be numerical or a string that is convertible to a number.
Password
PhoneNumberThe value to type must be a string.
RadioThe option to choose can be either the label or the value.
SelectThe option to choose can be either the label or the value.
Selectboxeschoose: testCase.choose(key, true/false, "Label", optionLabel)
press: testCase.press(key, optionLabel)
SurveySyntax: testCase.choose(key, value, "Question", question) where value can be the label or the value of the answer and question can be the label or value of the question to answer.
TagsEnter tags one by one with multiple gestures.
TextArea
TextField
TimeThe value to type must be a string.

In addition to these gestures, you can use the following methods:

NameSyntaxDescription
getSubmissionData[data, isFound] = testCase​.getSubmissionData(​key, options)Return the submission data for the component with the given key. Optional name-value pairs are:
  • NestedForm: Key of the nested form the component is in.
  • Parent: Key of the parent component. Use this for example for components within DataGrids.
verifySubmissionDatatestCase​.verifySubmissionData(​key, expected, message, options)Verify that the submission data for the component with the given key equals the expected value. When the values are not equal, the optional message is added to the diagnostics of the test. This takes the same optional input arguments as the getSubmissionData method above.

Workflow

Consider the following Python and MATLAB forms where two numbers and an operation can be selected. By clicking the Calculate button, the calculation is performed.

def gui_init(payload: dict) -> dict:
    form = Form()
    payload["form"] = form

    numberOne = component.Number("number_one", form)
    numberOne.label = "First number"
    numberOne.defaultValue = 0

    numberTwo = component.Number("number_two", form)
    numberTwo.label = "Second number"
    numberTwo.defaultValue = 0

    operation = component.Radio("radio_operation", form)
    operation.label = "Operation"
    operation.inline = True
    operation.setValues(["Add", "Subtract", "Multiply"], ["plus", "minus", "times"])

    calculateButton = component.Button("calculate_button", form)
    calculateButton.label = "Calculate"
    calculateButton.setEvent("Calculate")

    numberThree = component.Number("number_three", form)
    numberThree.label = "Answer"
    numberThree.disabled = True

    return payload
function payload = guiInit(metaData)
    form            = Form();
    payload.form    = form;

    numberOne               = component.Number("number_one", form);
    numberOne.label         = "First number";
    numberOne.defaultValue  = 0;

    numberTwo               = component.Number("number_two", form);
    numberTwo.label         = "Second number";
    numberTwo.defaultValue  = 0;

    operation           = component.Radio("radio_operation", form);
    operation.label     = "Operation";
    operation.inline    = true;
    operation.setValues(["Add", "Subtract", "Multiply"], ["plus", "minus", "times"]);

    calculateButton         = component.Button("calculate_button", form);
    calculateButton.label   = "Calculate";
    calculateButton.setEvent("Calculate");

    numberThree             = component.Number("number_three", form);
    numberThree.label       = "Answer";
    numberThree.disabled    = true;
end

The resulting form looks as follows:

The gui_event functions as shown below perform the calculations and put the answer in the bottom Answer component:

def gui_event(meta_data: dict, payload: dict) -> dict:
    num_one, _ = utils.getSubmissionData(payload, "number_one")
    num_two, _ = utils.getSubmissionData(payload, "number_two")
    operation, _ = utils.getSubmissionData(payload, "radio_operation")

    if operation == "plus":
        answer = num_one + num_two
    elif operation == "minus":
        answer = num_one - num_two
    elif operation == "times":
        answer = num_one * num_two
    else:
        raise RuntimeError(f"Unknown operation '{operation}'")

    utils.setSubmissionData(payload, "number_three", answer)

    return payload
function payload = guiEvent(metaData, payload)
    numOne      = getSubmissionData(payload, "number_one");
    numTwo      = getSubmissionData(payload, "number_two");
    operation   = getSubmissionData(payload, "radio_operation");

    switch operation
        case "plus"
            answer = numOne + numTwo;
        case "minus"
            answer = numOne - numTwo;
        case "times"
            answer = numOne * numTwo;
        otherwise
            error("Unknown operation '%s'.", operation)
    end

    payload = setSubmissionData(payload, "number_three", answer);
end

Testing in Python

The calculator form above can be tested in Python as follows: Create a class that inherits from testing.Testing. Your class must implement a property namespace that contains the namespace of your form. After this you can start defining test methods using the methods described above.

For the above calculator form we can create a test method to test whether the multiplication of the example is properly executed. Let us enter some values using the type method, choose the operation with choose and press the calculate button using the press method. After that we can verify whether the value of the third Number component is set to the value we are expecting.

from simian.gui import testing

class testExample(testing.Testing):
    namespace = "myprogram.calculator"

    def test_multiplication(self):
        """Test whether multiplication is properly executed."""
        self.type("number_one", 3)
        self.type("number_two", 8)
        self.choose("radio_operator", "Multiply")
        self.press("button_calculate")
        self.verifySubmissionData("number_three", 24, "Multiplication failed")

Testing in MATLAB

This application can be tested in MATLAB as follows: Create a class that inherits from Testing. In turn, that class inherits from matlab.unittest.TestCase, so all testing-related functionality will be available. Your class must implement a property Namespace that has no attributes and assign a default (string) value to it. This is the namespace of the application, the same one you use for initializing the application in local MATLAB.

Create a test method to test whether the multiplication of the example is properly executed. Enter values using the type method, choose the operation with choose and press the button using the press method. Verify that the value of the third Number component is correctly set after pressing the button.

classdef testExample < Testing
    properties
        Namespace = "myprogram.calculator"
    end
    
    methods (Test)
        function testMultiplication(testCase)
            % Test whether multiplication is properly executed.
            testCase.type("number_one", 3);
            testCase.type("number_two", 8);
            testCase.choose("radio_operator", "Multiply");
            testCase.press("button_calculate");
            testCase.verifySubmissionData("number_three", 24, "Multiplication failed");
        end            
    end
end