Testing your application

The functionality described in this chapter allows you to programmatically test the interaction between the frontend and backend of your applications. It allows you to mock end-user inputs in your app and verify that the backend returns the expected result. Note that frontend only functionality like validations, or conditional hiding or disabling of components cannot be tested in this way.

The descriptions here are based on the functionality in MATLAB. The functionality in Python is mostly the same, so only significant differences are described here.

In Python a pytest fixture (with scope 'function' and autouse set to True) creates the form object tree if the form is on the path. If you are using a testing framework other than pytest, call the _initialize() method of the Testing class in your test method setup code to get the same behaviour.

In MATLAB, Simian GUI provides a testing framework that works similar to the MATLAB App Testing Framework.

Mocking user actions

There are three types of user actions you can mock in the tests:

• choose: Choosing an option from multiple possibilities. For example, selecting one of multiple radio buttons.
• press: Pressing a button, checkbox etc.
• type: Typing text in a component, for example in a TextField or Number component.

The submission data is updated when one of the actions is performed. When a button is pressed using press, the corresponding event is triggered and the registered function is executed via the gui_event function of your app. When a component is configured to trigger an event when its value is changed, this event is triggered when the value is changed in the test.

The table below shows you what actions are available for which components. If a component is not listed in the table, no actions are available for it. Note: this testing framework does not create an actual instance of your application. Instead, the form object tree of the form being tested is created using the gui_init method during the setup of the test method. In MATLAB this is done in a TestMethodSetup method. In Python a test function setup fixture is used.

Note that these action methods are generic Testing class methods. To identify the intended component they must be called with the key of the component, and optionally with the parent or nested form key. For tabular components or components with multiple values, the optional index input can be used to select a row or cell in the submission data and change only that value.

In addition to these actions, you can use the following methods:

Workflow

Consider the following Python and MATLAB forms where two numbers and an operation can be selected. By clicking the Calculate button, the calculation is performed.

def gui_init(payload: dict) -> dict:
    form = Form()
    payload["form"] = form

    numberOne = component.Number("number_one", form)
    numberOne.label = "First number"
    numberOne.defaultValue = 0

    numberTwo = component.Number("number_two", form)
    numberTwo.label = "Second number"
    numberTwo.defaultValue = 0

    operation = component.Radio("radio_operation", form)
    operation.label = "Operation"
    operation.inline = True
    operation.setValues(["Add", "Subtract", "Multiply"], ["plus", "minus", "times"])

    calculateButton = component.Button("calculate_button", form)
    calculateButton.label = "Calculate"
    calculateButton.setEvent("Calculate")

    numberThree = component.Number("number_three", form)
    numberThree.label = "Answer"
    numberThree.disabled = True

    return payload

function payload = guiInit(metaData)
    form            = Form();
    payload.form    = form;

    numberOne               = component.Number("number_one", form);
    numberOne.label         = "First number";
    numberOne.defaultValue  = 0;

    numberTwo               = component.Number("number_two", form);
    numberTwo.label         = "Second number";
    numberTwo.defaultValue  = 0;

    operation           = component.Radio("radio_operation", form);
    operation.label     = "Operation";
    operation.inline    = true;
    operation.setValues(["Add", "Subtract", "Multiply"], ["plus", "minus", "times"]);

    calculateButton         = component.Button("calculate_button", form);
    calculateButton.label   = "Calculate";
    calculateButton.setEvent("Calculate");

    numberThree             = component.Number("number_three", form);
    numberThree.label       = "Answer";
    numberThree.disabled    = true;
end

The resulting form looks as follows:

The gui_event functions as shown below perform the calculations and put the answer in the bottom Answer component:

def gui_event(meta_data: dict, payload: dict) -> dict:
    num_one, _ = utils.getSubmissionData(payload, "number_one")
    num_two, _ = utils.getSubmissionData(payload, "number_two")
    operation, _ = utils.getSubmissionData(payload, "radio_operation")

    if operation == "plus":
        answer = num_one + num_two
    elif operation == "minus":
        answer = num_one - num_two
    elif operation == "times":
        answer = num_one * num_two
    else:
        raise RuntimeError(f"Unknown operation '{operation}'")

    utils.setSubmissionData(payload, "number_three", answer)

    return payload

function payload = guiEvent(metaData, payload)
    numOne      = getSubmissionData(payload, "number_one");
    numTwo      = getSubmissionData(payload, "number_two");
    operation   = getSubmissionData(payload, "radio_operation");

    switch operation
        case "plus"
            answer = numOne + numTwo;
        case "minus"
            answer = numOne - numTwo;
        case "times"
            answer = numOne * numTwo;
        otherwise
            error("Unknown operation '%s'.", operation)
    end

    payload = setSubmissionData(payload, "number_three", answer);
end

Testing in Python

The calculator form above can be tested in Python as follows: Create a class that inherits from testing.Testing. Your class must implement a property namespace that contains the namespace of your form. After this you can start defining test methods using the methods described above. Optionally, fill the add_meta_data property with metaData properties that would normally be set by the Portal or local run call.

For the above calculator form we can create a test method to test whether the multiplication of the example is properly executed. Let us enter some values using the type method, choose the operation with choose and press the calculate button using the press method. After that we can verify whether the value of the third Number component is set to the value we are expecting.

from simian.gui import testing

class testExample(testing.Testing):
    namespace = "myprogram.calculator"
    add_meta_data = {"application_data": {"brand": "new"}}

    def test_multiplication(self):
        """Test whether multiplication is properly executed."""
        self.type("number_one", 3)
        self.type("number_two", 8)
        self.choose("radio_operator", "Multiply")
        self.press("button_calculate")
        self.verifySubmissionData("number_three", 24, "Multiplication failed")

Testing in MATLAB

This application can be tested in MATLAB as follows: Create a class that inherits from simian.gui.Testing. In turn, that class inherits from matlab.unittest.TestCase, so all testing-related functionality will be available. Your class must implement a property namespace that has no attributes and assign a default (string) value to it. Optionally, define an addMetaData property with metaData properties that would normally be set by the Portal or local run call.

Create a test method to test whether the multiplication of the example is properly executed. Enter values using the type method, choose the operation with choose and press the button using the press method. Verify that the value of the third Number component is correctly set after pressing the button.

classdef testExample < Testing
    properties
        namespace   = "myprogram.calculator"
        addMetaData = struct("application_data": struct("brand", "new"))
    end
    
    methods (Test)
        function testMultiplication(testCase)
            % Test whether multiplication is properly executed.
            testCase.type("number_one", 3);
            testCase.type("number_two", 8);
            testCase.choose("radio_operator", "Multiply");
            testCase.press("button_calculate");
            testCase.verifySubmissionData("number_three", 24, "Multiplication failed");
        end            
    end
end