This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Flow

Overview

A test flow is a sequence of scenarios that verifies the functionality or correctness of your business system. The Flow module allows you to manage all your test flows in a project. This includes defining Scenarios, choosing or creating the steps and assertions for each scenario, and modifying or running test flows.

View

In the upper left corner, select your project from the Select Project dropdown list and version from the Version dropdown list. Only the flows associated with the selected project and version are displayed on the left pane.

Edit a test flow

To view or modify the details of a flow, select the flow from the left pane that lists all flows associated with the selected project. You can change any detail by directly editing the corresponding field. Ensure to click Update after making your modifications.

The filter feature allows you to filter the list of test flows by providing a keyword or the full name of the flow. Only the flows that match the provided keyword or name will be displayed in the view.

Filter

Click the icon to display additional filtering and sorting options. For filtering, a dropdown list of available filter parameters will show all associated values for the selected project and version.

Create a test flow

To create a simple sequential test flow, follow these steps. Same steps apply when creating a parallel execution with multiple scenarios connected from the Start node or any scenario. For more information, see Parallel executions.

  1. Select the Project and the Version on the upper left dropdown lists. If there are existing test flows in the selected project and version, they will be displayed on the left pane.

  2. Click the plus icon.

  3. Enter a valid test flow title, replacing Untitled Test Flow.

  4. Click on the Untitled Scenario, which is automatically added when creating a new flow.

  5. Add your first scenario by selecting an existing scenario within your project in the Create/Select Scenario field in the Scenario Details section. You can also Create a new scenario.

    Note that if you are using an existing scenario, the steps, sources, targets, and assertions configured in that scenario for other flows will be included.

  6. Verify that all the associated steps for the selected and applied scenario are displayed. You can also Create a step in the flow module.

  7. Optional. Add a source and target parameters to the steps.

  1. Verify if the automatically added assertion is correct.
  1. Click Save.

Scenario fail nodes

Scenarios connected from a fail node are only executed only after the parent scenario fails. If the parent scenario passed, the fail-node scenario will be skipped.

Step fail nodes

Steps connected from a fail node are only executed only after the parent step fails. If the parent step passed, the fail-node step will be skipped.

Modify a test flow

You can also modify the configuration of test flows that have already been added to a test suite.

You can change the execution order of test flows by using the drag-and-drop functionality.

You can also modify the the dataset associated with a test flow or configure multiple datasets for it even after it has been added to the test suite. For more information, see Edit a dataset.

Similarly, if you need to run the test flow in a different environment you can also modify the environment. For more information, see Modify a step in the Flow module

Run a test flow

To run the test flow, click the Run button to view the Run Setup Form. Configure the necessary parameters in the following sections, then click the Run button.

Once the flow is executed, a new browser tab displaying the Run Details page will open. For more information, see Results.

Run config

The Run configuration allows you to enable the following options for flow execution:

  • Sync run to TestRail — Automatically syncs the run to the TestRail project associated with the flow.
  • Skip execution on failure — If enabled, the flow automatically stops execution when a scenario fails.
  • Use New Execution Egine — If enabled, the flow execution will use the new engine.

Datasets

Add the datasets to be used for execution by searching in the Search datasets field. All datasets associated with the flow are displayed. A counter in the header shows the number of selected datasets. Once selected, the datasets appear in the list.

Environment Setup

Before running a flow, you can also configure the environment for each scenario. All scenarios are listed in this section. The following settings can be configured for each scenario:

  • Environment
  • Database
  • Schema for each step in a scenario

Resume or stop an execution

Resume an execution

To resume an execution, click the Resume button.

You will have an option to rerun steps that failed assertion by enabling or disabling the toggle. If this is enabled, steps with failed assertions will be reexecuted. If not, steps with failed assertions will be skipped.

Then, click Resume on the confirmation prompt.

The execution will then be resumed from where it was stopped.

Stop an execution

Note: Only executions with an enabled Use New Execution Engine can be stopped.

If you want to stop an execution, click the Stop button on the upper-right corner of the Results screen.

Then, click the Stop run button on the confirmation prompt.

Once an execution has been stopped, the Resume button will be displayed.

Iterations

To view all iterations generated during a resumed execution, click Iterations on the upper-right of the Results page.

The currently viewed iteration will be mark as Current. If you wish to view the results of a different iteration, click the -> icon for that iteration.

Import a test flow

The Flow module enables you to import existing flows by clicking the down arrow in the left menu and select Import.

Drag or browse the file to be imported. The only supported file type is .ta_export.

Export a test flow

To export a flow, hover over the flow you want to export. An icon will appear. Click on this icon and select Export.

A list of all items to be exported is displayed.

After confirming the list, TA begins the export process.

Once complete, the file is automatically downloaded to your machine.

Delete a test flow

To delete a flow, hover over the flow you want to remove. An icon will appear. Click on this icon and select Delete.

The flow will be deleted from the version, but the scenarios, steps, and assertions used in it will not be deleted.

1 - Scenarios

Overview

A scenario is a high-level description of what you are testing. It consists of Steps that break down the scenario further, and Assertions that serve as validation criteria. A scenario can only be considered a success if it meets the validation criteria. Otherwise, it will be tagged as failed.

Create a new scenario

To create a new scenario, follow these steps:

  1. Add a new scenario node.

  2. Rename the scenario.

  3. Enter a valid scenario name in the Scenario name field.

  4. Click the Rename scenario button.

  5. Optional. Add a description for your scenario.

To add steps and assertions to your scenario, see Create a Test Flow.

Clone a scenario

To clone a scenario, follow these steps:

  1. Select a scenario you wish to clone.

  2. Click the scenario dropdown in the Scenario Details pane.

  3. Select Create new from <name of the selected scenario>.

  4. Enter a valid name in the Scenario name field.

  5. Click the Create copy button.

The cloned scenario replaces the older scenario.

Delete a scenario

To delete a scenario, click the more options icon in the upper-right corner of the scenario box, then select the Remove icon from the menu.

A prompt will ask whether to delete the scenario from this flow or from the entire version. Select your preferred option to continue.

Delete from test flow

If you select Remove from test flow, the scenario is deleted only from the current flow. Other flows using the scenario remain unaffected. There is no confirmation prompt, so verify you have selected the correct scenario before proceeding.

Delete scenario entirely

If you select Delete scenario entirely, the scenario is permanently removed from the version.

If the scenario is used in other flows, a pop-up will display all affected flows. You must delete the scenario from these flows before proceeding.

After deleting the scenario from all flows, you can now delete it entirely. A prompt specifying the version where it will be deleted is displayed.

Scenario usage

In the scenario usage tab, all the flows using the selected scenario node are displayed.

When the hyperlink is clicked, a new tab will open displaying the selected flow.

2 - Steps

Overview

The Flow modules allow you to use the steps defined in the Step module or add a new step to the project within the module.

Create a step in the Flow module

To create a new step in the Flow module, follow these steps:

  1. Click the scenario where you wish to create a step.

  2. Select the step type.

  1. Select the newly-created step node to see the details.
  1. Rename the step node.
  1. Enter a valid name in the Step name field.
  1. Click the Rename button.
  1. Add the necessary details by clicking the Details button. For more information, see:

  1. Optional. Enter valid values in the Source and Target fields. For more information, see Step sources and Step targets. You can also add new steps following the newly-created one.

  2. Click Save.

You can link steps to pass parameters to each other. However, you can only link steps that are part of the same scenario.

Step sources

The following are the possible sources for a step:

Source Description
Current step request payload Uses the value of a JSON node from the current request payload as a source for the selected step.
Current and previous step response payloads Uses the value of a JSON node from the previous and current response payload as a source for the selected step.
Previous variables Uses the value of a test flow variable defined in the step prior to the selected step.

Step targets

The following are the possible targets for a step:

Target Description
Request body Uses the value of a JSON node from the current request payload as a target for the selected step.
Query params Assigns the value of the source to a query parameter defined within the selected step.
Path params Assigns the value of the source to a path parameter defined within the selected step.
Header Assigns the value of the source to a header.
New variable

Creates a new test flow variable as the target for the selected step.

The created variable can be used in succeeding steps or validations of the current step as a source.
Previous variables Uses a test flow variable defined prior to the selected step or scenario as a target.

Modify a step in the Flow module

To modify a step in the Flow module, follow these steps:

  1. Click the step type link on the upper-right corner of the step node.

  2. Click the edit icon.

  1. Make the necessary changes in the Edit Step window.

Clone a step

To clone a step, follow these steps:

  1. Select a step you wish to clone.

  2. Click the step dropdown in the Step Details pane.

  3. Select Create new from <name of the selected step>.

  4. Enter a valid name in the Step name field.

  5. Click the Create copy button.

The cloned scenario replaces the older scenario.

Delete a step

To delete a step, click the step type link on the upper right corner of the step box. More otions are displayed, click the Remove Step icon.

A confirmation popup window will be displayed. Click the Remove node button.

Steps usage

In the step usage tab, all the flows using the selected step node are displayed.

When the hyperlink is clicked, a new tab will open displaying the selected flow.

3 - Assertion

Overview

The Flow modules allow you to use the assertions defined in the Assertion module or add a new step to the project within the module.

Create an assertion in the Flow module

To create a new assertion in the Flow module, follow these steps:

  1. Select the assertion type you wish to add.
  1. Select the newly-created step node to see the details.
  1. Rename the step node.
  1. Enter a valid name in the Step name field.
  1. Click the Rename button.

  1. Click the Details button.

  2. Add valid values to the necessary assertion parameters. For more information, see

  3. Click the Save button.

Disable/Enable an assertion in the Flow module

In the Flow module, you can disable an assertion for a specific scenario, preventing it from executing during flow execution.

To disable an assertion, follow these steps:

  1. Click the more options icon beside the assertion you wish to disable.
  1. Click Disable for this test flow.

The disabled assertion will also be automatically be disabled in the associated dataset.

To enable it back, follow these steps:

  1. Click the more options icon beside the assertion you wish to enable.
  1. Click Enable for this flow.

The enabled assertion will also be automatically enabled in the associated dataset.

Remove an assertion from a step

To remove an assertion from a step, follow these steps:

  1. Click the more options icon beside the assertion you wish to remove.
  1. Click Remove and confirm the action.

The assertion will be removed from the step, but it will remain in the Assertion module.

Dynamic row validation

Dynamic row validation allows you to validate queries that return multiple rows of output data.

Traditionally, query assertions only validate the first row of actual data against the expected conditions defined in assertion validations. With dynamic row validation, you can configure conditions to validate multiple rows using one of three evaluation modes: Fixed row, Any row, or All rows.

Configure a row validation

To configure a row validation, follow these steps:

  1. Select the assertion your wish to configure.

  2. Click on the Assertion Details tab.

  3. Add your required inputs.

  4. Click the Validations button in the Validations section.

  5. Select the validation type.

  6. Depending on the validation type, follow the specified steps in

  • Configure a fixed row validation
  • Configure an any row validation
  • Configure an all rows validation

Fixed row

A fixed row validation validates the query response at a specific, pre-configured row index. The system extracts the row at the defined index and compares its attribute values against your conditions.

For example, if a query returns 10 person records and the assertion is configured for Fixed Row mode at index 4, the system isolates the 4th record. If your conditions dictate that ENTITY_NAME must be Quinn,Tom and NAME_TYPE_FLAG must be PRIM, the assertion will succeed only if the 4th record meets both criteria.

Configure a fixed row validation

To configure a fixed row validation, follow the steps in Configure a row validation first.

  1. Set the target index number for the row you want to validate in the Row Index field.

  2. Define your validation criteria by mapping the actual value, selecting an operator, and providing an expected data.

  3. Run the flow.

The tool will validate the configured criteria exclusively against the row at the defined index.

Any row

The assertion succeeds if at least one row in the query response satisfies all defined validation conditions.

For example, if a query returns 10 person records under Any Row mode, the system scans the dataset. If even a single person record contains ENTITY_NAME = ‘Quinn,Tom’ and NAME_TYPE_FLAG = ‘PRIM’, the entire assertion succeeds.

Configure an any row validation

To configure a fixed row validation, follow the steps in Configure a row validation first.

  1. Define your validation criteria by mapping the actual value, selecting an operator, and providing an expected data.

  2. Run the flow.

The tool will validate the configured criteria against all rows. The assertion succeeds if at least one row meets all the configured criteria.

All rows

The assertion succeeds only if every single row in the query response satisfies all defined validation conditions. If even one row fails a single condition, the entire assertion fails.

For example, if a query returns 10 person records under All Rows mode with a condition checking that NAME_TYPE_FLAG is PRIM, then all 10 records must have a value of PRIM for the assertion to succeed.

Configure an all rows validation

To configure a fixed row validation, follow the steps in Configure a row validation first.

  1. Define your validation criteria by mapping the actual value, selecting an operator, and providing an expected data.

  2. Run the flow.

The tool will validate the configured criteria against all rows. The assertion succeeds only if all rows meet all the configured criteria.

4 - Sync to TestRail

Overview

This page provides the steps to sync test flows and test runs with TestRail. IF you want to learn how to configure your TestRail account for integration, see Settings. If you want to learn how to integrate your TestRail account with your project, see Integrations.

Sync test flows

Synchronize test flows to TestRail to link test cases and results across platforms.

  1. Select a test flow.

  2. Click the Sync to test flow button in the top-right corner of the page.


  1. Click Sync.

After a successful sync, a confirmation message will be displayed.

Sync test runs

Synchronize test runs to TestRail enables real-time tracking of test progress and outcomes.

  1. Click the Run button to view the Run Setup Form.
  1. Click the Sync Run To TestRail toggle.

You can also sync the run to TestRail in the following modules by repeating the steps above.

  • Results
    First, you have to select the flow you wish to sync from the Test Flow dropdown.

5 - Issue Tracking

Overview

Before linking an issue, ensure that you have enabled the JIRA integration in the Settings module and have configured JIRA in the Project Integrations.

You have the option to create a new issue or link a flow to an existing one.

Create a new issue

  1. Click the Issue Tracking icon.

  2. Click the Link Issue button.

  3. Enter valid values in the JIRA fields displayed.

  4. Click Create Issue. A confirmation message will be displayed.

    A JIRA ticket is now displayed when the Issue Tracking icon is clicked.

  5. Click the Save button to save the linked issue.

Synchronize test runs to TestRail enables real-time tracking of test progress and outcomes.

  1. Click the Issue Tracking icon.
  1. Click the Link Issue button.
  1. Click the toggle button to the Existing Issue.
  1. Select how you want to search for an existing issue.

  1. In the Search Issue field, enter a number if you selected By Issue Number.

    A dropdown listing all tickets matching your search is displayed.

    If you selected By Issue Title, enter a keyword.

  2. Click the Link Issue button.

    A JIRA ticket is now displayed when the Issue Tracking icon is clicked.

  3. Click the Save button to save the linked issue.

6 - Parallel Execution

Overview

A test flow with parallel executions is a flow with multiple scenarios that has been connected from the Start node or any scenario. You can follow the same steps in Create a test flow.

Dataset

The steps in creating a dataset for a flow with parallel executions are just the same when creating datasets for sequential executions. For more information, see Dataset.

Run a test flow

To run a test flow with multiple flows, enable the Use New Execution Engine toggle.

Scenario fail nodes

Similar to sequential flows, fail-node scenarios in parallel flows are only executed only after the parent scenario fails. If the parent scenario passed, the fail-node scenario will be skipped.

Step fail nodes

Similar to sequential flows, fail-node steps in parallel flows are only executed only after the parent step fails. If the parent step passed, the fail-node step will be skipped.