Running workflows from the Studio¶
- Started server
- Started and connected Studio
- Started and connected agent that fulfils capabilities required by the workflow
- Required components (e.g. a browser) have to be installed on the machine it is running on
The button to start execution of a workflow
The active tab - the workflow that will be executed
To run a recorded, manually built or saved workflow from the Studio, you need to open the specific workflow or activate its tab if it’s opened already. Now you can run it by simply clicking on the “run”-button in the upper toolbar of the studio.
Before running the workflow(s), Studio will check if there is an agent it can reserved or enough licenses are available. Should the Studio find that it is not able to run this workflow - either because all agents are already in use, no agents are connected or you don’t have enough licenses - you will get a specific error message in the test results toolbar showing you what exactly went wrong.
If one agent (or more - if needed) has been reserved for execution, this workflow will now be run on it. The specific activity that is being executed will be marked by coloured border - so you can know exactly what part of the workflow is being executed.
While the execution is running, you will have no way to edit the workflow inside of the tab that is being executed - neither by editing properties or by trying to record something into it - you have will to wait for the end of execution or manually stop it by clicking on the “stop”-button.
Record errors during execution¶
Web activities have the configurable property to take a screenshot of the browser as they fail. If you have activated this setting and execute a workflow and one of the activities fails, you have the option to export this screenshot via a test report.
Use encrypted input during execution¶
To resolve and use tokens referenced in the workflow, you must be aware of two things:
The password of the database must be stored in the configuration file. The Studio must be connected to the correct server - the server that created the token.
The Studio will check whether this token can be resolved before executing a workflow - if this is not successful, the workflow will not be executed and an error message will appear indicating which token could not be resolved.
- TWS is installed
- Server is started
- Agent is started
- Playlist has been created
- Required applications are installed (e.g. specific browsers)
You can run playlists using the test adapter or the command-line interface.
- How to create playlists?
- You can find this guide here Creating Playlists
- How to configure the test adapter and command line interface?
- You can find this guide here Configuration
The TestWorkflow test adapter runs workflows contained in a project or project collections to automate the running of tests. By using workflow-playlists you can define the workflows that are to be run in later test-runs.
This part discusses how to run a Test Workflow Playlist: locally via the VSTest.Console and automatically using the Team Foundation Server (TFS)
To execute test workflows, a Test Workflow Server must be available, since it is responsible for executing the workflows. Please note that a certain number of agents should always be available so that they can be used for the automated tests.
Execution via VSTest.console¶
Test workflows can be executed locally via the VSTest.Console.exe, which is usually located at the following path:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\VSTest.Console.exe
The path of the Test Adapter and the Test Workflow Playlist must be passed as an argument.
VSTest.Console.exe /TestAdapterPath:<PathToTestAdapter> "C:\TestSolution\TestProject\Tests\bin\Debug\Tests.twp"
In addition, a host name or Uri can be specified on which a test workflow server runs and can be used for test execution.
localhost is used as the hostname.
VSTest.Console.exe /TestAdapterPath:<PathToTestAdapter> "C:\TestSolution\TestProject\bin\Debug\Tests.twp" -- ServerAdress="TWS-Server-01"
Please note that the arguments are not specified directly after a “-“ or “–“ as usual, but are separated by a space character and only mark the separation of arguments.
Both the playlist and the workflows must be copied to the output directory during the build process.
All workflows specified in the playlist are then executed.
If one of the workflows within the playlist contains an encrypted input and therefore a token, it is absolutely necessary that you specify the database password when starting the Test Adapter. To do this, proceed in the same way as for specifying the server address, but do not separate the two arguments with additional “–“.
VSTest.Console.exe /TestAdapterPath:<PathToTestAdapter> "C:\TestSolution\TestProject\bin\Debug\Tests.twp" -- ServerAdress="TWS-Server-01" dbpassword="password"
Also, make sure that you are connected to the correct server - the one that created the tokens, otherwise the workflow cannot be executed.
Execution via the Team Foundation Server¶
The Team Foundation Server build process provides the ability to automatically call the test adapter and execute workflows. To do this, the Visual Studio Test task must be added to the current build definition.
The task is then configured.
A wild card can be used as a test assembly path. All
*.twp files in the specified directory are automatically searched for and executed.
In order for the test execution to start, the task still needs the host name of the TWS server.
The command-line interface offers another way to run workflows contained in a playlist. To run test workflows, the CLI has to be run with arguments.
Always indicate a Workflow-Playlist that contains the workflows you want to execute. (Bsp.: tws-cli.exe -twp C:\user\documents\playlist.twp)
You can specify an address where the Test Workflows Server is running. If you specify no address then the address “localhost” will be used. (Bsp.: tws-cli.exe -server 192.168.0.1)
You can specify the path, at which the test results will be written. The default path is the path at which the tws-cli.exe is located. (Bsp.: tws-cli.exe -o C:\user\documents)
You can specify the name with which the Test Workflows CommandLineInterface registers itself at the Test Workflows Server. (Bsp.: tws-cli.exe -name HITC)
You can specify a timeout in seconds after which the execution of the workflow will be aborted. (Bsp.: tws-cli.exe -timeout 10)
Required if one of the workflows within the playlist references a token. A password for the database is required to resolve and use it during execution.
tws-cli.exe --dbpassword password