Skip to main content
All CollectionsAPI and Continuous Integration (CI)
Continuous integration with Azure Pipelines
Continuous integration with Azure Pipelines
D
Written by Daniel Servidie
Updated over 3 months ago

Usetrace allows you to run traces from your Azure Pipelines, helping to catch issues early in your development process. This guide will walk you through setting up and using Usetrace in your Azure Pipelines projects.

Prerequisites

Configure your pipeline

1. Edit your pipeline instructions in your azure-pipelines/{nameOfYourYMLFile}.yml file:

  • If you have a pipeline already created:

    1. Navigate to your project summary page on Azure DevOps page
      Note: It is located in https://dev.azure.com/{yourOrganizationName}/{yourProjectName}

    2. Click on "Pipelines"

    3. Click on the pipeline you want to trigger the integration

    4. Click "Edit"

      - You will be shown the pipeline editor

    5. Here you will be capable of using our sample YML as a base.

  • If you are creating a new pipeline you will be following steps similar to these:

2. Add instructions to the YML file:
Now that you have the editor open and the repository connected, you can change the YML file and commit those changes directly from here. Copy the code snippet below to your editor, or merge your existent commands with the ones of the snippet:

trigger:
- main

pool:
  name: 'Azure Pipelines'

jobs:
- job: triggerATrace
  steps:
  - script: |
      git clone https://github.com/usetrace/universal-integration.git
      npm install --prefix universal-integration

      # Passing some parameters as env vars
      export INPUT_BUILD_TIMEOUT_SECONDS=900
      export INPUT_FAIL_ON_FAILED_TRACES=false
      
      npm run usetrace --prefix universal-integration -- --traceId {Trace Id} --browsers chrome,firefox
    displayName: 'Runs the integration'

  - publish: $(System.DefaultWorkingDirectory)/output.json
    artifact: output
    displayName: 'Publish the output artifact'


- job: failedExecution
  dependsOn: triggerATrace
  condition: failed('triggerATrace')
  steps:
  - download: current
    artifact: output
    displayName: 'Download integrationOutput artifact'
  
  - script: |
      echo "Contents of output.json:"
      cat $(Pipeline.Workspace)/output/output.json | jq '.status'
      echo THE BUILD FAILED. Taking appropiate measures.
    displayName: 'Failed execution'


- job: successfulExecution
  dependsOn: triggerATrace
  condition: succeeded('triggerATrace')
  steps:
  - download: current
    artifact: output
    displayName: 'Download output artifact'

  - script: |
      echo "Contents of output.json:"
      cat $(Pipeline.Workspace)/output/output.json | jq '.status'
      echo All went as expected
    displayName: 'Successful Execution'

3. Replace {Trace Id} with the actual Trace ID and click Run. You will be prompted regarding creating a commit to your repository. Make any changes you need and click Run again to push that change to your repository.
​

How to find the Id of your Trace

You can find it by selecting the project that contains the Trace you want to trigger in your Usetrace account and then:
​Click All Traces in the top navigation bar
↳ Click on the Trace you want to trigger to open it
↳ Copy the last part of the URL of your browser
​
You will have a URL like this: https://team.usetrace.com/editor/#trace/ZoMvAy6weAAXXcb3jfGGG9QPEvnsqTNY and you need to copy the last part. In this case: ZoMvAy6weAAXXcb3jfGGG9QPEvnsqTNY

How to find the Id of your Project

You can find it by selecting the project in your Usetrace and then:
Click Preferences in the top navigation bar
↳ Click on Project in the left menu
↳ Scroll down and you will find the field Project ID
​
It will look something like this: ZodMfx6ReAAXXca7VhlygnfF8lGO5Pok

4. Now once any change is pushed, the pipeline will be triggered. Azure Pipelines will detect and read the instructions in your YML file. Then the Trace you passed as input (Env var or CLI parameter) will be invoked and you will be able to see the result in the logs.

Inputs

How to pass Inputs

Inputs can be provided to the integration either by command line parameters or by environment variables, or both.

Required Inputs

One of the following parameters must be provided:


To trigger a single Trace:

  • --traceId <id>/INPUT_TRACE_ID: ID of a valid Usetrace Project or Trace to be triggered.

To trigger an entire Project:

  • --projectId <id>/INPUT_PROJECT_ID: ID of a valid Usetrace Project or Trace to be triggered.

Optional Inputs

General Arguments

  • --browsers <list>/INPUT_BROWSERS: Comma-separated list of browsers (e.g., 'chrome, firefox') If none is specified test will run in chrome.

  • --baseUrl <url>/INPUT_BASE_URL: Base URL to execute against (defaults to the project base URL).

  • --parameters <json>/INPUT_PARAMETERS: Object trace parameters. You can pass them as json attributes. Ex: '"key1": "value1", "key2": "value2"'.

  • --usetraceApiKey <key>/INPUT_USETRACE_API_KEY: Usetrace API Key for authentication.

  • --buildTimeoutSeconds <seconds>/INPUT_BUILD_TIMEOUT_SECONDS: Maximum time to wait for the build before timing out the workflow. Default: 3600 seconds (60 minutes).

Workflow Control

  • --failOnFailedTraces/INPUT_FAIL_ON_FAILED_TRACES: Determines whether the workflow should fail if any traces fail. Set to 'true' to fail the workflow if the count of failed traces is not zero, 'false' to always pass the workflow regardless of trace results. Default: 'true'.

Reporter Webhook

  • --webhookUrl <url>/INPUT_WEBHOOK_URL: URL of the POST callback to send the result. If you want a webhook to be invoked when the build finishes, you must include this value.

  • --webhookWhen <option>/INPUT_WEBHOOK_WHEN: Designation of when the webhook should be triggered. Available values: 'always', 'fails' (on failures only), 'changes' (on result changes only). Default: 'always'.

  • --webhookSecretkey/INPUT_WEBHOOK_SECRETKEY: If provided, a HMAC signature will be created and passed via a Signature header to verify the validity of the POST response payload.

  • --webhookUsername <username>/INPUT_WEBHOOK_USERNAME: Username for basic auth if the callback URL is behind an auth wall.

  • --webhookPassword <password>/INPUT_WEBHOOK_PASSWORD: Password for basic auth.

Project-Only Arguments

These arguments only work if you are triggering a project (using a projectId instead of a traceId)

  • --tags <list>/INPUT_TAGS: Comma-separated list of tags. Only traces with those tags will be run (by default run all traces).

  • --commit <commit>/INPUT_COMMIT: Hash of the commit leading to this build.

  • --commitLink <link>/INPUT_COMMIT_LINK: Link to the commit.

Output artifact

The builds generate an artifact in the root folder called output.json with the following fields:
​id: Build ID executed.

status: Status of the run.

request: Count of requested traces.

finish: Count of finished traces.

pass: Count of passed traces.

fail: Count of failed traces.

report: Full JSON report of the build.


Here is an example report:

{
  "name": "Environment: https://www.wikipedia.org/",
  "tests": 2,
  "traces": 1,
  "expectedTracesToPass": 0,
  "tracesPassed": 1,
  "errors": 0,
  "failures": 0,
  "skip": 0,
  "bugs": 0,
  "bugsPassing": 0,
  "buildStable": true,
  "testCase": [
    {
      "className": "Usetrace.trace",
      "name": "chrome: Test Wikipedia",
      "time": 1.192,
      "error": null,
      "browserName": "chrome",
      "traceName": " Test Wikipedia",
      "taggedBug": false,
      "taggedFlaky": true
    },
    {
      "className": "Usetrace.trace",
      "name": "firefox: Test Wikipedia",
      "time": 1.74,
      "error": null,
      "browserName": "firefox",
      "traceName": " Test Wikipedia",
      "taggedBug": false,
      "taggedFlaky": true
    }
  ]
}


You can access all values of the report by loading the content of the report.json artifact generated after the execution of the Traces ends.

Please take a look at the official Usetrace documentation for more detailed information on creating and managing your tests.

Here are some more resources you should check:

Did this answer your question?