Skip to main content

Continuous Integration

Introduction​

copilotbrowser tests can be executed in CI environments. We have created sample configurations for common CI providers.

3 steps to get your tests running on CI:

  1. Ensure CI agent can run browsers: Use our Docker image in Linux agents or install your dependencies using the CLI.

  2. Install copilotbrowser:

    # Install NPM packages
    npm ci

    # Install copilotbrowser browsers and dependencies
    npx copilotbrowser install --with-deps
    pip install copilotbrowser
    copilotbrowser install --with-deps
    mvn exec:java -e -D exec.mainClass=com.microsoft.copilotbrowser.CLI -D exec.args="install --with-deps"
    dotnet build
    pwsh bin/Debug/netX/copilotbrowser.ps1 install --with-deps

  3. Run your tests:

    npx copilotbrowser test
    pytest
    mvn test
    dotnet test

Workers​

We recommend setting workers to "1" in CI environments to prioritize stability and reproducibility. Running tests sequentially ensures each test gets the full system resources, avoiding potential conflicts. However, if you have a powerful self-hosted CI system, you may enable parallel tests. For wider parallelization, consider sharding - distributing tests across multiple CI jobs.

copilotbrowser.config.ts
import { defineConfig, devices } from '@copilotbrowser/copilotbrowser/test';

export default defineConfig({
  // Opt out of parallel tests on CI.
  workers: process.env.CI ? 1 : undefined,
});

CI configurations​

The Command line tools can be used to install all operating system dependencies in CI.

GitHub Actions​

On push/pull_request​

Tests will run on push or pull request on branches main/master. The workflow will install all dependencies, install copilotbrowser and then run the tests. It will also create the HTML report.

.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v5
    - uses: actions/setup-node@v6
      with:
        node-version: lts/*
    - name: Install dependencies
      run: npm ci
    - name: Install copilotbrowser Browsers
      run: npx copilotbrowser install --with-deps
    - name: Run copilotbrowser tests
      run: npx copilotbrowser test
    - uses: actions/upload-artifact@v5
      if: ${{ !cancelled() }}
      with:
        name: copilotbrowser-report
        path: copilotbrowser-report/
        retention-days: 30

On push/pull_request​

Tests will run on push or pull request on branches main/master. The workflow will install all dependencies, install copilotbrowser and then run the tests.

.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v5
    - name: Set up Python
      uses: actions/setup-python@v6
      with:
        python-version: '3.13'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Ensure browsers are installed
      run: python -m copilotbrowser install --with-deps
    - name: Run your tests
      run: pytest --tracing=retain-on-failure
    - uses: actions/upload-artifact@v5
      if: ${{ !cancelled() }}
      with:
        name: copilotbrowser-traces
        path: test-results/
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v5
    - uses: actions/setup-java@v5
      with:
        distribution: 'temurin'
        java-version: '25'
    - name: Build & Install
      run: mvn -B install -D skipTests --no-transfer-progress
    - name: Ensure browsers are installed
      run: mvn exec:java -e -D exec.mainClass=com.microsoft.copilotbrowser.CLI -D exec.args="install --with-deps"
    - name: Run tests
      run: mvn test
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v5
    - name: Setup dotnet
      uses: actions/setup-dotnet@v5
      with:
        dotnet-version: 8.0.x
    - name: Build & Install
      run: dotnet build
    - name: Ensure browsers are installed
      run: pwsh bin/Debug/net8.0/copilotbrowser.ps1 install --with-deps
    - name: Run your tests
      run: dotnet test

On push/pull_request (sharded)​

GitHub Actions supports sharding tests between multiple jobs. Check out our sharding doc to learn more about sharding and to see a GitHub actions example of how to configure a job to run your tests on multiple machines as well as how to merge the HTML reports.

Via Containers​

GitHub Actions support running jobs in a container by using the jobs.<job_id>.container option. This is useful to not pollute the host environment with dependencies and to have a consistent environment for e.g. screenshots/visual regression testing across different operating systems.

.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  copilotbrowser:
    name: 'copilotbrowser Tests'
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
      options: --user 1001
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v6
        with:
          node-version: lts/*
      - name: Install dependencies
        run: npm ci
      - name: Run your tests
        run: npx copilotbrowser test
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  copilotbrowser:
    name: 'copilotbrowser Tests'
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/copilotbrowser/python:v%%VERSION%%-noble
      options: --user 1001
    steps:
      - uses: actions/checkout@v5
      - name: Set up Python
        uses: actions/setup-python@v6
        with:
          python-version: '3.13'
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r local-requirements.txt
          pip install -e .
      - name: Run your tests
        run: pytest
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  copilotbrowser:
    name: 'copilotbrowser Tests'
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/copilotbrowser/java:v%%VERSION%%-noble
      options: --user 1001
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-java@v5
        with:
          distribution: 'temurin'
          java-version: '25'
      - name: Build & Install
        run: mvn -B install -D skipTests --no-transfer-progress
      - name: Run tests
        run: mvn test
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  copilotbrowser:
    name: 'copilotbrowser Tests'
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/copilotbrowser/dotnet:v%%VERSION%%-noble
      options: --user 1001
    steps:
      - uses: actions/checkout@v5
      - name: Setup dotnet
        uses: actions/setup-dotnet@v5
        with:
          dotnet-version: 8.0.x
      - run: dotnet build
      - name: Run your tests
        run: dotnet test

On deployment​

This will start the tests after a GitHub Deployment went into the success state. Services like Vercel use this pattern so you can run your end-to-end tests on their deployed environment.

.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  deployment_status:
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
    - uses: actions/checkout@v5
    - uses: actions/setup-node@v6
      with:
        node-version: lts/*
    - name: Install dependencies
      run: npm ci
    - name: Install copilotbrowser
      run: npx copilotbrowser install --with-deps
    - name: Run copilotbrowser tests
      run: npx copilotbrowser test
      env:
        copilotbrowser_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  deployment_status:
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
    - uses: actions/checkout@v5
      uses: actions/setup-python@v6
      with:
        python-version: '3.13'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Ensure browsers are installed
      run: python -m copilotbrowser install --with-deps
    - name: Run tests
      run: pytest
      env:
        # This might depend on your test-runner
        copilotbrowser_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  deployment_status:
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
    - uses: actions/checkout@v5
    - uses: actions/setup-java@v5
      with:
        distribution: 'temurin'
        java-version: '25'
    - name: Build & Install
      run: mvn -B install -D skipTests --no-transfer-progress
    - name: Install copilotbrowser
      run: mvn exec:java -e -D exec.mainClass=com.microsoft.copilotbrowser.CLI -D exec.args="install --with-deps"
    - name: Run tests
      run: mvn test
      env:
        # This might depend on your test-runner
        copilotbrowser_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  deployment_status:
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
    - uses: actions/checkout@v5
    - name: Setup dotnet
      uses: actions/setup-dotnet@v5
      with:
        dotnet-version: 8.0.x
    - run: dotnet build
    - name: Ensure browsers are installed
      run: pwsh bin/Debug/net8.0/copilotbrowser.ps1 install --with-deps
    - name: Run tests
      run: dotnet test
      env:
        # This might depend on your test-runner
        copilotbrowser_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}

Fail-Fast​

Large test suites can take very long to execute. By executing a preliminary test run with the --only-changed flag, you can run test files that are likely to fail first. This will give you a faster feedback loop and slightly lower CI consumption while working on Pull Requests. To detect test files affected by your changeset, --only-changed analyses your suites' dependency graph. This is a heuristic and might miss tests, so it's important that you always run the full test suite after the preliminary test run.

.github/workflows/copilotbrowser.yml
name: copilotbrowser Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v5
      with:
        # Force a non-shallow checkout, so that we can reference $GITHUB_BASE_REF.
        # See https://github.com/actions/checkout for more details.
        fetch-depth: 0
    - uses: actions/setup-node@v6
      with:
        node-version: lts/*
    - name: Install dependencies
      run: npm ci
    - name: Install copilotbrowser Browsers
      run: npx copilotbrowser install --with-deps
    - name: Run changed copilotbrowser tests
      run: npx copilotbrowser test --only-changed=$GITHUB_BASE_REF
      if: github.event_name == 'pull_request'
    - name: Run copilotbrowser tests
      run: npx copilotbrowser test
    - uses: actions/upload-artifact@v5
      if: ${{ !cancelled() }}
      with:
        name: copilotbrowser-report
        path: copilotbrowser-report/
        retention-days: 30

Docker​

We have a pre-built Docker image which can either be used directly or as a reference to update your existing Docker definitions. Make sure to follow the Recommended Docker Configuration to ensure the best performance.

Azure Pipelines​

For Windows or macOS agents, no additional configuration is required, just install copilotbrowser and run your tests.

For Linux agents, you can use our Docker container with Azure Pipelines support running containerized jobs. Alternatively, you can use Command line tools to install all necessary dependencies.

For running the copilotbrowser tests use this pipeline task:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: UseNode@1
  inputs:
    version: '22'
  displayName: 'Install Node.js'
- script: npm ci
  displayName: 'npm ci'
- script: npx copilotbrowser install --with-deps
  displayName: 'Install copilotbrowser browsers'
- script: npx copilotbrowser test
  displayName: 'Run copilotbrowser tests'
  env:
    CI: 'true'
trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.13'
  displayName: 'Use Python'
- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  displayName: 'Install dependencies'
- script: copilotbrowser install --with-deps
  displayName: 'Install copilotbrowser browsers'
- script: pytest
  displayName: 'Run copilotbrowser tests'
trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: JavaToolInstaller@1
  inputs:
    versionSpec: '25'
    jdkArchitectureOption: 'x64'
    jdkSourceOption: AzureStorage
- script: mvn -B install -D skipTests --no-transfer-progress
  displayName: 'Build and install'
- script: mvn exec:java -e -D exec.mainClass=com.microsoft.copilotbrowser.CLI -D exec.args="install --with-deps"
  displayName: 'Install copilotbrowser browsers'
- script: mvn test
  displayName: 'Run tests'
trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: UseDotNet@2
  inputs:
    packageType: sdk
    version: '8.0.x'
  displayName: 'Use .NET SDK'
- script: dotnet build --configuration Release
  displayName: 'Build'
- script: pwsh bin/Release/net8.0/copilotbrowser.ps1 install --with-deps
  displayName: 'Install copilotbrowser browsers'
- script: dotnet test --configuration Release
  displayName: 'Run tests'

Uploading copilotbrowser-report folder with Azure Pipelines​

This will make the pipeline run fail if any of the copilotbrowser tests fails. If you also want to integrate the test results with Azure DevOps, use the task PublishTestResults task like so:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: UseNode@1
  inputs:
    version: '22'
  displayName: 'Install Node.js'

- script: npm ci
  displayName: 'npm ci'
- script: npx copilotbrowser install --with-deps
  displayName: 'Install copilotbrowser browsers'
- script: npx copilotbrowser test
  displayName: 'Run copilotbrowser tests'
  env:
    CI: 'true'
- task: PublishTestResults@2
  displayName: 'Publish test results'
  inputs:
    searchFolder: 'test-results'
    testResultsFormat: 'JUnit'
    testResultsFiles: 'e2e-junit-results.xml'
    mergeTestResults: true
    failTaskOnFailedTests: true
    testRunTitle: 'My End-To-End Tests'
  condition: succeededOrFailed()
- task: PublishPipelineArtifact@1
  inputs:
    targetPath: copilotbrowser-report
    artifact: copilotbrowser-report
    publishLocation: 'pipeline'
  condition: succeededOrFailed()

Note: The JUnit reporter needs to be configured accordingly via

import { defineConfig } from '@copilotbrowser/copilotbrowser/test';

export default defineConfig({
  reporter: [['junit', { outputFile: 'test-results/e2e-junit-results.xml' }]],
});

in copilotbrowser.config.ts.

Azure Pipelines (sharded)​

trigger:
- main

pool:
  vmImage: ubuntu-latest

strategy:
  matrix:
    chromium-1:
      project: chromium
      shard: 1/3
    chromium-2:
      project: chromium
      shard: 2/3
    chromium-3:
      project: chromium
      shard: 3/3
    firefox-1:
      project: firefox
      shard: 1/3
    firefox-2:
      project: firefox
      shard: 2/3
    firefox-3:
      project: firefox
      shard: 3/3
    webkit-1:
      project: webkit
      shard: 1/3
    webkit-2:
      project: webkit
      shard: 2/3
    webkit-3:
      project: webkit
      shard: 3/3
steps:
- task: UseNode@1
  inputs:
    version: '22'
  displayName: 'Install Node.js'

- script: npm ci
  displayName: 'npm ci'
- script: npx copilotbrowser install --with-deps
  displayName: 'Install copilotbrowser browsers'
- script: npx copilotbrowser test --project=$(project) --shard=$(shard)
  displayName: 'Run copilotbrowser tests'
  env:
    CI: 'true'

Azure Pipelines (containerized)​

trigger:
- main

pool:
  vmImage: ubuntu-latest
container: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble

steps:
- task: UseNode@1
  inputs:
    version: '22'
  displayName: 'Install Node.js'

- script: npm ci
  displayName: 'npm ci'
- script: npx copilotbrowser test
  displayName: 'Run copilotbrowser tests'
  env:
    CI: 'true'
trigger:
- main

pool:
  vmImage: ubuntu-latest
container: mcr.microsoft.com/copilotbrowser/python:v%%VERSION%%-noble

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.13'
  displayName: 'Use Python'

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  displayName: 'Install dependencies'
- script: pytest
  displayName: 'Run tests'
trigger:
- main

pool:
  vmImage: ubuntu-latest
container: mcr.microsoft.com/copilotbrowser/java:v%%VERSION%%-noble

steps:
- task: JavaToolInstaller@1
  inputs:
    versionSpec: '25'
    jdkArchitectureOption: 'x64'
    jdkSourceOption: AzureStorage

- script: mvn -B install -D skipTests --no-transfer-progress
  displayName: 'Build and install'
- script: mvn test
  displayName: 'Run tests'
trigger:
- main

pool:
  vmImage: ubuntu-latest
container: mcr.microsoft.com/copilotbrowser/dotnet:v%%VERSION%%-noble

steps:
- task: UseDotNet@2
  inputs:
    packageType: sdk
    version: '8.0.x'
  displayName: 'Use .NET SDK'

- script: dotnet build --configuration Release
  displayName: 'Build'
- script: dotnet test --configuration Release
  displayName: 'Run tests'

CircleCI​

Running copilotbrowser on CircleCI is very similar to running on GitHub Actions. In order to specify the pre-built copilotbrowser Docker image, simply modify the agent definition with docker: in your config like so:

executors:
  pw-noble-development:
    docker:
      - image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
executors:
  pw-noble-development:
    docker:
      - image: mcr.microsoft.com/copilotbrowser/python:v%%VERSION%%-noble
executors:
  pw-noble-development:
    docker:
      - image: mcr.microsoft.com/copilotbrowser/java:v%%VERSION%%-noble
executors:
  pw-noble-development:
    docker:
      - image: mcr.microsoft.com/copilotbrowser/dotnet:v%%VERSION%%-noble

Note: When using the docker agent definition, you are specifying the resource class of where copilotbrowser runs to the 'medium' tier here. The default behavior of copilotbrowser is to set the number of workers to the detected core count (2 in the case of the medium tier). Overriding the number of workers to greater than this number will cause unnecessary timeouts and failures.

Sharding in CircleCI​

Sharding in CircleCI is indexed with 0 which means that you will need to override the default parallelism ENV VARS. The following example demonstrates how to run copilotbrowser with a CircleCI Parallelism of 4 by adding 1 to the CIRCLE_NODE_INDEX to pass into the --shard cli arg.

  copilotbrowser-job-name:
    executor: pw-noble-development
    parallelism: 4
    steps:
      - run: SHARD="$((${CIRCLE_NODE_INDEX}+1))"; npx copilotbrowser test --shard=${SHARD}/${CIRCLE_NODE_TOTAL}

Jenkins​

Jenkins supports Docker agents for pipelines. Use the copilotbrowser Docker image to run tests on Jenkins.

pipeline {
   agent { docker { image 'mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble' } }
   stages {
      stage('e2e-tests') {
         steps {
            sh 'npm ci'
            sh 'npx copilotbrowser test'
         }
      }
   }
}
pipeline {
   agent { docker { image 'mcr.microsoft.com/copilotbrowser/python:v%%VERSION%%-noble' } }
   stages {
      stage('e2e-tests') {
         steps {
            sh 'pip install -r requirements.txt'
            sh 'pytest'
         }
      }
   }
}
pipeline {
   agent { docker { image 'mcr.microsoft.com/copilotbrowser/java:v%%VERSION%%-noble' } }
   stages {
      stage('e2e-tests') {
         steps {
            sh 'mvn -B install -D skipTests --no-transfer-progress'
            sh 'mvn test'
         }
      }
   }
}
pipeline {
   agent { docker { image 'mcr.microsoft.com/copilotbrowser/dotnet:v%%VERSION%%-noble' } }
   stages {
      stage('e2e-tests') {
         steps {
            sh 'dotnet build'
            sh 'dotnet test'
         }
      }
   }
}

Bitbucket Pipelines​

Bitbucket Pipelines can use public Docker images as build environments. To run copilotbrowser tests on Bitbucket, use our public Docker image (see Dockerfile).

image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
image: mcr.microsoft.com/copilotbrowser/python:v%%VERSION%%-noble
image: mcr.microsoft.com/copilotbrowser/java:v%%VERSION%%-noble
image: mcr.microsoft.com/copilotbrowser/dotnet:v%%VERSION%%-noble

GitLab CI​

To run copilotbrowser tests on GitLab, use our public Docker image (see Dockerfile).

stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
  script:
  ...
stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/copilotbrowser/python:v%%VERSION%%-noble
  script:
  ...
stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/copilotbrowser/java:v%%VERSION%%-noble
  script:
  ...
stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/copilotbrowser/dotnet:v%%VERSION%%-noble
  script:
  ...

Sharding​

GitLab CI supports sharding tests between multiple jobs using the parallel keyword. The test job will be split into multiple smaller jobs that run in parallel. Parallel jobs are named sequentially from job_name 1/N to job_name N/N.

stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
  parallel: 7
  script:
    - npm ci
    - npx copilotbrowser test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

GitLab CI also supports sharding tests between multiple jobs using the parallel:matrix option. The test job will run multiple times in parallel in a single pipeline, but with different variable values for each instance of the job. In the example below, we have 2 PROJECT values and 10 SHARD values, resulting in a total of 20 jobs to be run.

stages:
  - test

tests:
  stage: test
  image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
  parallel:
    matrix:
      - PROJECT: ['chromium', 'webkit']
        SHARD: ['1/10', '2/10', '3/10', '4/10', '5/10', '6/10', '7/10', '8/10', '9/10', '10/10']
  script:
    - npm ci
    - npx copilotbrowser test --project=$PROJECT --shard=$SHARD

Google Cloud Build​

To run copilotbrowser tests on Google Cloud Build, use our public Docker image (see Dockerfile).

steps:
- name: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
  script: 
  ...
  env:
  - 'CI=true'

Drone​

To run copilotbrowser tests on Drone, use our public Docker image (see Dockerfile).

kind: pipeline
name: default
type: docker

steps:
  - name: test
    image: mcr.microsoft.com/copilotbrowser:v%%VERSION%%-noble
    commands:
      - npx copilotbrowser test

Caching browsers​

Caching browser binaries is not recommended, since the amount of time it takes to restore the cache is comparable to the time it takes to download the binaries. Especially under Linux, operating system dependencies need to be installed, which are not cacheable.

If you still want to cache the browser binaries between CI runs, cache these directories in your CI configuration, against a hash of the copilotbrowser version.

Debugging browser launches​

copilotbrowser supports the DEBUG environment variable to output debug logs during execution. Setting it to cb:browser is helpful while debugging Error: Failed to launch browser errors.

DEBUG=cb:browser npx copilotbrowser test
DEBUG=cb:browser pytest
DEBUG=cb:browser mvn test
DEBUG=cb:browser dotnet test

Running headed​

By default, copilotbrowser launches browsers in headless mode. See in our Running tests guide how to run tests in headed mode.

On Linux agents, headed execution requires Xvfb to be installed. Our Docker image and GitHub Action have Xvfb pre-installed. To run browsers in headed mode with Xvfb, add xvfb-run before the actual command.

xvfb-run npx copilotbrowser test
xvfb-run pytest
xvfb-run mvn test
xvfb-run dotnet test