cosmos-explorer/test/README.md

End-to-End Test Suite

This directory contains end-to-end tests for Cosmos Data Explorer. These tests require that you either deploy, or have access to, several Cosmos test Accounts. The tests run in Playwright, using the official Playwright test framework.

Required Resources

To run all the tests, you need:

• A CosmosDB Account using the Cassandra API
• A CosmosDB Account using the Gremlin API
• A CosmosDB Account using the MongoDB API, API version 6.0
• A CosmosDB Account using the MongoDB API, API version 3.2
• A CosmosDB Account using the NoSQL API
• A CosmosDB Account using the Tables API

Each Account must have at least 1000 RU/s of throughput available for new databases/collections/etc. The tests create new databases/keyspaces/etc. for each test, and delete them when the test is done. So it should be safe to use these accounts for other testing purposes, as long as you make sure to have enough throughput available when running the tests.

You can specify the resource to use using the Environment Variables configuration below. Or, you can deploy resources specifically for testing using the deploy script.

Using the deploy script

Note

This script currently only works on Windows.

The resources directory contains a deploy.ps1 script that will deploy the required resources for testing. They use a Bicep template to deploy the resources. All you need to provide is a resource group to deploy in to.

To use this script, there are a few prerequisites that must be done at least once:

1. This script requires Powershell 7+. Install it here.
2. Install Azure PowerShell if you don't already have it.
3. Install Bicep CLI if it is not already installed.
4. Connect to your Azure account using Connect-AzAccount.
5. Ensure you have a Resource Group ready to deploy into, the deploy script requires an existing resource group. This resource group should be named [username]-e2e-testing, where [username] is your Windows username, (Microsoft employees: This should be your alias). The easiest way to do this is by running the create-resource-group.ps1 script, specifying the Subscription (Name or ID) and Location in which you want to create the Resource Group. For example:

.\test\resources\create-resource-group.ps1 -SubscriptionName "My Subscription" -Location "West US 3"

Then, whenever you want to create/update the resources, you can run the deploy.ps1 script in the resources directory. As long as you're using the default naming convention ([username]-e2e-testing), you just need to specify the Subscription. For example:

.\test\resources\deploy.ps1 -SubscriptionName "My Subscription"

You'll get a confirmation prompt before anything is deployed:

Found a resource group with the default name (ashleyst-e2e-testing). Using that resource group. If you want to use a different resource group, specify it as a parameter.
Deploying test resource sets: tables cassandra gremlin mongo mongo32 sql
  in West US 3
  to resource group ashleyst-e2e-testing
  in subscription ... (...)
Do you want to continue? (y/n):

This prompt shows:

• The resources that will be deployed, in this case, all of them. You can filter to deploy only a subset by specifying the -ResourceTypes parameter. For example -ResourceTypes @("cassandra", "sql").
• The location the resources will be deployed to, West US 3 in this case.
• The resource group that will be used, ashleyst-e2e-testing in this case.
• The subscription that will be used.

Once you confirm, the resources will be deployed using Azure PowerShell and the Bicep templates in the resources directory. The script will wait for all the deployments to complete before exiting.

You can re-run this script at any time to update the resources, if the Bicep templates have changed.

Preparing the test environment

Before running the tests, you need to configure your environment to specify the accounts to use for testing. The following environment variables are used:

• DE_TEST_RESOURCE_GROUP - The resource group to use for testing. This should be the same resource group that the resources were deployed to.
• DE_TEST_SUBSCRIPTION_ID - The subscription ID to use for testing. This should be the same subscription that the resources were deployed to.
• DE_TEST_ACCOUNT_PREFIX - If you used the default naming scheme provided by the deploy.ps1 script, this should be your Windows username (or whatever value you passed in for the -ResourcePrefix argument when deploying). This is used to find the accounts that were deployed.

In the event you didn't use the deploy.ps1 script, you can specify the accounts directly using the following environment variables:

• DE_TEST_ACCOUNT_NAME_CASSANDRA - The name of the CosmosDB Account using the Cassandra API.
• DE_TEST_ACCOUNT_NAME_GREMLIN - The name of the CosmosDB Account using the Gremlin API.
• DE_TEST_ACCOUNT_NAME_MONGO - The name of the CosmosDB Account using the MongoDB API, API version 6.0.
• DE_TEST_ACCOUNT_NAME_MONGO32 - The name of the CosmosDB Account using the MongoDB API, API version 3.2.
• DE_TEST_ACCOUNT_NAME_SQL - The name of the CosmosDB Account using the NoSQL API.
• DE_TEST_ACCOUNT_NAME_TABLES - The name of the CosmosDB Account using the Tables API.

If you used all the standard deployment scripts and naming scheme, you can set these environment variables using the following command:

.\test\scripts\set-test-accounts.ps1

If Azure Powershell's current subscription is not the one you want to use for testing, you can set the subscription using the following command:

.\test\scripts\set-test-subscription.ps1 -Subscription "My Subscription"

That script will confirm the resource group exists and then set the necessary environment variables:

The currently-selected subscription is ... (...)
Do you want to use this subscription? (y/n): y

Found a resource with the default resource prefix (ashleyst-e2e-). Configuring that prefix for E2E testing.
Configuring for E2E Testing
  Subscription: ... (...)
  Resource Group: ashleyst-e2e-testing
  Resource Prefix: ashleyst-e2e-
    Found CosmosDB Account: ashleyst-e2e-cassandra
    Found CosmosDB Account: ashleyst-e2e-gremlin
    Found CosmosDB Account: ashleyst-e2e-mongo
    Found CosmosDB Account: ashleyst-e2e-mongo32
    Found CosmosDB Account: ashleyst-e2e-sql
    Found CosmosDB Account: ashleyst-e2e-tables

Running the tests

If Azure CLI is not installed, please install it.

Log into Az CLI with the following command:

az login --scope https://management.core.windows.net//.default

To run all tests in a headless browser, run the following command from the root of the repo:

npx playwright test

Note

You may be prompted to install the Playwright browsers the first time you run the tests.

The tests will use your existing server if you have one running, or start a new server if you don't.

When running individual tests, you may find it most useful to use the Playwright UI to run the tests. You can do this by running the following command:

npx playwright test --ui

The UI allows you to select a specific test to run and to see the results of the test in the browser.

See the Playwright docs for more information on running tests.