cosmos-explorer/scripts/TEST_SCENARIOS.md

Cosmos DB Connectivity Diagnostic - Test Scenarios

Overview

This document defines test scenarios, expected outcomes, and validation procedures for the diagnostic script. Use these to verify script functionality across different network configurations.

---

Test Infrastructure Setup

Prerequisites

• Test Cosmos DB accounts in multiple configurations:
  • Public endpoint only
  • Private endpoint only
  • Both public + private endpoints
• Test networks:
  • Clean network (no corporate proxy/VPN)
  • Behind corporate proxy
  • Behind VPN (if possible)
  • Restricted network (firewall blocking 443)

---

Test Scenarios

Scenario 1: Healthy Public Endpoint (All Checks Pass)

Setup:

• Cosmos account with public endpoint enabled
• Running from clean network (no VPN/proxy)
• Azure CLI authenticated (optional)

Run:

.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-public-01.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-public-01"

Expected Results:

• ✅ DNS resolution: succeeded = true
• ✅ TCP connectivity: succeeded = true
• ✅ HTTPS probe: statusCode = 401 (expected without auth)
• ✅ Private network: isPrivateRange = false
• ✅ Classification: status = "success", code = "network_connectivity_healthy"

Validation Checklist:

• Console shows "✓ PASS" for DNS and TCP
• Recommended Actions mention checking RBAC/auth
• JSON file created successfully
• Latency values are reasonable (< 1000ms)

---

Scenario 2: DNS Resolution Failure

Setup:

• Network with DNS resolver that blocks documents.azure.com
• OR simulate by providing invalid hostname

Run:

.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://invalid-account-xyz123.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "invalid-account"

Expected Results:

• ❌ DNS resolution: succeeded = false, error = "No such host is known"
• ❌ TCP connectivity: succeeded = false
• ❌ Classification: status = "failure", code = "dns_resolution_failed"

Validation Checklist:

• Console shows "✗ FAIL" for DNS
• Error message is clear
• Root cause in classification mentions DNS/VPN/proxy
• Recommended actions include running manual nslookup
• JSON contains error details

---

Scenario 3: TCP Blocked (Public Endpoint)

Setup:

• Network with firewall blocking outbound port 443 to documents.azure.com
• DNS resolves successfully but TCP fails

Run:

.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-public-02.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-public-02"

Expected Results:

• ✅ DNS resolution: succeeded = true
• ❌ TCP connectivity: succeeded = false, error = "Connection timeout after 5000ms"
• ❌ HTTPS probe: statusCode = null, error contains "timeout"
• ❌ Private network: isPrivateRange = false
• ❌ Classification: status = "failure", code = "tcp_connectivity_blocked"

Validation Checklist:

• DNS shows success, TCP shows timeout
• Console summary distinguishes DNS success from TCP failure
• Root cause mentions firewall/ISP/proxy
• Recommended actions include corporate network contact
• Timeout latency is approximately 5000ms

---

Scenario 4: Healthy Private Endpoint

Setup:

• Cosmos account with private endpoint configured
• Client connected to VPN that can route to PE
• PE IP known and provided

Run:

.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-private-01.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-private-01" `
  -PrivateEndpointIP "10.123.171.30"

Expected Results:

• ✅ DNS resolution: succeeded = true, addresses = ["10.123.171.30"]
• ✅ TCP connectivity: succeeded = true
• ✅ Private network: isPrivateRange = true, matchesExpectedPrivateEndpoint = true
• ✅ Azure config: publicNetworkAccessRestricted = true (if checked)
• ✅ Classification: status = "success", code = "network_connectivity_healthy"

Validation Checklist:

• DNS resolves to private IP (10.x)
• TCP succeeds to private IP
• Indicators correctly identify private endpoint
• Expected PE IP matches resolved IP
• Classification recognizes healthy private path

---

Scenario 5: Private Endpoint Network Path Blocked

Setup:

• Private endpoint configured
• Client on VPN but routing to PE subnet is blocked
• DNS resolves to PE IP but TCP times out

Run:

.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-private-02.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-private-02" `
  -PrivateEndpointIP "10.123.171.30"

Expected Results:

• ✅ DNS resolution: succeeded = true, addresses = ["10.123.171.30"]
• ❌ TCP connectivity: succeeded = false, error = "Connection timeout after 5000ms"
• ✅ Private network: isPrivateRange = true, matchesExpectedPrivateEndpoint = true, vpnRouteWarning != null
• ❌ Classification: status = "failure", code = "private_endpoint_network_path_blocked"

Validation Checklist:

• DNS resolves to expected PE IP
• TCP to PE IP fails with timeout
• VPN route warning is populated
• Classification correctly identifies PE path issue
• Recommended actions mention network team + routing
• Source IP is captured (if available)

---

Scenario 6: RBAC Insufficient

Setup:

• Network connectivity is working
• Azure CLI authenticated as user with limited RBAC (e.g., only Reader role)
• Account queried successfully

Run:

az login  # Login as limited user first
.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-rbac-01.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-rbac-01"

Expected Results:

• ✅ DNS resolution: succeeded = true
• ✅ TCP connectivity: succeeded = true
• ✅ HTTPS probe: statusCode = 401 or 200
• ❌ RBAC: classification = "insufficient", canReadAccount = false
• ⚠️ Classification: status = "warning", code = "rbac_insufficient"

Validation Checklist:

• Network checks all pass
• RBAC assessment shows limited permissions
• Classification code is rbac_insufficient
• Recommended actions mention role assignment
• Error message explains what permissions are missing

---

Scenario 7: Azure CLI Not Authenticated

Setup:

• All network checks work fine
• Azure CLI not installed OR not authenticated

Run:

# Without running az login first
.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-public-03.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-public-03"

Expected Results:

• ✅ DNS resolution: succeeded = true
• ✅ TCP connectivity: succeeded = true
• ⚠️ Azure CLI: authenticated = false, error = "Not authenticated with Azure CLI. Run 'az login' to proceed."
• ⚠️ Azure config: checked = false, error = "Skipped"
• ⚠️ Classification: May reference azure_config_check_skipped in warnings

Validation Checklist:

• Network checks complete normally
• Azure CLI context shows unauthenticated
• Console warning mentions az login
• Recommended actions suggest re-running after authentication
• Script doesn't crash; gracefully continues

---

Scenario 8: Interactive Mode Input Flow

Setup:

• User runs script with -Interactive flag
• Has all inputs ready

Run:

.\Diagnose-CosmosConnectivity.ps1 -Interactive

Expected Sequence:

1. Show input instructions with Portal navigation guide
2. Prompt: "Endpoint URL (e.g., https://my-cosmos.documents.azure.com)"
3. Validate input format; re-prompt if invalid
4. Prompt: "Subscription ID (12345678-...)"
5. Validate GUID format; re-prompt if invalid
6. Prompt: "Resource Group name"
7. Prompt: "Account Name"
8. Prompt: "Private Endpoint IP (optional, press Enter to skip)"
9. Prompt: "VPN Subnet Range (optional, press Enter to skip)"
10. Run diagnostics
11. Display results

Validation Checklist:

• Input instructions are clear and helpful
• Format validation rejects invalid inputs
• Optional fields can be skipped (Enter key)
• All inputs accepted without error
• Diagnostics run successfully after inputs collected

---

Scenario 9: Non-Interactive with Redaction

Setup:

• Run with -Redact flag
• Collect JSON output

Run:

$json = .\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-public-04.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-public-04" `
  -Redact 2>&1 | Select-String -Pattern '^\{' -SimpleMatch | ConvertFrom-Json

Expected Results:

• ✅ JSON output completes successfully
• ✅ Target section: subscriptionId = "REDACTED-SUBSCRIPTION-ID"
• ✅ Target section: resourceGroup = "REDACTED"
• ✅ Target section: accountName = "REDACTED"
• ✅ Hostname is NOT redacted (needed for triage): hostname = "test-public-04.documents.azure.com"
• ✅ Azure CLI: currentUser = "REDACTED-USER-NAME"
• ✅ Azure CLI: currentTenant = "REDACTED-TENANT-ID"

Validation Checklist:

• Sensitive fields masked as "REDACTED-*"
• Hostname NOT masked
• JSON still parseable
• Redaction doesn't break classification
• All RBAC role names preserved (not redacted)

---

Scenario 10: Private Endpoint IP Mismatch

Setup:

• Private endpoint exists but expected IP is different from resolved IP
• Can happen if PE reconfigured or DNS zone stale

Run:

.\Diagnose-CosmosConnectivity.ps1 `
  -EndpointUrl "https://test-private-03.documents.azure.com" `
  -SubscriptionId "12345678-1234-1234-1234-123456789012" `
  -ResourceGroup "test-cosmos-rg" `
  -AccountName "test-private-03" `
  -PrivateEndpointIP "10.123.171.99"  # Expected IP (not matching actual)

Expected Results (if actual PE IP is 10.123.171.30):

• ✅ DNS resolution: succeeded = true, addresses = ["10.123.171.30"]
• ✅ TCP connectivity: succeeded = true (connects to actual PE)
• ⚠️ Private network: matchesExpectedPrivateEndpoint = false, indicators contains "WARNING: Resolved to 10.123.171.30 but expected ..."
• ⚠️ Classification: May include private_endpoint_mismatch warning

Validation Checklist:

• Mismatch detected
• Warning includes both expected and actual IPs
• TCP still attempts with actual resolved IP
• Classification identifies discrepancy
• Recommended actions mention checking PE config

---

Scenario 11: Latency Metrics

Setup:

• Healthy connection
• Measure and log latency values

Run:

$json = .\Diagnose-CosmosConnectivity.ps1 -EndpointUrl "..." -SubscriptionId "..." ... 2>&1 | 
  Select-String -Pattern '^\{' | ConvertFrom-Json
$json.diagnostics.dns.latencyMs
$json.diagnostics.tcp.latencyMs
$json.diagnostics.https.latencyMs

Expected Results:

• DNS latency: 10-100ms (typical)
• TCP latency: 20-200ms (depends on network)
• HTTPS latency: 50-500ms (full round trip)
• All values > 0 and < 10000 (reasonable)

Validation Checklist:

• Latency values are integers (milliseconds)
• Values are reasonable for network conditions
• No values are unrealistic (0 or > 60000)
• Timeouts show latencyMs = 0

---

Scenario 12: Multiple Endpoints (Batch Testing)

Setup:

• Multiple accounts to test
• Non-interactive batch mode

Run:

$accounts = @(
  @{Url="https://account1.documents.azure.com"; Sub="..."; RG="rg1"; Name="account1"},
  @{Url="https://account2.documents.azure.com"; Sub="..."; RG="rg2"; Name="account2"},
  @{Url="https://account3.documents.azure.com"; Sub="..."; RG="rg3"; Name="account3"}
)

$results = @()
foreach ($acct in $accounts) {
  $json = .\Diagnose-CosmosConnectivity.ps1 @acct 2>&1 | 
    Select-String -Pattern '^\{' | ConvertFrom-Json
  $results += @{
    Account = $acct.Name
    Classification = $json.classification.code
    DNS = $json.diagnostics.dns.succeeded
    TCP = $json.diagnostics.tcp.succeeded
  }
}
$results | Format-Table

Expected Results:

• All accounts processed without error
• JSON output captured for each
• Results table shows aggregated status
• Classification codes vary based on network conditions

Validation Checklist:

• Batch processing completes
• All JSON files created
• No cross-account contamination
• Timestamp differs for each run

---

Regression Test Checklist

Use this checklist before each release:

• Script Execution

  • Interactive mode completes
  • Non-interactive mode with all parameters
  • Redaction flag works
  • Help/documentation displays correctly

• Network Diagnostics

  • DNS resolution succeeds on good network
  • DNS resolution fails on blocked network
  • TCP succeeds on open port
  • TCP times out on blocked port
  • HTTPS probe returns status code

• Private Endpoints

  • Detects private IP ranges correctly
  • Compares against expected PE IP
  • Handles PE IP mismatches gracefully

• Azure Integration

  • Works with authenticated Azure CLI
  • Gracefully handles unauthenticated state
  • Queries account config successfully
  • RBAC assessment runs

• JSON Output

  • Valid JSON syntax
  • All expected fields present
  • Field values are correct types
  • Redacted fields are properly masked

• Classification

  • Success code for healthy network
  • DNS failure code for DNS issues
  • TCP failure code for blocked ports
  • PE path blocked code for PE issues
  • RBAC code for permission issues

• Documentation

  • Recommended actions are actionable
  • Error messages are helpful
  • Output is readable and organized

• Edge Cases

  • Invalid URL format rejected
  • Invalid GUID format rejected
  • Timeout handling works
  • No unhandled exceptions

---

Performance Expectations

Operation	Expected Time	Timeout
DNS resolution	10-100ms	5000ms
TCP connect	20-200ms	5000ms
HTTPS probe	50-500ms	5000ms
Azure CLI queries	1-5 seconds	10000ms
Full script (good network)	10-20 seconds	N/A
Full script (blocked port)	~5 seconds	N/A

---

Success Criteria

A test scenario passes if:

1. ✅ Script completes without unhandled exceptions
2. ✅ JSON output is valid and contains all expected fields
3. ✅ Classification code matches expected scenario
4. ✅ Recommended actions are relevant to the issue
5. ✅ Latency values are reasonable
6. ✅ Redaction (if enabled) properly masks sensitive fields

---

Sign-Off

QA Tester: _________________ Date: _________

Reviewed By: _________________ Date: _________

Approved for Release: _________________ Date: _________

---

Version

• Script Version: 1.0.0
• Test Plan Version: 1.0.0
• Last Updated: 2026-05-13