[← Back to Reviews Index](../Stewards%20Reviews%20Index.md)

# Bicep Testing Review — Azure-AI-RAG-CSharp-Semantic-Kernel-Functions

| Field | Value |
|---|---|
| Project | Azure-AI-RAG-CSharp-Semantic-Kernel-Functions |
| Review date | 2026-03-22 |
| Steward | Bicep Testing Steward (ITST) |
| Critical | 3 |
| Notable | 3 |
| Minor | 3 |
| Info | 1 |
| Total | 10 |

---

## 1. Infrastructure Testing Overview

The `infra/` folder contains a multi-module Bicep deployment covering 20 `.bicep` files across the following concerns:

- App Service Plans (Linux and Windows)
- Azure Function App (Python document loader)
- API App Service (.NET 8)
- React Web App (Node 18)
- Azure Cosmos DB for NoSQL (serverless, vector search)
- Azure AI Search (Standard, semantic search)
- Azure OpenAI (GPT-4o, text-embedding-ada-002)
- Azure Storage (blob, queue, table containers)
- Managed Identity (user-assigned, RBAC)
- Monitoring (Log Analytics, Application Insights, dashboard)

The entry point is `infra/main.bicep` (subscription scope). Deployment is driven manually by `infra/deploy.ps1`, a PowerShell script that calls `az deployment sub create`.

There is **no infrastructure testing of any kind** in this repository. No Bicep test files, no linting configuration, no what-if validation, no PSRule/compliance testing, and no CI pipeline. This represents a complete absence of infrastructure testing controls.

---

## 2. Bicep Test Files Assessment

Bicep supports native unit tests via `test` blocks (introduced in Bicep 0.21+). These blocks invoke a module with a specific set of parameters and assert that the template compiles and the ARM output is valid without deploying.

**Findings:**

- No `.bicep` file in the repository contains a `test` block.
- No dedicated `*.biceptest` or `*-test.bicep` files exist anywhere under `infra/`.
- There are no integration test scripts that validate Bicep output (e.g., comparing ARM JSON output against expected shapes).

**Impact:** Any parameter misconfiguration, broken module reference, or invalid resource property combination will not be caught until an actual deployment attempt. This is particularly risky given the 20-module composition in `main.bicep`, where a mis-wired output/input dependency can fail silently until runtime.

---

## 3. Linting Configuration Assessment

The Bicep linter runs automatically when you `build`, `deploy`, or `what-if` a Bicep file, but its behavior can be tuned or enforced through `bicepconfig.json`.

**Findings:**

- No `bicepconfig.json` exists anywhere in the repository.
- Without this file, the linter uses default settings, which means several important rules are set to `warning` (not `error`) or are disabled entirely.
- The following rules — which should be enforced as errors — are not explicitly configured:
  - `no-unused-params`: Not enforced. Several module files accept parameters they always pass through to nested modules, but there is no gate preventing dead parameters from accumulating.
  - `no-hardcoded-env-urls`: Not enforced. The storage module hard-codes a blob URI construction pattern (`'https://${StorageAccountName}.blob.core.windows.net/'`), and the OpenAI output uses `openAiService.properties.endpoint` rather than a hardcoded URL — but the lack of a linting gate means future additions are unguarded.
  - `secure-params-in-nested-deploy`: Not enforced. The `api-app.bicep` passes a `CosmosDb_ConnectionString` app setting with an empty string value; if a real connection string were ever threaded through as a plain `string` parameter rather than `@secure()`, there would be no lint gate to catch it.
  - `no-unused-vars`: Not enforced.

**Note:** The `key-vault.bicep` file exists in `infra/core/security/` but is empty (0 bytes). The linter has no configuration to treat this as an error rather than silently ignoring it.

---

## 4. What-if Validation Assessment

`az deployment what-if` (or the Bicep equivalent `az deployment sub what-if`) produces a preview of all resource changes before they are applied. It is the primary safety gate for infrastructure changes.

**Findings:**

- The deployment script `infra/deploy.ps1` calls `az deployment sub create` directly with no `--what-if` step beforehand.
- There is no CI/CD pipeline file (no `.github/workflows/`, no `.azuredevops/`, no `azure-pipelines.yml`) in the repository.
- What-if validation is therefore never run — changes are applied directly to the target subscription without any preview or approval gate.
- The script also hard-codes `environmentName = "demo"` and `projectName = "chatapi"`, meaning there is no mechanism to differentiate a dev/staging preview from a production deployment at the pipeline level.

---

## 5. PSRule / Compliance Testing Assessment

PSRule for Azure (`ps-rule-azure`) analyses Bicep and ARM templates against the Azure Well-Architected Framework and Azure Security Benchmark rules. It can be run locally or in CI.

**Findings:**

- No `.ps-rule/` folder exists in the repository.
- No `ps-rule.yaml` configuration file exists.
- No PSRule baseline is configured.
- No Azure Policy assignment templates or compliance testing artifacts exist.

**Impact:** Known Azure best-practice violations in the current Bicep code would be caught by PSRule, including:

- Storage account allows shared key access (`allowSharedKeyAccess: true` in `blob-storage-account.bicep`), which PSRule flags under `Azure.Storage.DisableAccessKey`.
- OpenAI account has `publicNetworkAccess: 'Enabled'` with no network restrictions, flagged under `Azure.CognitiveServices.PublicAccess`.
- Role assignments use the deprecated API version `2020-04-01-preview` in multiple modules; PSRule flags deprecated API versions.
- Log Analytics workspace sets `retentionInDays: 30` with no explicit cost/compliance justification; PSRule can flag retention below a minimum for compliance scenarios.

---

## 6. Parameter Validation Assessment

Bicep decorators (`@minLength`, `@maxLength`, `@minValue`, `@maxValue`, `@allowed`) protect deployments against invalid parameter values being silently accepted.

**What is present:**

- `main.bicep` applies `@minLength(1)` / `@maxLength(64)` on `environmentName` and `projectName`, and `@minLength(1)` on `location`. This is the only file with parameter constraints.
- `core/search/search-services.bicep` uses `@allowed` on `hostingMode`, `publicNetworkAccess`, and `semanticSearch`.
- `core/database/cosmos-db/account.bicep` uses `@allowed` on the `kind` parameter.

**What is missing:**

- `core/host/app-service.bicep` — `name` and `location` have no length or format constraints. An overly long name would fail at ARM deployment time with an unhelpful error rather than a clear validation message.
- `core/storage/blob-storage-account.bicep` — `accountName` has no `@minLength(3)` / `@maxLength(24)` / `@description` decorators, even though Azure Storage account names have strict rules (3–24 chars, lowercase alphanumeric only). Violating this causes a deployment failure.
- `core/ai/openai/openai-account.bicep` — `name` and `customSubdomain` have no length constraints. Azure Cognitive Services names must be 2–64 characters.
- `app/loader-function.bicep` and `app/api-app.bicep` — all string parameters (`functionAppName`, `appServicePlanName`, etc.) have no decorators.
- `app/web-app.bicep` — no parameter decorators on any parameter.
- `core/database/cosmos-db/nosql/container.bicep` — `throughput` accepts any `int`, but the valid range for Cosmos DB manual throughput is 400–1,000,000 RU/s. No `@minValue(400)` / `@maxValue(1000000)` decorator.
- `core/monitor/loganalytics.bicep` — no constraints on `name`.

---

## 7. Findings

| Severity | ID | Title | File |
|---|---|---|---|
| 🔴 Critical | ITST-TEST-001 | No Bicep test files exist anywhere in the infra folder | `infra/` (entire folder) |
| 🔴 Critical | ITST-LINT-001 | No `bicepconfig.json` — linting is unconfigured and unenforced | `infra/` (missing file) |
| 🔴 Critical | ITST-WHATIF-001 | No what-if validation before deployment — changes applied directly | `infra/deploy.ps1` |
| 🟡 Notable | ITST-PSRULE-001 | No PSRule for Azure configuration — no compliance testing | `infra/` (missing `.ps-rule/`) |
| 🟡 Notable | ITST-LINT-002 | `key-vault.bicep` is empty (0 bytes) — no lint or content check catches this | `infra/core/security/key-vault.bicep` |
| 🟡 Notable | ITST-LINT-003 | `secure-params-in-nested-deploy` rule not enforced — CosmosDB connection string passed as plain string | `infra/app/api-app.bicep` line 66 |
| 🟢 Minor | ITST-PARAM-001 | Storage account name has no `@minLength(3)` / `@maxLength(24)` constraint | `infra/core/storage/blob-storage-account.bicep` line 1 |
| 🟢 Minor | ITST-PARAM-002 | Cosmos DB container `throughput` has no `@minValue(400)` / `@maxValue(1000000)` constraint | `infra/core/database/cosmos-db/nosql/container.bicep` line 19 |
| 🟢 Minor | ITST-PARAM-003 | Multiple app module parameters have no `@description` or length constraints | `infra/app/api-app.bicep`, `infra/app/loader-function.bicep`, `infra/app/web-app.bicep` |
| ℹ️ Info | ITST-PARAM-004 | `main.bicep` and `search-services.bicep` use `@minLength`, `@maxLength`, and `@allowed` decorators correctly on top-level parameters | `infra/main.bicep`, `infra/core/search/search-services.bicep` |

---

## 8. Recommended Testing Improvements

| Finding | Recommended Action | Priority |
|---|---|---|
| ITST-TEST-001 | Create Bicep test files (e.g., `infra/tests/main.test.bicep`) using native `test` blocks for each key module. Cover at minimum: main deployment scenario, database module, storage module, and OpenAI module. | High |
| ITST-LINT-001 | Add `infra/bicepconfig.json` configuring `no-unused-params`, `no-unused-vars`, `no-hardcoded-env-urls`, and `secure-params-in-nested-deploy` as `"level": "error"`. | High |
| ITST-WHATIF-001 | Add a what-if step to `deploy.ps1` before `az deployment sub create`. Also introduce a CI pipeline (GitHub Actions or Azure Pipelines) that runs `az deployment sub what-if` on every pull request. | High |
| ITST-PSRULE-001 | Add `.ps-rule/` folder with `ps-rule.yaml` referencing the `Azure.Default` baseline. Run `Invoke-PSRule` in CI against all Bicep files in `infra/`. | Medium |
| ITST-LINT-002 | Either implement `key-vault.bicep` with the required resources or remove it. An empty file in the module tree is a gap that will cause deployment failures if ever referenced. | Medium |
| ITST-LINT-003 | Apply `@secure()` decorator to any parameter that will carry secrets (connection strings, API keys). With `bicepconfig.json` enforcing `secure-params-in-nested-deploy`, violations will be caught at lint time. | Medium |
| ITST-PARAM-001 | Add `@minLength(3)` `@maxLength(24)` and `@description` to the `accountName` parameter in `blob-storage-account.bicep`. | Low |
| ITST-PARAM-002 | Add `@minValue(400)` `@maxValue(1000000)` to the `throughput` parameter in `container.bicep`. | Low |
| ITST-PARAM-003 | Add `@description` decorators to all public parameters in app module files. For name parameters, add appropriate `@minLength`/`@maxLength` based on the Azure resource naming limits. | Low |

### Example: Bicep test block (ITST-TEST-001 remediation sketch)

```bicep
// infra/tests/storage.test.bicep
test storageDefaultTest 'core/storage/blob-storage-account.bicep' = {
  params: {
    accountName: 'satestchatapidemo'
    location: 'southcentralus'
    identityName: 'id-chatapi-demo'
  }
}
```

### Example: `bicepconfig.json` (ITST-LINT-001 remediation sketch)

```json
{
  "analyzers": {
    "core": {
      "enabled": true,
      "verbose": false,
      "rules": {
        "no-unused-params": { "level": "error" },
        "no-unused-vars": { "level": "error" },
        "no-hardcoded-env-urls": { "level": "error" },
        "secure-params-in-nested-deploy": { "level": "error" },
        "adminusername-should-not-be-literal": { "level": "error" },
        "no-hardcoded-location": { "level": "warning" }
      }
    }
  }
}
```

### Example: What-if step in `deploy.ps1` (ITST-WHATIF-001 remediation sketch)

```powershell
# Preview changes before applying
az deployment sub what-if `
    --name $deploymentName `
    --location $Location `
    --template-file $templateFile `
    --parameters environmentName=$environmentName projectName=$projectName location=$Location

# Prompt for confirmation before proceeding
$confirm = Read-Host "Review the what-if output above. Type 'yes' to proceed with deployment"
if ($confirm -ne 'yes') { exit 1 }
```

---

## Footer

This review is based on static analysis of Bicep source files and PowerShell deployment scripts as of 2026-03-22. No live Azure deployment was interrogated and no runtime state was inspected. Findings reflect the state of the repository at the time of analysis.

Steward: Bicep Testing Steward (`bicep-testing-steward.md`) — PREFIX `ITST`