fix(SDK-4748): honour buildIdentifier precedence in _handleBuildIdentifier - #236
shivam5643 wants to merge 4 commits into
Conversation
…ifier
_handleBuildIdentifier deleted the buildIdentifier whenever
BROWSERSTACK_BUILD_NAME was exported, even though a buildName was present
in the capabilities and the identifier had been configured explicitly.
The clause was ported from the yml-driven SDKs, where it exists only to
suppress the *default* '#${BUILD_NUMBER}' identifier. This service has no
default identifier and never reads BROWSERSTACK_BUILD_NAME as a buildName
source, so the clause could only ever discard user intent.
Three changes, all aligning this service with the SDK-wide precedence
contract (CLI args > env vars > config file > script):
- Skip the identifier only when there is no buildName at all.
- Resolve BROWSERSTACK_BUILD_IDENTIFIER, then BROWSERSTACK_BUILD_RUN_IDENTIFIER,
ahead of the service options / capabilities value. Neither env var was
read before; they appeared only in telemetry payloads.
- Sweep any remaining ${ENV_VAR} placeholder against process.env after the
${DATE_TIME} / ${BUILD_NUMBER} substitutions, leaving unset variables literal.
Mirrors browserstack-node-agent computeBuildIdentifier(),
browserstack-python-sdk ENV_CAPS_TO_CONFIG['buildIdentifier'] and
browserstack-csharp-sdk GetBuildIdentifier().
The unit test named "should delete buildIdentifier if BROWSERSTACK_BUILD_NAME
is defined as env var" asserted the broken behaviour by name only - its caps
carried no buildName, so it passed on the !this._buildName branch regardless.
It is replaced by a non-degenerate pair: one keeping the identifier when a
buildName is present, one still deleting it when it is absent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited) Review profile: ASSERTIVE Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
shivam5643
left a comment
There was a problem hiding this comment.
🔴 Blocking — 1 Critical, 2 Warnings, 2 Suggestions
Recommendation: REQUEST_CHANGES — posted as a COMMENT-event review (GitHub rejects a formal decision on a self-authored PR). Reviewed at head b3ed8ef50eee44717a1b8d9b6a45664ba2f47911. Coverage ledger: 3/3 files judged, 0 gap.
Reviewed jointly with the v8 sibling #237. The two PRs were first reviewed independently, and each pass returned findings the other missed entirely — zero overlap, despite both reporting complete coverage. A diff-of-diffs confirms the
src/launcher.tschanges are identical across the pair apart fromCapabilities.TestrunnerCapabilities(v9) vsCapabilities.RemoteCapabilities(v8), and all 141 added test lines are byte-identical. Every finding below therefore applies to both PRs, and the same unioned set is posted on #237. Please fix in both.
🔴 Critical — env-sourced identifier leaks into the TestOps build-start payload after being suppressed
The new env tier sets the instance field unconditionally whenever either env var is present — including when nothing was configured anywhere:
const envBuildIdentifier = [
process.env.BROWSERSTACK_BUILD_IDENTIFIER,
process.env.BROWSERSTACK_BUILD_RUN_IDENTIFIER
].find((value) => value && value.trim())
if (envBuildIdentifier) {
this._buildIdentifier = envBuildIdentifier.trim()
}The if (!this._buildIdentifier) return guard below it now passes because the env tier just set it. Then the narrowed delete-gate strips the identifier from capabilities but never resets the field:
if (!this._buildName) {
this._updateCaps(capabilities, 'buildIdentifier')
BStackLogger.warn('Skipping buildIdentifier as buildName is not passed.')
return
}The caller then reads that same stale field and forwards it to launchTestSession, which puts it on the build-start POST as build_identifier: bsConfig.buildIdentifier.
The field-staleness itself pre-dates this PR, but it was only reachable if a user had explicitly configured a buildIdentifier. This PR makes it reachable with zero user configuration, because — as this PR's own description notes — CI injects BROWSERSTACK_BUILD_RUN_IDENTIFIER globally.
Failure scenario: a CI job that exports BROWSERSTACK_BUILD_RUN_IDENTIFIER runs a wdio suite with no buildName. The dashboard build name correctly shows no identifier — which is what the new tests check — while O11y's build-start record for that same build carries a non-empty build_identifier that was never applied anywhere visible. A dashboard/telemetry disagreement, on a path this PR turns from rare into common.
Suggested fix: clear the in-memory field in the !this._buildName branch before returning — this._buildIdentifier = undefined alongside the existing _updateCaps(...) call — so what reaches launchTestSession always matches what was actually applied to capabilities.
🟠 Warning — the generic ${ENV_VAR} sweep reprocesses the reserved ${BUILD_NUMBER} token
this._buildIdentifier = this._buildIdentifier.replace(
/\$\{([A-Za-z_][A-Za-z0-9_]*)\}/g,
(match, varName) => process.env[varName] ?? match
)This runs unconditionally after the ${BUILD_NUMBER} block, with no exclusion list. Previously, when getCiInfo() returned null and _getLocalBuildNumber() was falsy, ${BUILD_NUMBER} was left literal. Now the sweep matches that leftover token and substitutes raw process.env.BUILD_NUMBER if set — silently converting an unresolved placeholder into a real value, and one without the 'CI ' prefix that every other resolution path applies.
getCiInfo() recognizes a fixed vendor list that excludes GitHub Actions; TeamCity conventionally exports a bare BUILD_NUMBER with none of the markers it checks. So: identifier '#${BUILD_NUMBER}' on such a CI previously rendered literally, and now renders as #<raw value> in a format inconsistent with every getCiInfo()-resolved build.
This also puts an existing, untouched test at risk — the one that mocks both resolution paths to null and asserts the identifier stays literal. This PR's new afterEach clears only BROWSERSTACK_BUILD_NAME / BROWSERSTACK_BUILD_IDENTIFIER / BROWSERSTACK_BUILD_RUN_IDENTIFIER, not BUILD_NUMBER, so that test's guarantee depends on the ambient environment rather than on the code.
Suggested fix: exclude the already-special-cased tokens from the sweep, and add BUILD_NUMBER to the afterEach cleanup:
const RESERVED_BUILD_IDENTIFIER_TOKENS = new Set(['DATE_TIME', 'BUILD_NUMBER'])
this._buildIdentifier = this._buildIdentifier.replace(
/\$\{([A-Za-z_][A-Za-z0-9_]*)\}/g,
(match, varName) => RESERVED_BUILD_IDENTIFIER_TOKENS.has(varName) ? match : (process.env[varName] ?? match)
)🟠 Warning — the "buildName absent" test asserts only the caps side-effect, not the property its name promises
it('should not set buildIdentifier from env when buildName is absent', async() => {
process.env.BROWSERSTACK_BUILD_RUN_IDENTIFIER = 'per-run-identifier'
const caps: any = [{ 'bstack:options': {} }]
const service = new BrowserstackLauncher(options as any, caps, config)
service._handleBuildIdentifier(caps)
expect(caps[0]['bstack:options']?.buildIdentifier).toBeUndefined()
})The single assertion is satisfied by three different worlds: (a) the intended behavior — env read, then correctly stripped before the caps write; (b) the pre-PR code, which had no env-reading path at all and would pass this assertion unchanged; and (c) the Critical above, where the instance field keeps the env value and only the caps write is suppressed.
None of the new tests assert on service._buildIdentifier after the call, or on what would be forwarded downstream. So the one property this test is named for is the one property nothing here verifies — which is exactly why the Critical was able to ship.
Suggested fix: add expect((service as any)._buildIdentifier).toBeUndefined(), or spy on the launchTestSession boundary to confirm no stale identifier is forwarded.
💡 Suggestions
A blank-but-set env var silently blanks part of the identifier. The sweep's process.env[varName] ?? match guards only nullish, not empty string — so FOO='' erases that piece of the identifier. This contradicts the adjacent doc comment ("An unset variable is left as its literal placeholder rather than blanked, so nothing is silently lost") and is inconsistent with the tier-1 env check a few lines above, which correctly treats whitespace-only as absent via .find((value) => value && value.trim()). One-line fix: (match, varName) => { const v = process.env[varName]; return v && v.trim() ? v : match }.
No coverage for multiremote/object-form capabilities. Every new test uses array-form caps: any = [{ 'bstack:options': {...} }]. _handleBuildIdentifier and _updateCaps both have a distinct branch for the object/multiremote capabilities shape, which the new tiers never exercise. The two branches look structurally parallel so a latent divergence is unlikely, but it's untested — worth one mirrored case given this method is capability-format agnostic by design.
What's good
- The precedence chain is well-commented and explicitly cross-references the equivalent logic in node/python/C# — exactly the cross-SDK consistency this contract is meant to buy. Cross-checked against
browserstack-python-sdk'ssubstitute_env_vars_in_build_identifier(): identical${VAR}regex and the same leave-unresolved-literal semantics. - Strong, genuinely discriminating regression coverage for the actual SDK-4748 bug — separate tests for "keep identifier with BUILD_NAME env + buildName present" vs "still delete when buildName truly absent", correctly distinguishing the narrowed delete-gate from the previous over-broad one. Each of the four-tier precedence tests fails under the pre-fix code.
- Blank/whitespace-only env values correctly ignored at tier 1, and explicitly unit-tested.
- The env read uses plain
process.envand doesn't depend on any CLI-vs-legacy config-normalization layer, so the SDK-7075 class of gap ("the legacy path reads no config env var") does not apply here. - Changeset present, correctly scoped to
@wdio/browserstack-serviceatpatch, release notes accurate. - Proactive, specific disclosure of the paired BStackAutomation break — exact failing assertion, exact injected env var, and the fixture pattern that fixes it.
Merge sequencing
This PR self-discloses that it turns a currently-green BSA lane red (test_wdio_cucumber_wrapper_build_identifier_null.py) because BSA Jenkins injects BROWSERSTACK_BUILD_RUN_IDENTIFIER globally and env outranks yml. The remedy chosen is correct — unset in the BSA fixture rather than weakening the precedence contract — but this PR shouldn't merge or release ahead of that BSA-side change landing.
🤖 Generated with Claude Code
…d tokens
Critical: the !buildName branch stripped buildIdentifier from capabilities but
left this._buildIdentifier set. launchTestSession reads that field for the
build-start payload's build_identifier, so an env-sourced value was reported to
O11y while never being applied to any capability. Pre-existing as a shape, but
the new env tier makes it reachable with zero user configuration, since CI
commonly injects BROWSERSTACK_BUILD_RUN_IDENTIFIER globally. Both the field and
browserStackConfig.buildIdentifier are now cleared alongside the caps update.
Warning: the generic ${ENV_VAR} sweep reprocessed ${BUILD_NUMBER}. When neither
getCiInfo() nor _getLocalBuildNumber() could resolve it the token was left
literal by design, and the sweep then substituted a raw process.env.BUILD_NUMBER
- without the 'CI ' prefix every other path applies. getCiInfo() recognises a
fixed vendor list that excludes GitHub Actions, and TeamCity exports a bare
BUILD_NUMBER, so this was reachable in practice. DATE_TIME and BUILD_NUMBER are
now excluded from the sweep.
Suggestion: `?? match` guarded only nullish, so an exported-but-empty variable
blanked that part of the identifier - contradicting the adjacent comment and
inconsistent with the tier-1 check, which treats whitespace-only as absent.
Empty and whitespace-only values now leave the literal placeholder.
Tests: the "buildName absent" case asserted only the caps side-effect, which the
pre-PR code would also have satisfied; it now asserts _buildIdentifier too. Added
regression tests for the reserved-token and empty-env cases. afterEach now clears
BUILD_NUMBER, so the pre-existing "stays literal" assertions no longer depend on
the ambient environment.
_handleBuildIdentifier block 17 -> 19 tests. Full suite 1347 passed (3 errors
pre-existing in uploadLogsArchive.test.ts, unchanged on a clean tree).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks — all four actionable findings are addressed. Pushed as a single commit on both PRs ( 🔴 Critical — stale identifier reaching the build-start payloadConfirmed and fixed. The Both the instance field and 🟠 Warning —
|
What is this about?
_handleBuildIdentifierdeleted an explicitly configuredbuildIdentifierwheneverBROWSERSTACK_BUILD_NAMEwas present in the environment, and never consultedBROWSERSTACK_BUILD_RUN_IDENTIFIERat all. A user-set identifier silently vanished from the dashboard build name.This is the WDIO half of SDK-4748. The python (browserstack/browserstack-python-sdk#1206, merged) and node (browserstack/browserstack-node-agent#2571) halves fix the equivalent defect in their SDKs.
The bug — in
packages/browserstack-service/src/launcher.ts, the delete fired on an env-supplied build name:Separately,
BROWSERSTACK_BUILD_RUN_IDENTIFIERappeared only in telemetry payloads (src/util.ts,src/testorchestration/test-ordering-server.ts) — never as a source forbuildIdentifier.BROWSERSTACK_BUILD_IDENTIFIERhad zero references anywhere in the repo.The fix — three changes, aligned with the precedence contract used across all BrowserStack SDKs (CLI args > env vars > config file > script):
BROWSERSTACK_BUILD_IDENTIFIER, thenBROWSERSTACK_BUILD_RUN_IDENTIFIER. Blank/whitespace values ignored.if (!this._buildName). The env-buildName clause is removed outright rather than narrowed: unlike python, this service has no default identifier (_handleBuildIdentifierearly-returns when none is configured), so the clause had no legitimate job here. Python's equivalent suppresses only the default#${BUILD_NUMBER}, never an explicitly requested identifier.${ENV_VAR}sweep after the existing${DATE_TIME}/${BUILD_NUMBER}substitutions, mirroring node'scomputeBuildIdentifierand python'ssubstitute_env_vars_in_build_identifier. Unset vars stay literal, so no silent data loss.this.browserStackConfig.buildIdentifieris now set alongside the caps update so funnel telemetry andlaunchTestSessionsee the resolved value.Verification
BStackAutomation App-Automate wdio-cucumber lanes — 6 passed / 0 failed, run against this branch built and linked through BSA's own clone/build/link flow on Node 20.11.1. Clean before/after on the same two lanes:
test_build_identifier_custom_date...customdate_hYBbAQ, no identifiertest_build_identifier_env_variable...envvariable_WKmZlI, no identifiertest_sesion_run_statusx2test_app_automate_cbtx2Identifiers exercised:
custom_date: 2026-09-27_17-37-26andBROWSERSTACK_BUILD_RUN_IDENTIFIER: test_run_20260927_174002, both of which reached the dashboard build name.Unit tests: the
_handleBuildIdentifierblock goes from 7 to 17 cases, covering each precedence tier, blank-env handling, the absent-buildName path and the${ENV_VAR}sweep. Full suite green (3 pre-existing unrelated ENOENT errors inuploadLogsArchive.test.ts, confirmed identical on a clean tree).tsc --noEmitand eslint clean.One misnamed pre-existing test was replaced:
'should delete buildIdentifier if BROWSERSTACK_BUILD_NAME is defined as env var'asserted nothing about the env var — its caps carried nobuildName, so it passed through the!this._buildNamearm and duplicated the test above it.This will turn a currently-green lane red in CI unless BSA is updated in the same window.
SDK/api/tests/automate/wdio_cucumber/wdio_cucumber_wrapper/test_wdio_cucumber_wrapper_build_identifier_null.pyconfiguresbuildIdentifier: Noneand assertsassert not re.search(r'\d+$', fetched_build_name). BSA Jenkins injectsBROWSERSTACK_BUILD_RUN_IDENTIFIERglobally (observed asSDKWdioTestSharded-394). After this change the service honours it even when the config sets none, so the build name ends in digits and the assertion fails.This is correct per the precedence contract — node, python and C# all let an env var beat an explicit yml null — so the remedy belongs in BSA:
unset BROWSERSTACK_BUILD_RUN_IDENTIFIER BROWSERSTACK_BUILD_IDENTIFIERin those fixtures, the pattern already used inSDK/api/run_fixverify_6547.sh. The same shape exists intest_android_wdio_mocha_wrapper_build_identifier_null.py.A matching v8 port is raised separately. After both, the two
_handleBuildIdentifierbodies differ by exactly one line (the capabilities type).Related Jira task/s
Release (mandatory for every PR — required for the
ready-for-reviewlabel)Version bump: (required — tick exactly one)
Release notes type: (optional)
Release notes (customer-facing): (optional but encouraged)
buildIdentifierbeing dropped from the build name whenBROWSERSTACK_BUILD_NAMEwas set via environment variable.BROWSERSTACK_BUILD_RUN_IDENTIFIERandBROWSERSTACK_BUILD_IDENTIFIERare now honoured as build-identifier overrides.${CUSTOM_DATE}inbuildIdentifierare now substituted from the environment.Release notes (internal): (required — engineer-facing; what actually changed / why)
_handleBuildIdentifier(packages/browserstack-service/src/launcher.ts) reworked into an explicit precedence chain (envBROWSERSTACK_BUILD_IDENTIFIER->BROWSERSTACK_BUILD_RUN_IDENTIFIER-> service options/caps), replacing the clause that deleted the identifier wheneverBROWSERSTACK_BUILD_NAMEwas in env.${ENV_VAR}sweep after the existing${DATE_TIME}/${BUILD_NUMBER}substitutions; unresolved placeholders are left literal rather than blanked.Checklist
PR Validations
Run Tests: Comment RUN_TESTS to trigger sanity tests.