Skip to content

MULTIARCH-6023: PowerVS IPI install signal handling to prevent deprovision failures #80917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Neha-dot-Yadav wants to merge 3 commits into
base: main
Choose a base branch
from
+19 −7
Open

MULTIARCH-6023: PowerVS IPI install signal handling to prevent deprovision failures #80917

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ae07ba4
Fix signal handling to ensure metadata.json is copied on timeout
Neha-dot-Yadav Jun 23, 2026
6196dfd
Fix trap double-execution and secret filtering issues
Neha-dot-Yadav Jun 23, 2026
2bb8be2
TEMPORARY: Add 30m timeout and 15m grace period for TERM trap testing
Neha-dot-Yadav Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions ...perator/step-registry/ipi/install/powervs/install/ipi-install-powervs-install-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -704,8 +704,11 @@ function dump_resources() {
fi
}

trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM
trap 'prepare_next_steps' EXIT TERM
# Separate traps for EXIT and TERM to handle cleanup properly
# TERM: Kill children, run cleanup, then exit with proper signal status
# EXIT: Just run cleanup (for normal termination)
trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill -TERM ${CHILDREN} 2>/dev/null; fi; wait; trap - EXIT; prepare_next_steps; exit 143' TERM
trap 'prepare_next_steps' EXIT

if [[ -z "$OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE" ]]; then
echo "OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE is an empty string, exiting"
Expand Down Expand Up @@ -935,8 +938,12 @@ export TF_LOG=debug

echo "8<--------8<--------8<--------8<-------- BEGIN: create cluster 8<--------8<--------8<--------8<--------"
echo "DATE=$(date --utc '+%Y-%m-%dT%H:%M:%S%:z')"
openshift-install --dir="${dir}" create cluster 2>&1 | grep --line-buffered -v 'password\|X-Auth-Token\|UserData:'
ret=${PIPESTATUS[0]}
# Run openshift-install in background with process substitution to allow trap to execute on SIGTERM
# Note: Redirect order matters - stdout first, then stderr to stdout, so both go through grep
openshift-install --dir="${dir}" create cluster > >(grep --line-buffered -v 'password\|X-Auth-Token\|UserData:') 2>&1 &
INSTALL_PID=$!
wait $INSTALL_PID
ret=$?
Comment thread
coderabbitai[bot] marked this conversation as resolved.
echo "ret=${ret}"
if [ ${ret} -gt 0 ]; then
SKIP_WAIT_FOR=false
Expand Down Expand Up @@ -964,8 +971,12 @@ echo "SKIP_WAIT_FOR=${SKIP_WAIT_FOR}"
if ! ${SKIP_WAIT_FOR}; then
echo "8<--------8<--------8<--------8<-------- BEGIN: wait-for install-complete 8<--------8<--------8<--------8<--------"
echo "DATE=$(date --utc '+%Y-%m-%dT%H:%M:%S%:z')"
openshift-install wait-for install-complete --dir="${dir}" | grep --line-buffered -v 'password\|X-Auth-Token\|UserData:'
ret=${PIPESTATUS[0]}
# Run openshift-install in background with process substitution to allow trap to execute on SIGTERM
# Note: Redirect order matters - stdout first, then stderr to stdout, so both go through grep
openshift-install wait-for install-complete --dir="${dir}" > >(grep --line-buffered -v 'password\|X-Auth-Token\|UserData:') 2>&1 &
INSTALL_PID=$!
wait $INSTALL_PID
ret=$?
echo "ret=${ret}"
echo "8<--------8<--------8<--------8<-------- END: wait-for install-complete 8<--------8<--------8<--------8<--------"
fi
Expand Down
3 changes: 2 additions & 1 deletion ci-operator/step-registry/ipi/install/powervs/install/ipi-install-powervs-install-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
ref:
as: ipi-install-powervs-install
from: upi-installer
grace_period: 2h
timeout: 30m0s # Temporary: reduced from default for TERM trap testing

@hamzy hamzy Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you do this?

@prb112 prb112 Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neha forced an error case to test the trap's behavior
She is reverting it.

grace_period: 15m0s # Temporary: reduced from 2h for TERM trap testing
commands: ipi-install-powervs-install-commands.sh
cli: latest
resources:
Expand Down