Skip to content

nut-scanner: balance worker admission and cleanup - #3649

Open
user01010111 wants to merge 9 commits into
networkupstools:masterfrom
user01010111:fix/scanner-admission-cleanup
Open

user01010111 wants to merge 9 commits into
networkupstools:masterfrom
user01010111:fix/scanner-admission-cleanup

Conversation

@user01010111

@user01010111 user01010111 commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Disabled protocol limits still reached a protocol semaphore, and partial admission or worker-start failures could lose permits. Allocate thread tracking before starting a worker, balance acquired permits on failure, and retain NUT hostname and authentication data until its worker finishes. The existing batch scheduling and address-skipping behaviour after failed thread creation are retained.

Add offline regression tests for the actual SNMP, IPMI, NUT and XML/HTTP admission loops using real threads and semaphores, and document the change in NEWS. The test build rules include SSL header paths and order shared libraries and generated headers for clean parallel builds.

Validation:

  • Full make check with NSS, make check-parallel-builds, and a clean tests/ build with make -s -j.
  • 272 Linux semaphore cases with sanitizers; 144 counter-fallback cases; additional cleanup checks with an unfinished worker.
  • Native macOS named-semaphore coverage (68 cases), native Windows coverage (204 cases), GNU89 checks, and complete non-threaded and counter-configured scanner builds.
  • make spellcheck, make -j2 distcheck-light, and release-note HTML generation. Spelling used Aspell 0.60.8.2 with the unmodified GNU English 2020.12.07 dictionary; the 2026.02.25 dictionary rejects unchanged baseline possessives. The distribution check excludes full manual rendering and an all-drivers build.

The patch builds independently on the upstream base. No live device or hardware scan was performed.

Related to #1166. The global semaphore initialisation failure policy remains separate, as discussed in the existing clarification.

Prepared with assistance from OpenAI Codex (gpt-6-astra). The human contributor remains responsible for reviewing and validating the change.

Skip inactive protocol semaphores, unwind partial admission, and return
owned permits when worker allocation or creation fails. Allocate tracking
entries before starting workers and keep NUT hostname and authentication
cleanup with the worker until it finishes.

Add offline regression tests using real threads and semaphores, preserve
the counter and non-threaded paths, and record the change in NEWS.

Related to networkupstools#1166. Global semaphore initialisation policy remains separate.

Prepared with assistance from OpenAI Codex (gpt-6-astra).

Signed-off-by: user01010111 <12504630+user01010111@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

A ZIP file with standard source tarball and another tarball with pre-built docs for commit 481d706 is temporarily available: NUT-tarballs-PR-3649.zip.

Pass the configured SSL include flags to the scanner tests for NSS
builds. Order scanner recursion after common and client dependencies,
and generate the SNMP header before compiling the test source.

Prepared with assistance from OpenAI Codex (gpt-6-astra).

Signed-off-by: user01010111 <12504630+user01010111@users.noreply.github.com>
Comment thread tests/nutscan-thread-test.c Fixed
nutscan_free_ip_ranges(&ranges);
}

static void scenario(unsigned int limit, int failure)
}
# endif

static void scan(unsigned int limit)
Comment thread tests/nutscan-thread-test.c Fixed
Comment thread tests/nutscan-thread-test.c Fixed
void *arg;
};

static void *offline_worker(void *opaque)
Comment thread tests/nutscan-thread-test.c Dismissed
Comment thread tests/nutscan-thread-test.c Dismissed
Comment thread tests/nutscan-thread-test.c Dismissed
Comment thread tests/nutscan-thread-test.c Dismissed
Track the protocol semaphore only after successful initialisation, and
check destruction or close before clearing fixture state. Assert clean
state at scenario entry and clear the global semaphore after teardown.

Assisted by OpenAI Codex (gpt-6-astra).

Signed-off-by: user01010111 <12504630+user01010111@users.noreply.github.com>
Comment thread tests/nutscan-thread-test.c Dismissed
@jimklimov jimklimov added bug nut-scanner portability We want NUT to build and run everywhere possible labels Sep 18, 2026
@jimklimov jimklimov added this to the 2.8.6 milestone Sep 18, 2026
@jimklimov

Copy link
Copy Markdown
Member

CI errors could be due to crashed server, builds will be restarted soon. However content-wise, is there a particular reason to not provide the code in tools/nut-scanner/nutscan-thread.h, shared by several C files, as a compiled file so the same copy is linked into them all?

Move the shared worker helpers into nutscan-thread.c and keep declarations in the header. Preserve internal symbol visibility and the existing failure-injection tests.

Assisted by OpenAI Codex (gpt-6-astra).

Signed-off-by: user01010111 <12504630+user01010111@users.noreply.github.com>
@user01010111

Copy link
Copy Markdown
Contributor Author

CI errors could be due to crashed server, builds will be restarted soon. However content-wise, is there a particular reason to not provide the code in tools/nut-scanner/nutscan-thread.h, shared by several C files, as a compiled file so the same copy is linked into them all?

Good catch, Jim. Thank you. Have updated now.

Comment thread tests/nutscan-thread-test.c Dismissed
@jimklimov

jimklimov commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Some makefile nuances missed, LIBSSL_* flags for TESTCOMBO=NUT_SSL_VARIANT=nss?

  • OmniOS:
FAIL: nutscan-nut-test
======================

ld.so.1: nutscan-nut-test: fatal: libssl3.so: open failed: No such file or directory
/tmp/jenkins-swarm/jenkins-nutci/nut_nut_PR-3649@2/test-driver: line 119: 15600: Killed
FAIL nutscan-nut-test (exit status: 265)

FAIL: nutscan-snmp-test
=======================

ld.so.1: nutscan-snmp-test: fatal: libssl3.so: open failed: No such file or directory
/tmp/jenkins-swarm/jenkins-nutci/nut_nut_PR-3649@2/test-driver: line 119: 15615: Killed
FAIL nutscan-snmp-test (exit status: 265)
  • NetBSD:
FAIL: nutscan-nut-test
======================

/var/shm/jenkins-danutci-netbsd-92-amd64/nut_nut_PR-3649/obj/tests/.libs/nutscan-nut-test: Shared object "libssl3.so" not found
FAIL nutscan-nut-test (exit status: 1)

FAIL: nutscan-snmp-test
=======================

/var/shm/jenkins-danutci-netbsd-92-amd64/nut_nut_PR-3649/obj/tests/.libs/nutscan-snmp-test: Shared object "libssl3.so" not found
FAIL nutscan-snmp-test (exit status: 1)

FAIL: nutscan-xml-test
======================

/var/shm/jenkins-danutci-netbsd-92-amd64/nut_nut_PR-3649/obj/tests/.libs/nutscan-xml-test: Shared object "libssl3.so" not found
FAIL nutscan-xml-test (exit status: 1)

Link the scanner test with configured Net-SNMP libraries when static SNMP
support is enabled. Keep the libraries after its object file so MinGW
resolves the SNMP symbols.

Verified the failing baseline and 68 passing cases on native x86_64
Windows, 204 Linux scanner cases, and distcheck-light. The i686 build
remains untested.

Assisted by OpenAI Codex.

Signed-off-by: user01010111 <12504630+user01010111@users.noreply.github.com>
Retain ordinary Net-SNMP executable flags alongside the existing library
passthrough flags and link every scanner fixture with them. Keep discovery
and cache handling in the existing dependency macro.

Give the included scanner mutex its own fixture name and preserve assertions
with NDEBUG. Order the two common-library sub-makes so their shared
replacement-function objects cannot race during parallel builds.

Validated static scanner tests on Linux and native MinGW Windows with
NDEBUG, clean two-job builds, and distcheck-light with install/uninstall.

Prepared with OpenAI Codex (GPT-6) assistance, following the earlier
Claude Opus 5.5 review.

Signed-off-by: user01010111 <12504630+user01010111@users.noreply.github.com>
@user01010111

Copy link
Copy Markdown
Contributor Author

The scanner tests pass against static libnutscan on Linux and MinGW Windows, including builds with -DNDEBUG. The fixtures now use a test-specific mutex name, avoiding the duplicate definition from static libnutscan, and keep assertions enabled.

The Windows all-static check also showed why the Net-SNMP dependency needs to cover every scanner fixture: loading the initialiser can pull in the SNMP scanner even for the NUT test. Configure now retains the ordinary executable linker flags alongside the existing libtool passthrough flags, and the common test rule places them after libnutscan. The production library workaround is unchanged.

The test prerequisites also order the two common-library sub-makes, which otherwise race while rebuilding the same replacement-function objects. Clean parallel builds with two jobs now pass. The distribution and install/uninstall checks pass too.

Prepared with Codex assistance, following the earlier Claude Opus 5.5 review.

@user01010111
user01010111 marked this pull request as ready for review October 14, 2026 21:31
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@github-actions github-actions Bot deleted a comment from AppVeyorBot Oct 23, 2026
@AppVeyorBot

Copy link
Copy Markdown

✅ Build nut 2.8.5.5899-master completed (commit 51b8cadefa by @jimklimov)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug nut-scanner portability We want NUT to build and run everywhere possible

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants