Repository navigation
build: keep full git version out of config.h (ccache-friendly) - #3759
Conversation
|
A ZIP file with standard source tarball and another tarball with pre-built docs for commit b6ff843 is temporarily available: NUT-tarballs-PR-3759.zip. |
jimklimov
left a comment
There was a problem hiding this comment.
Looks like a promising start, thanks! Added a few thoughts as I was reading through this PR, to challenge some assumptions. Feel free to prove me misled :D
Pass a new SEMVER_ACINIT query of gitlog2version.sh to AC_INIT instead of the bare SEMVER: it is the SEMVER for releases, and by the legacy convention a nominal "SEMVER.1" for any development iteration after that release, so the include/config.h macros still tell releases and development builds apart, but do not change between releases and keep build caches valid. It is built from the triplet, so a 4-component NUT_VERSION_DEFAULT (e.g. 2.8.5.1) does not get a second ".1" suffix. For pre-release tagged commits SEMVER is the upcoming release, so the VER50 triplet is used there (unless NUT_VERSION_FORCED_SEMVER is set). Checked with sh, dash and bash for git and NUT_VERSION_DEFAULT values (release, development, tarball and pre-release ones), and that config.h gets 2.8.5.1 while Makefiles keep the full 2.8.5.1503.2. Spellcheck passes with both current and old aspell-en 7.1. Suggested in PR networkupstools#3759 review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
|
Tested A clean rebuild after changing the full version produced 174 direct hits out of 178 cacheable calls. The four misses were exactly One test expectation update turned up: Keeping the PR’s helper unchanged and updating only those 13 expected diagnostic strings in a temporary test copy gave 136 passes and no failures. Looks like those expectations need updating alongside the helper. |
include/config.h is included by every source file, and carried the full git-derived version (e.g. 2.8.5.1466.13) in PACKAGE_VERSION, PACKAGE_STRING and VERSION macros. It changed with every commit, so ccache direct mode missed for all objects on each new commit or fresh CI checkout. Without -g3, preprocessed mode mostly recovered the hits (no code uses these macros), but with -g3 (e.g. AppVeyor) the preprocessed output keeps all #define lines, so every object was recompiled. Pass the base SEMVER (e.g. 2.8.5) to AC_INIT, so config.h only changes with releases, and set the shell PACKAGE_VERSION/PACKAGE_STRING right after it back to the full VER50 value, so all @PACKAGE_VERSION@ substitutions (Makefiles, *.pc files, packaging recipes, tarball names, configure reports) keep the full version. Use AM_INIT_AUTOMAKE no-define (PACKAGE and VERSION macros are unused) and reset VERSION for distdir. Programs still report the full version via NUT_VERSION_MACRO in include/nut_version.h, consumed by two files. Checked with CCACHE_DEBUG=1 builds across two commits (WSL gcc 13, ubuntu-plucky gcc 14): config.h byte-identical, all objects direct hits except upsclient.o and common-nut_version.o. Also checked a dist tarball build (VERSION_DEFAULT), NUT_VERSION_FORCED, and autoconf 2.69 on CentOS 7 and Ubuntu Trusty. Docs go into the "Repetitive build/test optimizations" chapter of INSTALL.nut.adoc. Follow-up to PR networkupstools#3754. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
With the full git version kept out of include/config.h, there is no need for ci_build.sh to force a shortened version into autogen.sh to keep ccache hits across pull requests. The workaround was AppVeyor only, still changed whenever master advanced, and also shortened the substituted @PACKAGE_VERSION@ (tarball, *.pc). Restore the plain autogen.sh call and drop the setting from appveyor.yml and the docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
Pass a new SEMVER_ACINIT query of gitlog2version.sh to AC_INIT instead of the bare SEMVER: it is the SEMVER for releases, and by the legacy convention a nominal "SEMVER.1" for any development iteration after that release, so the include/config.h macros still tell releases and development builds apart, but do not change between releases and keep build caches valid. It is built from the triplet, so a 4-component NUT_VERSION_DEFAULT (e.g. 2.8.5.1) does not get a second ".1" suffix. For pre-release tagged commits SEMVER is the upcoming release, so the VER50 triplet is used there (unless NUT_VERSION_FORCED_SEMVER is set). Checked with sh, dash and bash for git and NUT_VERSION_DEFAULT values (release, development, tarball and pre-release ones), and that config.h gets 2.8.5.1 while Makefiles keep the full 2.8.5.1503.2. Spellcheck passes with both current and old aspell-en 7.1. Update the gitlog2version test expectations in tests/nut-driver-enumerator-test.sh for the SEMVER_ACINIT value now included in the debug output, and add test cases for the new query (release, development, 4-component default and release candidate). Suggested in PR networkupstools#3759 review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
bb5de63 to
4878552
Compare
|
Thanks, fixed in 4878552 (also added SEMVER_ACINIT test cases). |
…SEMVER_ACINIT corner case of `X.Y.Z.0(.0...)` [networkupstools#3759] Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
… not worth mentioning in these notes [networkupstools#3759] Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
Add a "GHA-07: ShellCheck" workflow for pull requests, which runs ShellCheck on the shell scripts changed by the PR. The existing scripts have many known findings (about 2900 in the tree, 28 of them errors), so checking the whole tree would always fail. Instead, the new tools/shellcheck-ci.pl helper checks both the base and the PR versions of the changed scripts, and reports only new findings, matched by file, ShellCheck code and source line text (so moved code does not count as new). It fails for new "warning" or "error" level findings; new "note" level (info and style) ones are shown as notices. In GitHub Actions, the findings are shown as annotations on the changed lines and listed in the job summary, with links to the lines and the ShellCheck wiki. It can be run locally against any base revision, including uncommitted changes. The helper is in Perl, like tools/check-source-nonascii.pl (Perl is already needed for autogen.sh in Git workspaces). It only uses core Perl 5 modules, and parses the "gcc" output format of shellcheck, so no JSON module is needed on older systems. The workflow uses only actions/checkout (no new third-party actions) and compares the PR merge commit with its first parent. A new .shellcheckrc file disables the checks for deliberate portability idioms of NUT scripts: backticks (SC2006), x"$var" comparisons (SC2268) and expr (SC2003). It is distributed like .editorconfig, to help the tarball users contribute too. Describe both in docs/qa-guide.adoc, replacing the note that the tool could be introduced at a later time. The .shellcheckrc backticks setting does for ShellCheck what the previous commit did for "make shellcheck" (issue networkupstools#1058). Checked with the "build: keep full git version out of config.h" series (PR networkupstools#3759): its first version is reported for one new SC2166 warning ([ p -a q ]) and the final one passes. Checked renamed, added, removed and non-shell files, a bad base revision, and the workflow with actionlint and zizmor. In a test PR on a fork, deliberate findings showed as annotations on the changed lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
…SEMVER_ACINIT corner case of `X.Y.Z.0(.0...)` [networkupstools#3759] Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
… not worth mentioning in these notes [networkupstools#3759] Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
1759fa5 to
6056077
Compare
Add a "GHA-07: ShellCheck" workflow for pull requests, which runs ShellCheck on the shell scripts changed by the PR. The existing scripts have many known findings (about 2900 in the tree, 28 of them errors), so checking the whole tree would always fail. Instead, the new tools/shellcheck-ci.pl helper checks both the base and the PR versions of the changed scripts, and reports only new findings, matched by file, ShellCheck code and source line text (so moved code does not count as new). It fails for new "warning" or "error" level findings; new "note" level (info and style) ones are shown as notices. In GitHub Actions, the findings are shown as annotations on the changed lines and listed in the job summary, with links to the lines and the ShellCheck wiki. It can be run locally against any base revision, including uncommitted changes. The helper is in Perl, like tools/check-source-nonascii.pl (Perl is already needed for autogen.sh in Git workspaces). It only uses core Perl 5 modules, and parses the "gcc" output format of shellcheck, so no JSON module is needed on older systems. The workflow uses only actions/checkout (no new third-party actions) and compares the PR merge commit with its first parent. A new .shellcheckrc file disables the checks for deliberate portability idioms of NUT scripts: backticks (SC2006), x"$var" comparisons (SC2268) and expr (SC2003). It is distributed like .editorconfig, to help the tarball users contribute too. Describe both in docs/qa-guide.adoc, replacing the note that the tool could be introduced at a later time. The .shellcheckrc backticks setting does for ShellCheck what the previous commit did for "make shellcheck" (issue networkupstools#1058). Checked with the "build: keep full git version out of config.h" series (PR networkupstools#3759): its first version is reported for one new SC2166 warning ([ p -a q ]) and the final one passes. Checked renamed, added, removed and non-shell files, a bad base revision, and the workflow with actionlint and zizmor. In a test PR on a fork, deliberate findings showed as annotations on the changed lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: nmbro <nmbro@users.noreply.github.com>
|
❌ Build nut 2.8.5.5807-master failed (commit 27de48059c by @jimklimov) |
|
❌ Build nut 2.8.5.5816-master failed (commit 27de48059c by @jimklimov) |
|
@jimklimov I'm trying to collect all of the red crosses, got to catch them all! I have spotted some other improvements that can be done to AppVeyor build (again) (will open another PR with those) but this is still the biggest immediate save, since it's adding unused ccache per PR |
Follow-up to #3754: instead of an AppVeyor-only
CI_AUTOGEN_STABLE_VERSIONworkaround inci_build.sh, keep the volatile git-derived version out ofinclude/config.hfor everyone (CI, packagers/OBS, local builds).Problem
include/config.his included by every source file and carried the full version (e.g.2.8.5.1466.13) inPACKAGE_VERSION,PACKAGE_STRINGandVERSION. These change with every commit, so ccache direct mode missed for every object on each new commit or fresh checkout.Debug builds with
CCACHE_DEBUG=1across two commits showed:config.hdiffer between commits. No C code uses them (only an#if 0block and#undefs next to net-snmp headers).-g3, ccache's preprocessed mode mostly recovered the hits (364/381 on Linux), at the cost of a preprocessor pass per file.-g3(as on AppVeyor),gcc -Ekeeps all#definelines in its output, so preprocessed mode misses too, and every object was recompiled. AppVeyor stats before CI: speed up AppVeyor build, fix ccache misses across PRs #3754: preprocessed mode hit 15 times in 1071 lookups.Change
configure.ac:AC_INITgets the baseSEMVER(e.g.2.8.5), which only changes with releases, so that is what lands inconfig.h. Right afterAC_INIT, the shellPACKAGE_VERSION/PACKAGE_STRINGare set back to the fullVER50value, so all@PACKAGE_VERSION@substitutions are unchanged (Makefiles,*.pc, packaging recipes,driver.list, tarball name, configure report).AM_INIT_AUTOMAKE([... no-define]): drops the unusedPACKAGE/VERSIONmacros;VERSIONis reset to the full version fordistdir.NUT_VERSION_MACROininclude/nut_version.h, which onlycommon/common-nut_version.candclients/upsclient.cinclude.CI_AUTOGEN_STABLE_VERSIONfromci_build.sh(back to its pre-CI: speed up AppVeyor build, fix ccache misses across PRs #3754 form) andappveyor.yml. That workaround still changed whenever master advanced, and it also shortened the substituted version (tarball,*.pc).INSTALL.nut.adoc("Repetitive build/test optimizations"),docs/developers.txt,NEWS.adoc,UPGRADING.adoc.Testing
git clean -fdx, autogen, configure, make withCC="ccache gcc"), empty commit, then the same again:config.hbyte-identical; all objects direct hits except the 2nut_version.hconsumers (x2 for libtool).ubuntu-pluckycontainer, gcc 14, ccache 4.10: 427/431 hits, all direct.make distcheck-light, plus building from the resulting tarball without git (VERSION_DEFAULTpath):config.hgets2.8.5, Makefiles get the full version.NUT_VERSION_FORCEDset: the semver goes intoconfig.h, the forced value everywhere else.make spellcheck, ASCII-only check on all changed text.CI_AUTOGEN_STABLE_VERSION. This PR's AppVeyor run should show it (after a warm-up build, sinceconfig.hcontent changes once).Open question
./configure --versionand theconfig.logheader now show the base semver (2.8.5) instead of the full version. If that matters, the alternative is to keepAC_INITfull and strip the defines fromconfdefs.hbeforeAC_OUTPUT, which is uglier and more fragile.Checklist
NEWS.adoc, andUPGRADING.adocmake spellcheckmake distcheck-lightpasses🤖 Generated with Claude Code