Always set BOT_COUNT metric in the run_bot task loop#5360
Merged
Conversation
dylanjew
force-pushed
the
dylanj/bot-metrics
branch
from
July 14, 2026 20:14
5988e73 to
4dd5c13
Compare
Collaborator
Author
|
Updated PR to only move the metric and preserve the update task flow. |
dylanjew
changed the title
dylanjew
force-pushed
the
dylanj/bot-metrics
branch
from
July 15, 2026 17:30
4dd5c13 to
538dcc4
Compare
| update_task.prepare_environment_for_new_task() | ||
| update_task.run_platform_init_scripts() | ||
|
|
||
| update_task.track_revision() |
Collaborator
There was a problem hiding this comment.
I think it should be inside the else right, because it would be called twice for the cases that the update task is enabled, as it is called by the update_task.run.
Collaborator
Author
There was a problem hiding this comment.
Where does track_revision get called in update_task.run? I only see it called in run_bot#main, and only for untrusted workers, which IIUC the mac bots are not.
But aside from that, I think it's ok to set a Gauge Metric multiple times on the same VM instance because the metric is only stored to the dict again and then emitted on flush every 10 minutes, which prevents the bot from counting itself multiple times.
Collaborator
Author
|
Pushed to dev and things look fine. There are metrics now. I think I had pushed this yesterday but dev was reset to master at some point, so that's why the metrics were there yesterday for a period |
javanlacerda
approved these changes
Jul 16, 2026
jr2bg
added a commit
that referenced
this pull request
Jul 16, 2026
commit e4b33cd Author: Arturo Barrera <jabgonzalez@google.com> Date: Thu Jul 16 13:39:52 2026 +0000 Resolve comments on issue filing - issue_filer.py Get fuzzer from DataStore query, handles cases when no `primary_owner` is found, does not consider yet `external_contribution. - migrate_fuzzer_owners.py Removes the owner commit b39db4e Author: Arturo Barrera <jabgonzalez@google.com> Date: Mon Jun 15 18:47:45 2026 +0000 Migrate from monorail to buganizer. commit eae2447 Author: Arturo Barrera <jabgonzalez@google.com> Date: Fri Jun 12 20:07:11 2026 +0000 Add license. Modify locations. commit e0113e7 Author: Arturo Barrera <jabgonzalez@google.com> Date: Fri May 22 17:30:28 2026 +0000 Set fuzzer author as reporter for bugs from external fuzzers commit 5f7e9d7 Author: Dylan Jew <dylanjew@users.noreply.github.com> Date: Thu Jul 16 09:02:36 2026 -0400 Always set BOT_COUNT metric in the run_bot task loop (#5360) The update_task_enabled feature flag has been rolled out since Sept 2025 to all instances of clusterfuzz. However, the update task flow is used for non terraform managed bots. The BOT_COUNT metric is only set in the update task flow and at bot tart up if `is_untrusted_worker`. However, windows bots trigger neither of those. This PR only moves the BOT_COUNT metric outside of the update_task flow to ensure that `update_task.track_revision()` sets the BOT_COUNT metric for windows and other terraform managed bots. It's safe to update the BOT_COUNT metric multiple times, because we're calling `set` on a `Gauge` metric based on the GCE instance. I've added details to crbug.com/534369736 This will only affect mac, android and other bots not configured already in terraform with `update_task_enabled = False` Testing Dev [metrics](https://pantheon.corp.google.com/monitoring/metrics-explorer;duration=P2D?pageState=%7B%22xyChart%22:%7B%22dataSets%22:%5B%7B%22timeSeriesQuery%22:%7B%22timeSeriesFilter%22:%7B%22filter%22:%22metric.type%3D%5C%22custom.googleapis.com%2Fbot_count%5C%22%20resource.type%3D%5C%22gce_instance%5C%22%22,%22aggregation%22:%7B%22perSeriesAligner%22:%22ALIGN_MEAN%22,%22crossSeriesReducer%22:%22REDUCE_SUM%22,%22groupByFields%22:%5B%22metric.label.%5C%22os_type%5C%22%22%5D,%22alignmentPeriod%22:%2260s%22%7D%7D%7D,%22plotType%22:%22LINE%22,%22targetAxis%22:%22Y1%22,%22minAlignmentPeriod%22:%2260s%22%7D%5D,%22chartOptions%22:%7B%22mode%22:%22COLOR%22,%22displayHorizontal%22:false%7D,%22thresholds%22:%5B%5D,%22yAxis%22:%7B%22scale%22:%22LINEAR%22%7D%7D%7D&project=clusterfuzz-development) commit 900904e Author: Giovanni Ortuño <ortuno@google.com> Date: Thu Jul 16 08:44:19 2026 -0400 symbolizer: Stop using llvm-symbolizer if it crashes (#5361) Changes LLVMSymbolizer.symbolize() to return early if it previously raised an error. This avoids trying to use the crashed symbolizer for every new frame which is currently causing a lot of error logs. Also changes the error message to include the last successfully symbolized frame.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The update_task_enabled feature flag has been rolled out since Sept 2025 to all instances of clusterfuzz. However, the update task flow is used for non terraform managed bots.
The BOT_COUNT metric is only set in the update task flow and at bot tart up if
is_untrusted_worker. However, windows bots trigger neither of those.This PR only moves the BOT_COUNT metric outside of the update_task flow to ensure that
update_task.track_revision()sets the BOT_COUNT metric for windows and other terraform managed bots.It's safe to update the BOT_COUNT metric multiple times, because we're calling
seton aGaugemetric based on the GCE instance.I've added details to crbug.com/534369736
This will only affect mac, android and other bots not configured already in terraform with
update_task_enabled = FalseTesting
Dev metrics