fix(pagination): silent pagination truncation when native API uses PageSize or StartingToken#10443
Open
Abuhaithem wants to merge 1 commit into
Open
fix(pagination): silent pagination truncation when native API uses PageSize or StartingToken#10443Abuhaithem wants to merge 1 commit into
Abuhaithem wants to merge 1 commit into
Conversation
…global CLI paging arguments Corrects a typo ( -> ) and an omission () in the allowlist within . Previously, if an API operation naturally defined its paginator limit key as exactly PageSize (e.g., ), the global argument was incorrectly identified as an undocumented manual API parameter. This caused the CLI to silently disable automatic pagination and prematurely truncate results after fetching a single page. This fix restores the correct behavior, ensuring global paging parameters do not unintentionally disable automatic pagination.
Author
|
@kdaily could u check this PR ? |
Author
|
@ashovlin could u check this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10442
This pull request fixes a critical bug where providing the global pagination parameters
--page-sizeor--starting-tokensilently disabled automatic pagination for a subset of AWS API commands, resulting in silent data truncation (returning only the first page and exiting with code 0).Motivation and Context
The AWS CLI supports automatic pagination by injecting global parameters (
--page-size,--starting-token,--max-items) and intentionally disabling automatic pagination if the user specifies a raw, native API pagination parameter (e.g.,--next-tokenor--marker).The logic checking for this collision in
awscli.customizations.paginate.check_should_enable_paginationhad a typographical error (start_tokeninstead ofstarting_token) and completely omittedpage_size:As a result, if an API operation naturally defines its
limit_keyas exactly"PageSize"(rather than"MaxResults"or"MaxItems"), the raw API argument name shadows the global CLI argument. Becausepage_sizewas missing fromnormalized_paging_args, the CLI assumed the user was intentionally specifying the undocumented raw API parameter and disabled automatic pagination.This silently broke automated pagination for over 50 operations across
elb,elbv2,servicecatalog,pinpoint-email,mailmanager,sesv2,ce, andlakeformation(whenever--page-sizewas explicitly specified), as well asssm-quicksetup(when--starting-tokenwas specified).This fix corrects the tuple to properly exempt the global parameters from disabling pagination:
Testing Done
Validated via
aws elbv2 describe-load-balancers --page-size 10, confirming that the CLI now automatically paginates and fetches all load balancers instead of aborting after the first 10 items.Added a direct mock validation simulating the parser flow, verifying
parsed_globals.paginateremainsTrue.Types of Changes