Repository navigation
[bot] Merge master/4a5e851e into rel/dev - #1895
Merged
Merged
Conversation
In context mode an agent that hit a MAQL error and asked "Should I
retry with this corrected formula?" was classified as asking a
question, not as asking for confirmation. The question went to the
clarification judge, which graded it as lost context. The simulated
user was then asked to restate what had been said earlier, and the
only formula said earlier was the agent's own rejected proposal, so it
sent that back. The turn failed even though the metric the agent
finally created matched the expected one.
- Treat retry / try (it|this|that) again / resubmit after
"should/shall/may/can I", "do you want me to" and "would you like
me to" as a confirmation, so the user approves the revised proposal
with "Yes, go ahead."
- A retry that offers a choice stays a question: "Should I retry
with gross or net revenue?" and "Should I retry? Or adjust the
formula first?". An OR inside an inline formula or a {type/id}
reference is not a choice, and a dot inside a token ("2.5") does
not end the sentence.
- `use` and `run` stay out, because they would turn choice
questions such as "Which one should I use?" into confirmations.
- The verbs now end on a word boundary, so "Should I address ..." is
no longer read as "Should I add".
- Tell the simulated user in reply_restating and reply_from_facts
never to write MAQL, SQL, formulas or object definitions.
- Replace any generated reply that still contains MAQL or SQL with
NO_ANSWER_REPLY. That covers uppercase SELECT, an uppercase
aggregation call (SUM(...), COUNT(...), ...), "select" in any case
before an aggregation call, lowercase SQL shaped as select * /
a column list / a table followed by where, group by, order by or
join, and {metric/...}, {fact/...}, {label/...}, {attribute/...} or
{dataset/...} references in braces or angle brackets. A plain
"Select the first option from the list." is left alone. Both
callers answer a question, so the fallback has to answer one too,
not just approve. Fixture set answers are sent unchanged.
jira: QA-29876
risk: low
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The retry choice check read any "or" after "Should I retry" as a
second option, so the common "Should I retry with this corrected
formula, or would you like to adjust it?" went to the clarification
judge again, which is the lost-context path the retry confirmation
was meant to avoid. The MAQL filter, in turn, read any uppercase
SELECT as MAQL.
- An "or" followed by not, would/do you like/want/prefer to, is/are
there, anything, something, if you or let me know hands the turn
back to the user and is not an option.
- Only a prose "or"/"Or" counts. An uppercase MAQL OR in a formula
written without backticks, or an "or" inside a double- or
single-quoted literal ('Processed or Shipped'), is part of the
proposal. A single quote counts only outside a word, so apostrophes
("today's data or last week's") do not hide a real choice.
- "Alternatively" offers a second option like "or" does: "Should I
retry? Alternatively, should I adjust the formula first?" is a
question.
- "try once more" is a retry, like "try again".
- An uppercase SELECT counts as MAQL only when a reference, "(", "*",
DISTINCT or a call follows it, so a plain "SELECT the second
option." is no longer replaced by an answer that picks the first.
- The MAQL filter also catches a lowercase aggregation call with the
parenthesis attached ("sum(units) / count(orders)"); "Count (all)"
is still left alone.
jira: QA-29876
risk: low
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(gooddata-eval): keep the simulated user from writing MAQL
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## rel/dev #1895 +/- ##
========================================
Coverage 85.09% 85.10%
========================================
Files 337 337
Lines 24332 24343 +11
========================================
+ Hits 20705 20716 +11
Misses 3627 3627 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚀 Automated PR to perform merge from master into rel/dev with changes up to 4a5e851 (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/39755269934).