Skip to content

Fix top-sellers-in-category: rank against real sales, and stop reading buy-box position as idle stock - #35

Draft
fiharet34 wants to merge 1 commit into
joomcode:mainfrom
fiharet34:fix/top-sellers-rank-against-real-sales
Draft

Fix top-sellers-in-category: rank against real sales, and stop reading buy-box position as idle stock#35
fiharet34 wants to merge 1 commit into
joomcode:mainfrom
fiharet34:fix/top-sellers-rank-against-real-sales

Conversation

@fiharet34

Copy link
Copy Markdown
Contributor

Fix top-sellers-in-category: rank against real sales, and stop reading buy-box position as idle stock

Summary

Five defects, each reproduced against production data before the file was touched and
re-verified afterwards. Two of them are visible in the skill's own live output rather
than inferred from reading it, and one of those is about to get worse: since #33 made
the visual panel mandatory, a misreading that used to surface occasionally in prose now
has a chart reserved for it on every run.

Rebased onto 421426d. 9,421 → 14,826 bytes, 8 hunks. Description is 978 characters,
inside the 1024 limit.

The defects

1. The panel charts a misreading. Visualization asks for "a small 'registered
versus with sales' comparison for sellers and for products, to show how much of the
supply actually converts."
On a catalog product only the seller holding the buy-box
shows sales — every other seller of that product reads zero. So the product half of
that comparison is mostly measuring buy-box position, not dead inventory.

A live run put it in the third line of the answer, unprompted:

"Dos 115.134 anúncios que esses 50 mantêm na categoria, só 40.097 (34,8%) registram
vendas estimadas — mesmo no topo, dois terços do estoque está parado."

Two thirds of that gap is attribution. Before #33 this appeared when the model happened
to reach for it; now the panel renders every time, so an unlabelled version of the chart
would mislead on every run instead of occasionally.

2. Nothing checks the estimate against real sales before ranking. The ranking metric
is an estimate; completed sales over the last year are real, and the two can diverge by
orders of magnitude.

seller est. revenue/mo completed sales, 365d listings
RAIZENTECH (ranked 5th) R$ 5.484.074 352 225
LINDOPETS (ranked 36th) R$ 1.914.830 0 1
RMAQUINASK R$ 1.668.239 234 296

RAIZENTECH's estimate implies roughly 72.000 sales a year against 352 actually
completed. LINDOPETS is a single listing with no sales at all, in a pets-named shop
inside a tools category. Beleza produced seven rows of the same shape.

The skill sometimes flagged them in a closing note — and ranked them anyway. That is not
enough: the positions are finite. Excluding them surfaced HOME HOUSE_, GALANTELEDS,
RITEC FERRAMENTAS and LOJAOFICIALCASASBAHIA, all real businesses that the artifacts
had been displacing.

3. Shops sharing a brand are never grouped, and it changes who leads. In Ferramentas:

  • THE_BLACK_TOOLS_ + THE_BLACK_TOOLS = R$ 15.278.505
  • ISMAFER (Vila Guilherme + 06 + MG) = R$ 11.477.497
  • stated leader CABE_FERRAMENTAS = R$ 10.724.187

Either grouping displaces the leader, and the skill's "no one dominates" framing holds
only at shop level. Across three pre-fix runs this observation appeared in one.

Common ownership is not verifiable from this data — those shops' registration dates
span 2009 to 2023 with unrelated identifiers, which argues against a single operator as
often as for one. So the fix requires flagging the shared name stem and showing the
combined figure as an alternative reading, and explicitly forbids asserting ownership.

4. No missing-value rule. Four runs, four renderings of the same condition: sem dados, 0,00%, n/d, and once simply absent. The 0,00% case is the harmful one — a
cancellation rate of zero on a seller with no completed sales reads as a perfect record
when nothing was measured at all.

5. No fallback when the table does not fit. Twelve columns across fifty rows exceeds
a terminal, and one run abandoned the table for a vertical record-per-seller list. That
breaks the skill's own premise: the user is told to save the table and send it back next
period for a comparison, and a per-seller list cannot be diffed row against row.

Related, from the same run: revenue rounded to R$ 0,7x mi, which flattened rows 48–50
into indistinguishable values in a table whose only purpose is ordering.

What changed

  • A two-branch check against real sales before ranking. No real trading history →
    out of the ranking, into a labelled group showing the estimate, the real completed
    sales and the listing count together. A real business with an inflated estimate →
    kept, with a note that its position may be too generous.
  • Brand-stem grouping with the combined figure and its effect on the leader stated,
    and an explicit prohibition on claiming common ownership.
  • The registered-versus-with-sales comparison is now sellers only. Charting the
    product split is forbidden, and describing the gap as idle stock or failed conversion
    is called out by name with the buy-box reason given.
  • Medalha promoted to a column. The downloadable file is the baseline the seller
    brings back, so it has to carry every field the report and its panel use — a file that
    cannot reproduce the panel is not a baseline.
  • Sales trend renamed to Crescimento mensal and pinned to month-over-month
    growth, so it cannot drift onto an older run-rate measure that is empty for most
    sellers.
  • Missing values, full money precision, base-effect caveat, width fallback
    (drop columns from the right and say which; never abandon the table), retry after a
    pause
    , and empty-result-is-not-an-outage.
  • The grain split is now stated every run, not only when the numbers look odd:
    sales, revenue, brands and product counts are category-scoped; completed sales,
    cancellation rate, growth and listing types are store-wide.
  • Headers are pt-BR by default, replacing the mixed Cancel rate | Sales trend | Classic
    row.

Validation

Three pre-fix runs and three regression runs, across two categories and both languages,
with output figures spot-checked against independent queries.

run scope outcome
1 Beleza, pt-BR, pre-fix 7 artifact rows ranked; 0,00% cancellation on a seller with none
2 Ferramentas, pt-BR, pre-fix artifact at position 5; table abandoned for a vertical list; brand groups unmentioned
3 Ferramentas, EN, pre-fix brand grouping appeared — the only run of three that found it
4 Ferramentas, post-fix 7 excluded, 4 real sellers surfaced, all brand groups grouped, columns dropped not abandoned
5 Beleza, post-fix over-exclusion found and fixed (below)
6 Beleza, post-fix, refined 17 of 80 excluded, flagged rows kept in place, full precision held

The panel was verified in the rendered artifact, not inferred from prose. Its
seller-split section is titled "Quantos vendedores realmente vendem" and charts only
Ativos 180.238 against Com vendas 92.763 · 51,5%. There is no product split anywhere
in the panel.

Two corrections made during validation

Worth recording, because both were wrong in the same way — right in direction, wrong at
the boundary — and only a run exposed them.

The exclusion test over-reached. Its first version cut 23 of 90 shops in Beleza,
including PACHECO_IMPORTS RP — 31.888 completed sales, 240 listings, 3,81%
cancellation, Platinum, and third in every prior run. That is a real business whose
estimate is inflated, not an artifact. The rule had no floor on the real side. It now
has two branches, and the distinction is the real figure rather than the ratio.

A blanket ban on showing a local file path was too wide. In a terminal, telling the
user where the file landed is the useful thing. Narrowed to internal working paths: a
temporary or scratch directory is not a save location.

Known limitations

  • The missing-value rule is unexercised. Excluding the artifacts removed the rows
    that carried the nulls, so it may rarely fire. Kept because the condition still exists;
    not claimed as verified.
  • Two categories, one marketplace. Beleza e Cuidado Pessoal and Ferramentas.
  • The panel path is verified once. Every other run was terminal-only.
  • Brand grouping is a name heuristic. It will miss related shops with unrelated names
    and may group unrelated shops that happen to share a stem — which is why the rule
    reports it as an alternative reading rather than restating the ranking.
  • No repeat-run measurement. The three regression runs vary category or language, so
    nothing here says whether the same prompt twice gives the same answer.

Notes for maintainers

#33 means this skill now publishes an artifact on every run, containing a named
competitive leaderboard of fifty real sellers. Artifacts are private by default, so this
is not a disclosure problem — but it is a change in what the skill does with the data,
and it arrived through a library-wide visualization change rather than a decision about
this skill. Flagging it because this PR is the first to land on top of it.

A fix we had prepared is already upstream. The disclaimer here was pt-BR only at the
base we started from; the English half is now present, so that change was dropped.

This PR does not touch CHANGELOG.md. The newest entry is 0.2.0 and the commits
after it are not recorded there, which reads like a release-time convention rather than a
per-PR one. Happy to add an entry if that is wrong.

Three of these rules are not skill-specific — empty-result-is-not-an-outage, retry
after a pause, and never presenting partial coverage as complete. They apply to most of
the library, and restating them per skill is why this file grew. They would sit better in
a shared style document, which the repo does not currently have.

Checklist

  • Rebased onto current main (421426d)
  • Behaviour reproduced against production data before any edit
  • Every fix re-verified by a live run afterwards
  • Visual panel verified in the rendered artifact
  • Description 978 characters, inside the 1024 limit
  • Canonical section order preserved; no new workflow steps
  • No prose line over 95 bytes
  • No cube, field, measure or query references introduced
  • pt-BR and English output paths both exercised
  • Missing-value rule — present but not exercised
  • Repeat-run consistency — not measured

…g buy-box position as idle stock

Checks each estimate against the seller's real completed sales before placing a
row: shops with no real trading history move out of the ranking into a labelled
group, while a real business whose estimate runs high stays in with a caveat.
Groups shops sharing a name stem and says whether that changes the leader,
without asserting common ownership.

Reframes the registered-versus-with-sales comparison as sellers only. On a
catalog product only the buy-box holder shows sales, so the product gap measures
buy-box position rather than idle inventory - and since the panel now renders
every run, an unlabelled version of that chart would mislead every time.

Also: medal as a column so the saved baseline can reproduce its own panel, the
growth column named for the metric it carries, full money precision in a
ranking, a missing-value rule, a width fallback that drops columns rather than
the table, retry after a pause, and an empty result separated from an outage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant