Skip to content

Commit 43a84df

Browse files
committed
lint
1 parent 93b10a5 commit 43a84df

File tree

4 files changed

+624
-616
lines changed

4 files changed

+624
-616
lines changed

.agents/skills/add-httparchive-metric-report/SKILL.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ description: Add new metrics to HTTPArchive reports config. USE FOR adding perfo
88
## Documentation
99

1010
**See [reports.md](../../../reports.md)** for complete guide including:
11+
1112
- Architecture and processing details
1213
- Quick Decision Guide table
1314
- Required SQL patterns checklist
@@ -35,18 +36,19 @@ description: Add new metrics to HTTPArchive reports config. USE FOR adding perfo
3536
metricName: {
3637
SQL: [
3738
{
38-
type: 'timeseries', // or 'histogram'
39-
query: DataformTemplateBuilder.create((ctx, params) => `
39+
type: "timeseries", // or 'histogram'
40+
query: DataformTemplateBuilder.create(
41+
(ctx, params) => `
4042
SELECT client, /* your calculations */
41-
FROM ${ctx.ref('crawl', 'pages')}
43+
FROM ${ctx.ref("crawl", "pages")}
4244
WHERE date = '${params.date}' AND is_root_page
4345
${params.lens.sql} ${params.devRankFilter}
4446
GROUP BY client ORDER BY client
45-
`)
46-
}
47-
]
47+
`,
48+
),
49+
},
50+
];
4851
}
4952
```
5053

5154
See [reports.md](../../../reports.md) for complete patterns and examples.
52-

.agents/skills/optimize-model-compute/SKILL.md

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -21,27 +21,26 @@ Automatically assign Dataform actions to BigQuery slot reservations based on pri
2121
Reservations are configured in `definitions/_reservations.js`:
2222

2323
```javascript
24-
const { autoAssignActions } = require('@masthead-data/dataform-package')
24+
const { autoAssignActions } = require("@masthead-data/dataform-package");
2525

2626
const RESERVATION_CONFIG = [
2727
{
28-
tag: 'reservation', // Human-readable identifier
29-
reservation: 'projects/.../reservations/...', // BigQuery reservation path
30-
actions: [ // Models assigned to this tier
31-
'httparchive.crawl.pages',
32-
'httparchive.f1.pages_latest'
33-
]
28+
tag: "reservation", // Human-readable identifier
29+
reservation: "projects/.../reservations/...", // BigQuery reservation path
30+
actions: [
31+
// Models assigned to this tier
32+
"httparchive.crawl.pages",
33+
"httparchive.f1.pages_latest",
34+
],
3435
},
3536
{
36-
tag: 'on_demand',
37-
reservation: 'none', // On-demand pricing
38-
actions: [
39-
'httparchive.sample_data.pages_10k'
40-
]
41-
}
42-
]
37+
tag: "on_demand",
38+
reservation: "none", // On-demand pricing
39+
actions: ["httparchive.sample_data.pages_10k"],
40+
},
41+
];
4342

44-
autoAssignActions(RESERVATION_CONFIG)
43+
autoAssignActions(RESERVATION_CONFIG);
4544
```
4645

4746
## Implementation Steps
@@ -54,8 +53,9 @@ autoAssignActions(RESERVATION_CONFIG)
5453

5554
1. Open `definitions/_reservations.js`
5655
2. Add or move actions between reservation tiers:
57-
- **Reserved slots** (`reservation: 'projects/...'`): Critical, high-priority, SLA-sensitive workloads
58-
- **On-demand** (`reservation: 'none'`): Low-priority, ad-hoc, or experimental workloads
56+
57+
- **Reserved slots** (`reservation: 'projects/...'`): Critical, high-priority, SLA-sensitive workloads
58+
- **On-demand** (`reservation: 'none'`): Low-priority, ad-hoc, or experimental workloads
5959

6060
### Step 3: Verify Changes
6161

@@ -69,12 +69,12 @@ grep -r "\.actions" definitions/_reservations.js
6969

7070
## Decision Criteria
7171

72-
| Factor | Reserved Slots | On-Demand |
73-
|--------|----------------|-----------|
74-
| **Priority** | High, SLA-bound | Low, flexible |
75-
| **Frequency** | Regular, scheduled | Ad-hoc, occasional |
76-
| **Cost Pattern** | Predictable usage | Variable, sporadic |
77-
| **Impact** | Critical pipelines | Experimental, samples |
72+
| Factor | Reserved Slots | On-Demand |
73+
| ---------------- | ------------------ | --------------------- |
74+
| **Priority** | High, SLA-bound | Low, flexible |
75+
| **Frequency** | Regular, scheduled | Ad-hoc, occasional |
76+
| **Cost Pattern** | Predictable usage | Variable, sporadic |
77+
| **Impact** | Critical pipelines | Experimental, samples |
7878

7979
## Key Notes
8080

.agents/skills/optimize-storage-costs/SKILL.md

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,13 @@ Identify and remove BigQuery tables that contribute to storage costs but have no
1313

1414
Masthead Data uses lineage analysis to identify tables, but relies on visible pipeline references. Modification timestamps are critical:
1515

16-
| Type | Definition | Indicators | Watch for |
17-
|------|------------|------------|---|
18-
| **Dead-end** | Regularly updated, no downstream consumption | Updated but never read in 30+ days | External writers outside lineage graph (manual jobs, independent pipelines) |
19-
| **Unused** | No upstream or downstream activity | No reads/writes in 30+ days | Recent `lastModifiedTime` despite "Unused" flag suggests external writer—**do not drop without verification** |
16+
| Type | Definition | Indicators | Watch for |
17+
| ------------ | -------------------------------------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------- |
18+
| **Dead-end** | Regularly updated, no downstream consumption | Updated but never read in 30+ days | External writers outside lineage graph (manual jobs, independent pipelines) |
19+
| **Unused** | No upstream or downstream activity | No reads/writes in 30+ days | Recent `lastModifiedTime` despite "Unused" flag suggests external writer—**do not drop without verification** |
2020

2121
### Key Signal
22+
2223
If a table is flagged `Unused` **and** has a recent modification timestamp, something outside Masthead's visibility is writing to it. This always warrants investigation before dropping.
2324

2425
## When to Use
@@ -60,18 +61,21 @@ ORDER BY savings_usd_30d DESC" > storage_waste.csv
6061
**Note:** Sorting by `savings_usd_30d` instead of `total_tib` prioritizes high-impact targets for review.
6162

6263
**Alternative: Use Masthead UI**
64+
6365
- Navigate to [Dictionary page](https://app.mastheadata.com/dictionary?tab=Tables&deadEnd=true)
6466
- Filter by `Dead-end` or `Unused` labels
6567
- Export table list for review
6668

6769
### Step 2: Review and Decide
6870

6971
Review `storage_waste.csv` and add a `status` column with values:
72+
7073
- `keep` - Table is needed
7174
- `to drop` - Safe to remove
7275
- `investigate` - Needs further analysis
7376

7477
**Review criteria:**
78+
7579
- Is this a backup or archive table? (consider alternative storage)
7680
- Is there a downstream dependency not captured in lineage?
7781
- Is this table part of an active experiment or migration?
@@ -94,6 +98,7 @@ bash drop_tables.sh
9498
```
9599

96100
**Safe mode (dry-run first):**
101+
97102
```bash
98103
# Add --dry-run flag to each command
99104
sed 's/bq rm/bq rm --dry-run/' drop_tables.sh > drop_tables_dryrun.sh
@@ -103,17 +108,18 @@ bash drop_tables_dryrun.sh
103108
### Step 4: Verify Savings
104109

105110
After 24-48 hours, check storage reduction in Masthead:
111+
106112
- [Storage Cost Insights](https://app.mastheadata.com/costs?tab=Storage+costs)
107113
- Compare before/after storage size and costs
108114

109115
## Decision Framework
110116

111-
| Monthly Savings | Action | Recency Check |
112-
|-----------------|--------|---------------|
113-
| < $10 | Consider keeping (low ROI) | Skip if `lastModifiedTime` > 12 months old (hygiene only) |
114-
| $10-$100 | Review and drop if unused | Check modification date; recent writes require owner verification |
115-
| $100-$1000 | Priority review, likely drop | Mandatory verification if modified in last 30 days |
116-
| > $1000 | Immediate investigation required | Always verify external writer before any action |
117+
| Monthly Savings | Action | Recency Check |
118+
| --------------- | -------------------------------- | ----------------------------------------------------------------- |
119+
| < $10 | Consider keeping (low ROI) | Skip if `lastModifiedTime` > 12 months old (hygiene only) |
120+
| $10-$100 | Review and drop if unused | Check modification date; recent writes require owner verification |
121+
| $100-$1000 | Priority review, likely drop | Mandatory verification if modified in last 30 days |
122+
| > $1000 | Immediate investigation required | Always verify external writer before any action |
117123

118124
## Key Notes
119125

0 commit comments

Comments
 (0)