Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
267 changes: 264 additions & 3 deletions Cargo.lock

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ regex = { version = ">=1.12.2" }
debian-changelog = { version = ">=0.2.14" }
debian-control = { version = ">=0.1.39" }
ignore = { version = ">=0.4.25" }
ar = { version = ">=0.9.0" }
tar = { version = ">=0.4.45" }
flate2 = { version = ">=1.1.9" }
xz2 = { version = ">=0.1.7" }
zstd = { version = ">=0.13.3" }

# dev dependencies
test-case = { version = ">=3.3.1" }
tempfile = { version = ">=3.27.0" }
insta = { version = ">=1.48.0" }
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Unified and future-proof developer tooling for increased productivity in the [De
Debmagic has two independent parts:
- tooling to [**build** and **test**](#debmagic-building) packages
- in isolated container environments (lxd, docker, ...)
- `debmagic build`, `debmagic check`, `debmagic test`, `debmagic shell`, ...
- `debmagic build`, `debmagic lint`, `debmagic test`, `debmagic shell`, ...
- create package [**build instructions**](#debmagic-packaging)
- using Python with `debian/rules.py` (instead of shell & Makefiles)

Expand All @@ -31,7 +31,7 @@ To learn packaging with debmagic, follow **[the documentation!](https://debmagic
| `debmagic build binary` | Build a binary package in a container |
| `debmagic build source` | Create a source package for upload |
| `debmagic test` | Run Debian autopkgtest tests (`debian/tests/`) against a prior build |
| `debmagic check` | Lint the package |
| `debmagic lint` | Lint the package |

> [!TIP]
> Want to know more about how to use debmagic to build a package? See [docs/usage/build.md](docs/usage/build.md) for a quickstart.
Expand Down Expand Up @@ -158,6 +158,10 @@ For questions, suggestions, problem support, please join and just ask!
| Matrix Chat | [`#sfttech:matrix.org`](https://app.element.io/#/room/#sfttech:matrix.org) |
| Support us | [![donations](https://liberapay.com/assets/widgets/donate.svg)](https://liberapay.com/SFTtech) |

## Acknowledgements

debmagic's lint implementation is heavily inspired by [oxlint](https://oxc.rs/docs/guide/usage/linter) and [Ruff](https://docs.astral.sh/ruff/). It draws on [lintian](https://lintian.debian.org/)’s checks and tag catalog, with the long-term goal of full lintian compatibility plus additional original rules.

## License

Released under the **GNU General Public License** version 2 or later, [LICENSE](legal/GPL-2) for details.
9 changes: 8 additions & 1 deletion debian/control
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,17 @@ Build-Depends:
librust-chrono-dev (>=0.4.42),
librust-regex-dev (>=1.12.2),
librust-test-case-dev (>=3.3.1),
librust-tempfile-dev (>=3.27.0),
librust-insta-dev (>=1.48.0),
librust-pyo3-dev (>=0.27.2),
librust-debian-changelog-dev (>=0.2.14),
librust-debian-control-dev (>= 0.1.39),
librust-ignore-dev (>=0.4.25)
librust-ignore-dev (>=0.4.25),
librust-ar-dev (>=0.9.0),
librust-tar-dev (>=0.4.45),
librust-flate2-dev (>=1.1.9),
librust-xz2-dev (>=0.1.7),
librust-zstd-dev (>=0.13.3)
Rules-Requires-Root: no
X-Style: black
Standards-Version: 4.7.2
Expand Down
101 changes: 101 additions & 0 deletions docs/develop/import-lintian-rule.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Import a lintian Rule

This is the working procedure for turning a lintian Tag into an LN Rule. Native Rule tests (`SourceTreeTester` + insta) come first. Lintian’s own recipes become the Parity oracle later — they cannot be dropped into `SourceTreeTester` as a source tree.

Related: [Parity is Tag-set](../adr/0007-lintian-parity-is-tag-set.md), [oracle is the test suite](../adr/0012-parity-oracle-is-lintian-test-suite.md), [Codes when catalogued](../adr/0015-ln-codes-assigned-when-tag-is-catalogued.md), [per-Subject Rule traits](../adr/0025-per-subject-rule-traits.md), [recipe layout](../research/lintian-test-recipes.md).

Pin lintian **2.139.0** (`575a2bf1`). Installed tags and checks are under `/usr/share/lintian/`; recipes are **not** in that package — clone git or `apt source lintian` at that version.

---

## 1. Identify the Tag

Read `/usr/share/lintian/tags/…/<tag>.tag`:

- `Tag:` — this is our Tag string, verbatim.
- `Severity:` / experimental — default Severity and whether `default_selected` is true (error and warning, non-experimental).
- `Check:` — path of the Perl check (`fields/required` → `Lintian::Check::Fields::Required`).
- `Renamed-From:` — old names; we still key on the current Tag.

Classification tags are not Rules.

## 2. Catalogue the Rule

Add a unit struct named from the Tag (`RequiredField`) under `src/lint/rules/lintian/<tag>.rs` (or `src/lint/rules/native/<tag>.rs` for a DM Rule), `declare_rule!` with a new `LNxxxx` / `DMxxxx` Code, `impl` the subject trait for each kind the Rule inspects now (`SourceTreeRule`, `BinaryPackageRule`, `SourcePackageRule`), and register it in `registry.rs` (the metadata catalog and each kind list whose trait it implements). Lintian files may later split into further subfolders under `lintian/`.

Until we generate Codes from the full Tag list, assign the next free `LN` Code when the Tag is catalogued and do not reuse it.

## 3. See what the check inspects

Read the Check module. Map I/O to the Subject’s context (Debian control, changelog, file index, …). Implement only the subject traits for kinds you inspect; a kind without a trait impl is not Applicability.

## 4. Native tests first

`SourceTreeTester` takes pass/fail Source trees as path → content maps and snapshots fail Diagnostics. That is **not** Parity. Use it to lock the Rule’s behaviour on a few trees (including udeb / missing-file edges the recipes never isolate). `BinaryPackageTester` and `SourcePackageTester` exist as names; they are not implemented yet.

Keep expected extra/context in the message in lintian’s order (for `required-field`: `(in section for …)` then the field name). Pointers like `[debian/control:1]` belong on `Location` when we have them; they are stripped from `eval/hints` extras and are not part of Parity.

## 5. Find upstream recipes

Recipes live at:

```text
t/recipes/checks/<check>/
```

for example `t/recipes/checks/fields/required/`. Each recipe has `build-spec/` (how to build) and `eval/hints` (expected universal output).

`eval/desc` lists a **Check**, not a Tag (`Test-For` is gone). Selecting `tag:required-field` in lintian’s runner therefore means “every recipe for `fields/required`”. Grep `eval/hints` for the Tag as the **third field** (a substring hits `doc-base-file-lacks-required-field`).

Classify each recipe by skeleton (`build-spec/fill-values` → `Skeleton:`):

| Skeleton | Artifact | Debmagic Subject today |
| --- | --- | --- |
| `upload-native` / `upload-non-native` | `.changes` via `dpkg-buildpackage` | Source tree *before* the build; Source package / Binary package after |
| `source-native` / `source-non-native` | `.dsc` | Source package; Source tree after fill |
| `deb` | hand-built `.deb` | Binary package |
| `changes` | `.changes` | none yet |

A single `hints` file often mixes `(source)` and `(binary)` lines. Import only the lines that match the Subject you are implementing. Drop `(source)` extras that name a `.dsc` until Source-package tests exist.

## 6. Why recipes are not a copy-paste into `SourceTreeTester`

A recipe is a **spec**: skeleton templates (`t/templates/…`), `[% $source %]` fills, optional `pre-build`, then usually `dpkg-buildpackage`. `build-spec/debian/` is an overlay, not a complete Source tree (there is no `debian/debian/` in 2.139.0).

The oracle is `eval/hints` in **universal** format:

```text
package (source|binary|changes|udeb): tag-name extra… [optional-pointer]
```

not EWI (`E: pkg: tag`) and not our Diagnostic stdout.

Do not submodule lintian or read `/usr/share/lintian` at test time. Native `SourceTreeTester` cases stay hand-written. Parity recipes are **filled sources** plus transcribed `eval/hints` (GPL-2+), with an `ORIGIN` pointer to `lintian 2.139.0 t/recipes/…` — not the raw `build-spec`, and not a built `.deb` / `.dsc`. They live under `packages/debmagic/tests/lintian-parity/`. Source-tree Parity lints that tree as-is. Binary-package Parity for `deb`-skeleton recipes assembles lintian’s Build-Product (`make` / `ar` / `tar`) in the test and lints that `.deb`. Source-package and upload-native `(binary)` lines wait.

Import or refresh the full Parity suite from a lintian version tag (every importable recipe: source-tree, deb, and changes):

```shell
python3 scripts/import_lintian_parity.py 2.139.0
```

Reuse a local clone:

```shell
python3 scripts/import_lintian_parity.py 2.139.0 --lintian-src /path/to/lintian
```

Import also rewrites `packages/debmagic/tests/lintian_parity.rs` with one test per vendored Source-tree or `deb`-skeleton recipe whose `eval/hints` mention a catalogued LN Tag. After cataloguing a Rule without re-filling recipes:

```shell
python3 scripts/import_lintian_parity.py --write-tests
```

## 7. Worked sketch: `required-field`

Check: `lib/Lintian/Check/Fields/Required.pm`. Tag file: `tags/r/required-field.tag`. Three recipes:

- **`generic-empty`** — `upload-native`; `control.in` omits `Standards-Version` and `Description`; `pre-build` deletes `debian/compat` and `debian/copyright`. Source-tree extras: `(in section for source) Standards-Version` and `(in section for generic-empty) Description`. Also `.dsc` and `.deb` lines — skip those until Source package / upload builds exist.
- **`fields-general-missing`** — `deb`; Binary package Parity (assemble the Build-Product, lint the `.deb`).
- **`changes-missing-fields`** — `changes`; no Subject yet.

`LN0001` Source-tree Parity is `generic-empty` under `tests/lintian-parity/` (two `debian/control` hint lines). Binary-package Parity is `fields-general-missing`. `.dsc` lines wait for Source package.
1 change: 1 addition & 0 deletions docs/develop/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,5 @@ uv run sphinx-autobuild docs docs/_build

```{toctree}

import-lintian-rule.md
```
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ usage/modules/index.md
:caption: Development

develop/index.md
develop/import-lintian-rule.md
develop/releasing.md
```

Expand Down
Loading
Loading