Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Pipfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
biocsetup = {path = ".", editable = true}
hatchit = {path = "../hatchit", editable = true}

[dev-packages]
tox = "*"
pytest = "*"
click = "*"

[requires]
python_version = "3.14"
827 changes: 827 additions & 0 deletions Pipfile.lock

Large diffs are not rendered by default.

6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

BiocSetup helps scaffold new Python packages in [BiocPy](https://github.com/biocpy) with consistent configuration for package management.

It automates the setup process by using PyScaffold with additional configurations specific to BiocPy projects, including documentation setup, GitHub Actions for testing and publishing, and code quality tools.
It automates the setup process by using [hatchit](https://github.com/BiocPy/hatchit) (by default) or [PyScaffold](https://pyscaffold.org/) with additional configurations specific to BiocPy projects, including documentation setup, GitHub Actions for testing and publishing, and code quality tools.

For more details, see our [developer guide](https://github.com/BiocPy/developer_guide).

Expand All @@ -31,6 +31,7 @@ Options:
- `--description`, `-d`: Project description
- `--license`, `-l`: License to use (default: MIT)
- `--rst`: To use reStructuredText, otherwise uses Markdown by default.
- `--pyscaffold`, `-ps`: To use `PyScaffold` for initialization instead of `hatchit`.

### Python API

Expand All @@ -44,14 +45,15 @@ create_repository(
description="Description of my package",
license="MIT",
rst=False,
use_pyscaffold=False,
)
```

## After setup

- The GitHub workflows use "trusted publisher workflow" to publish packages to PyPI. Read more instructions [here](https://docs.pypi.org/trusted-publishers/).
- Tagging the repository will trigger an action to test, generate documentation, and publish the package to PyPI.
- Install [tox](https://tox.wiki/en/4.23.2/) to handle package tasks. GitHub Actions relies on the tox configuration to test, generate documentation, and publish packages.
- Install [hatchit](https://github.com/BiocPy/hatchit) to handle package tasks. GitHub Actions relies on hatchit (if using the default setup) or tox (if using PyScaffold) to test, generate documentation, and publish packages.
- (Optional) Enable the [pre-commit.ci](https://pre-commit.ci/) bot for your repository.
- (Optional) Install [ruff](https://docs.astral.sh/ruff/) for code formatting.
- (Optional) Setup [codecov](https://about.codecov.io/) for coverage reports.
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
try:
import sphinx

cmd_line = f"sphinx-apidoc --implicit-namespaces -f -o {output_dir} {module_dir}"
cmd_line = f"sphinx-apidoc --implicit-namespaces -f -o {output_dir} {module_dir} {os.path.join(module_dir, 'templates')}"

args = cmd_line.split(" ")
if tuple(sphinx.__version__.split(".")) >= ("1", "7"):
Expand Down
6 changes: 4 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

BiocSetup helps scaffold new Python packages in [BiocPy](https://github.com/biocpy) with consistent configuration for package management.

It automates the setup process by using PyScaffold with additional configurations specific to BiocPy projects, including documentation setup, GitHub Actions for testing and publishing, and code quality tools.
It automates the setup process by using [hatchit](https://github.com/BiocPy/hatchit) (by default) or [PyScaffold](https://pyscaffold.org/) with additional configurations specific to BiocPy projects, including documentation setup, GitHub Actions for testing and publishing, and code quality tools.

For more details, see our [developer guide](https://github.com/BiocPy/developer_guide).

Expand All @@ -27,6 +27,7 @@ Options:
- `--description`, `-d`: Project description
- `--license`, `-l`: License to use (default: MIT)
- `--rst`: To use reStructuredText, otherwise uses Markdown by default.
- `--pyscaffold`, `-ps`: To use `PyScaffold` for initialization instead of `hatchit`.

### Python API

Expand All @@ -40,14 +41,15 @@ create_repository(
description="Description of my package",
license="MIT",
rst=False,
use_pyscaffold=False,
)
```

## After setup

- The GitHub workflows use "trusted publisher workflow" to publish packages to PyPI. Read more instructions [here](https://docs.pypi.org/trusted-publishers/).
- Tagging the repository will trigger an action to test, generate documentation, and publish the package to PyPI.
- Install [tox](https://tox.wiki/en/4.23.2/) to handle package tasks. GitHub Actions relies on the tox configuration to test, generate documentation, and publish packages.
- Install [hatchit](https://github.com/BiocPy/hatchit) to handle package tasks. GitHub Actions relies on hatchit (if using the default setup) or tox (if using PyScaffold) to test, generate documentation, and publish packages.
- (Optional) Enable the [pre-commit.ci](https://pre-commit.ci/) bot for your repository.
- (Optional) Install [ruff](https://docs.astral.sh/ruff/) for code formatting.
- (Optional) Setup [codecov](https://about.codecov.io/) for coverage reports.
Expand Down
28 changes: 15 additions & 13 deletions docs/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ This guide presents a consistent workflow to make Python packaging less painful
## Overview of this document

1. **Package structure:** Using a scaffolder (`BiocSetup & PyScaffold`) gives every project the same starting structure.
2. **Isolation for testing:** `tox` runs tests in clean, isolated environments. This mimics how your code will run elsewhere and catches issues before your users.
2. **Isolation for testing:** `hatchit` (or `tox` if using PyScaffold) runs tests in clean, isolated environments. This mimics how your code will run elsewhere and catches issues before your users.
3. **The `src` layout:** Putting your code in `src/package_name` prevents a common pitfall: accidentally importing your local code instead of the installed version during testing.
4. **Automation:** GitLab CI/CD or GitHub Actions handle testing, documentation builds, and publishing. Set it up once, and let the bots do all the work.
5. **Release and development cycles:** We strictly separate development (branches) from releases (tags), preventing accidental releases to PyPI or interfering with development cycles. For multi-developer projects, try to maintain master/main in a functional state and put incomplete work within a feature branch. Depending upon team size and the value of peer review, consider using pull requests prior to merging into main/master. Use [semantic versioning](https://semver.org/) for tags.

## Packaging setup

[BiocSetup](https://github.com/BiocPy/BiocSetup) (based on [PyScaffold](https://pyscaffold.org/en/stable/)) automates some of the common configurtion we use across all BiocPy packages. Yes, it's a package to create packages—very meta I know :).
[BiocSetup](https://github.com/BiocPy/BiocSetup) (based on [hatchit](https://github.com/BiocPy/hatchit) or optionally [PyScaffold](https://pyscaffold.org/en/stable/)) automates some of the common configurtion we use across all BiocPy packages. Yes, it's a package to create packages—very meta I know :).

First, install `biocsetup` if you haven't already (`pip install biocsetup`).

Expand Down Expand Up @@ -48,14 +48,16 @@ This command creates a complete project structure:
├── setup.py # Mostly a shim for compatibility now, config is in setup.cfg/pyproject.toml
├── src # <--- YOUR CODE GOES HERE!
├── tests # <--- YOUR TESTS GO HERE!
└── tox.ini # Configuration for testing and other tasks (tox)
└── tox.ini # Configuration for testing and other tasks (tox; only if using PyScaffold)

4 directories, 9 files

```

**Markdown vs. reStructuredText:** By default, `biocsetup` uses Markdown (`.md`) as the preferred format for documentation. If you're a fan of [reStructuredText](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html) (`.rst`) or just enjoy slightly more cryptic syntax, add the `--rst` flag when running `biocsetup`. The only noticeable difference will be file extensions that make your eyes bleed (`.rst` instead of `.md`).

**Using PyScaffold for initialization:** By default, `biocsetup` relies on `hatchit` to initialize the project structure. If you prefer to use [PyScaffold](https://pyscaffold.org/) for initialization, simply add the `--pyscaffold` (or `-ps`) flag to the command.

## Adding your source code

All your Python source code goes inside the `src` directory, specifically within the subdirectory named after your package (e.g., `src/my_awesome_package/`).
Expand Down Expand Up @@ -84,12 +86,12 @@ tests/
└── test_edge_cases.py
```

- **Running tests:** This is where [`tox`](https://tox.wiki/en/4.26.0/) shines. It reads the `tox.ini` file, creates a temporary virtual environment, installs your package and its dependencies _exactly_ as defined, and then runs `pytest`. This ensures your tests run in a clean, reproducible environment, mimicking how users will install your package.
- **Running tests:** We use [`hatchit`](https://github.com/BiocPy/hatchit) (or [`tox`](https://tox.wiki/en/4.26.0/) if you opted for PyScaffold). These tools create a temporary virtual environment, install your package and its dependencies _exactly_ as defined, and then run `pytest`. This ensures your tests run in a clean, reproducible environment, mimicking how users will install your package. Run `hatchit test` (or `tox`) to run the tests.

To run the default test suite (which includes running `pytest` and checking code coverage):

```sh
⋊> ~/P/s/my-awesome-package on master tox
⋊> ~/P/s/my-awesome-package on master hatchit test
.pkg: install_requires> python -I -m pip install 'setuptools>=46.1.0' 'setuptools_scm[toml]>=5'
..............
..............
Expand Down Expand Up @@ -117,7 +119,7 @@ What's neat is that you also get coverage reports, which tell you which parts of

### "Help\! My Dependencies Are Missing\!"

Your package probably uses other Python libraries (NumPy, Pandas, etc.). If these are not listed as dependencies, your isolated tox environment does not install them and you will run into errors about missing packages. Open `setup.cfg` and look for this section:
Your package probably uses other Python libraries (NumPy, Pandas, etc.). If these are not listed as dependencies, your isolated test environment does not install them and you will run into errors about missing packages. Depending on your setup (hatchit vs pyscaffold), open `pyproject.toml` (and look for `dependencies`) or `setup.cfg` (and look for `install_requires`):

```yaml
[options]
Expand Down Expand Up @@ -151,12 +153,12 @@ The scaffolding process sets up the `docs` directory with a default theme ([furo
2. Link to your new page from `docs/index.md` (or `docs/index.rst`) under the `toctree` (Table of Contents Tree) directive.
3. By default, `docs/changelog.md` includes the content of `/CHANGELOG.md`, so you only have to update the root changelog.
4. Write detailed docstrings and use [autodoc](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html) for API documentation. Docstring format is to taste, but [Google](https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings) or [NumPy](https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard) style are easily readable and both can be parsed by the [napoleon](https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html) Sphinx extension.
- **Building the docs:** Use `tox` again\! This ensures the docs build in a clean environment with all necessary extensions.
- **Building the docs:** Use `hatchit docs` (or `tox -e docs` if using PyScaffold)\! This ensures the docs build in a clean environment with all necessary extensions.

To generate the HTML files for the documentation:

```sh
⋊> ~/P/s/my-awesome-package on master tox -e docs
⋊> ~/P/s/my-awesome-package on master hatchit docs
docs: install_deps> python -I -m pip install -r /Users/kancherj/Projects/scratch/my-awesome-package/docs/requirements.txt
..............
..............
Expand Down Expand Up @@ -197,10 +199,10 @@ git checkout -b feature/perf
```

2. **Code & test:** Write your code in `src/`, add corresponding tests in `tests/`.
3. **Test locally:** Run `tox` frequently\! Catch errors early.
3. **Test locally:** Run `hatchit test` (or `tox`) frequently\! Catch errors early.

```sh
tox
hatchit test
```

4. **Commit & push:** Make small, logical commits. Push your branch to GitHub. If you want to follow a structure for commits, [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) is very helpful.
Expand All @@ -212,7 +214,7 @@ git push origin feature/perf
```

5. **Pull Request (PR):** Go to GitHub and open a Pull Request from your branch to `main`. Describe your changes.
6. **CI checks:** GitHub Actions (configured in `.github/workflows/run-tests.yml`) will automatically run `tox` on your PR to ensure tests pass on different Python versions and platforms (windows, mac and linux).
6. **CI checks:** GitHub Actions (configured in `.github/workflows/run-tests.yml`) will automatically run `hatchit test` (or `tox`) on your PR to ensure tests pass on different Python versions and platforms (windows, mac and linux).
7. **Merge:** Once reviewed and CI passes, merge the PR into `main`.
8. **Repeat:** Continue the cycle for the next feature or fix.

Expand Down Expand Up @@ -293,9 +295,9 @@ Consistent code is easier to read, review, and maintain.
- Get familiar with [git](https://git-scm.com/), or if you want to get fancy, use [jujustu](https://github.com/jj-vcs/jj)
- There are many alternatives to each of the tools mentioned here
- [quartodoc](https://github.com/machow/quartodoc)/[mkdocs](https://www.mkdocs.org/) instead of Sphinx
- [nox](https://nox.thea.codes/en/stable/) instead of tox
- [nox](https://nox.thea.codes/en/stable/) or [tox](https://tox.wiki/en/4.26.0/) instead of hatchit
- [poetry](https://python-poetry.org/) and [hatch](https://hatch.pypa.io/) (probably go with the later since its developed by the official PyPI working group on packaging) instead of setuptools

## Conclusion

And that's the gist of it\! It might seem like a lot initially, but once you go through the cycle a couple of times, it becomes second nature. Remember the key steps: scaffold, code in `src/`, test in `tests/` (run with `tox`), manage dependencies in `setup.cfg`, document in `docs/` (build with `tox -e docs`), use branches for development, and tag for releases.
And that's the gist of it\! It might seem like a lot initially, but once you go through the cycle a couple of times, it becomes second nature. Remember the key steps: scaffold, code in `src/`, test in `tests/` (run with `hatchit test`), manage dependencies, document in `docs/` (build with `hatchit docs`), use branches for development, and tag for releases.
2 changes: 2 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,12 @@ python_requires = >=3.9
# new major versions. This works if the required packages follow Semantic Versioning.
# For more information, check out https://semver.org/.
install_requires =
hatchit
importlib-metadata; python_version<"3.8"
pyscaffold>=4.0
pyscaffoldext-markdown
click>=8.0
uv

[options.packages.find]
where = src
Expand Down
4 changes: 3 additions & 1 deletion src/biocsetup/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@
@click.option("--description", "-d", help="Project description", default="Add a short description here!")
@click.option("--license", "-l", default="MIT", help="License (default: MIT)")
@click.option("--rst", "-rst", is_flag=True, help="Use rst for documentation, defaults to using markdown!")
def main(project_path: str, description: str, license: str, rst: bool):
@click.option("--pyscaffold", "-ps", is_flag=True, help="Use pyscaffold for project initialization instead of hatchit.")
def main(project_path: str, description: str, license: str, rst: bool, pyscaffold: bool):
"""Create a new BiocPy Python package."""
create_repository(
project_path=project_path,
description=description,
license=license,
rst=rst,
use_pyscaffold=pyscaffold,
)


Expand Down
27 changes: 25 additions & 2 deletions src/biocsetup/create_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ def create_repository(
description: str | None = "Add a short description here!",
license: str = "MIT",
rst: bool = False,
use_pyscaffold: bool = False,
) -> None:
"""
Create a new BiocPy Python package repository.
Expand All @@ -33,6 +34,18 @@ def create_repository(
Whether to use 'markdown' or 'rst'.
Defaults to False, to use 'markdown'.
"""
if not use_pyscaffold:
try:
from hatchit.scaffold import create_hatchit_repository

return create_hatchit_repository(project_path, description, license)
except ImportError:
import sys

print("Error: The 'hatchit' package is required for default scaffolds.")
print("Please install it in your environment: pip install hatchit")
sys.exit(1)

# Create project using pyscaffold with markdown extension
if description is None:
description = "Add a short description here!"
Expand Down Expand Up @@ -108,7 +121,7 @@ def create_repository(
# Update requirements.txt for docs
docs_requirements = Path(project_path) / "docs" / "requirements.txt"
with open(docs_requirements, "a") as f:
f.write("myst-nb\nfuro\nsphinx-autodoc-typehints\n")
f.write("myst-nb\nfuro\nsphinx-autodoc-typehints\nlinkify-it-py\n")
modified_files.append(docs_requirements)

# Modify README
Expand Down Expand Up @@ -177,6 +190,16 @@ def create_repository(
for f in modified_files:
shell.git("add", str(f.relative_to(project_path)))

shell.git("commit", "-m", "BiocSetup configuration")
try:
# Check if git user config exists before committing
import subprocess

name_configured = subprocess.run(["git", "config", "user.name"], capture_output=True).returncode == 0
email_configured = subprocess.run(["git", "config", "user.email"], capture_output=True).returncode == 0

if name_configured and email_configured:
shell.git("commit", "-m", "BiocSetup configuration")
except Exception:
pass # Git commit is optional

print("BiocSetup complete! 🚀 💥")
29 changes: 29 additions & 0 deletions src/biocsetup/templates/docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build
AUTODOCDIR = api

# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $?), 1)
$(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from https://sphinx-doc.org/")
endif

.PHONY: help clean Makefile

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

clean:
rm -rf $(BUILDDIR)/* $(AUTODOCDIR)

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
1 change: 1 addition & 0 deletions src/biocsetup/templates/docs/_static/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Empty directory
4 changes: 4 additions & 0 deletions src/biocsetup/templates/docs/authors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
```{include} ../AUTHORS.md
:relative-docs: docs/
:relative-images:
```
4 changes: 4 additions & 0 deletions src/biocsetup/templates/docs/changelog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
```{include} ../CHANGELOG.md
:relative-docs: docs/
:relative-images:
```
Loading
Loading