Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,12 +169,14 @@ The project provides type-safe builder classes for different environment types:
- Supports `pixi.toml` and `environment.yml` files
- Uses `pixi run --manifest-path <envDir>/pixi.toml` for activation
- Environment structure: `<envDir>/.pixi/envs/default`
- Lock files: `.lockFile()/.lockContent()/.lockUrl()` for reproducible builds (copies `pixi.lock` into envDir, installs with `pixi install --locked`)
- Location: `org.apposed.appose.builder.PixiBuilder`

**MambaBuilder** - Traditional conda environments via micromamba
- Created via `Appose.mamba()` or `Appose.mamba(source)`
- Supports `environment.yml` files
- Uses `mamba run -p <envDir>` for activation
- Does not yet support lock files
- Location: `org.apposed.appose.builder.MambaBuilder`

**UvBuilder** - Fast Python virtual environments via uv
Expand All @@ -183,6 +185,7 @@ The project provides type-safe builder classes for different environment types:
- Supports `requirements.txt` and `pyproject.toml`
- Standard Python venv structure (no special activation needed)
- Environment structure: `<envDir>/bin` (or `Scripts` on Windows)
- Lock files: `.lockFile()/.lockContent()/.lockUrl()` for reproducible builds (copies `uv.lock` into envDir, installs with `uv sync --locked`; requires a `pyproject.toml` declaration)
- Location: `org.apposed.appose.builder.UvBuilder`

**DynamicBuilder** - Auto-detects appropriate builder based on configuration content
Expand All @@ -196,6 +199,7 @@ The project provides type-safe builder classes for different environment types:
- Created via `Appose.custom()` or implicitly via `Appose.system()`
- No package installation; uses whatever executables are on the system
- Methods: `binPaths(paths...)`, `appendSystemPath()`, `inheritRunningJava()`
- Does not support lock files (no package management)
- Location: `org.apposed.appose.builder.SimpleBuilder`

### API Examples
Expand Down Expand Up @@ -241,6 +245,16 @@ Environment env = Appose.file("path/to/environment.yml")
.logDebug()
.build();

// Reproducible build pinned by a lock file (uv: uv.lock -> uv sync --locked)
Environment env = Appose.uv("path/to/pyproject.toml")
.lockFile("path/to/uv.lock")
.build();

// Reproducible build pinned by a lock file (pixi: pixi.lock -> pixi install --locked)
Environment env = Appose.pixi("path/to/pixi.toml")
.lockFile("path/to/pixi.lock")
.build();

// Wrap existing environment
Environment env = Appose.wrap("/path/to/existing/env");

Expand Down Expand Up @@ -270,6 +284,28 @@ All builders support subscription methods for monitoring:
- `subscribeError(consumer)` - Error output from build process
- `logDebug()` - Convenience method that logs output and errors to stderr

### Lock Files & Reproducible Builds

`PixiBuilder` and `UvBuilder` support reproducible, lock-file-pinned builds via the
`lockContent(String)`, `lockFile(String|File)`, and `lockUrl(String|URL)` methods (mirroring the
`content`/`file`/`url` declaration API). When a lock is supplied:

- The lock is copied into the environment directory (`uv.lock` / `pixi.lock`) before install.
- The install runs with `--locked` (`uv sync` / `pixi install`), so the environment matches the lock
exactly, and the build fails if the lock is out of date with the manifest.
- uv requires a `pyproject.toml` declaration; pixi requires `pixi.toml` or `pyproject.toml`, and
rejects programmatic `channels()` (which would rewrite the lock).
- The `appose.json` state snapshot records a `lockHash` (SHA-256 of the lock content), so a change to
the lock forces a rebuild via the normal `isUpToDate()` check.
- `DynamicBuilder` (`Appose.file/url/content`) forwards the lock to the detected pixi/uv builder.
- `wrap()` restores the lock only if `appose.json` has a `lockHash`, since pixi and uv write a lock
file even for lock-less builds.
- `MambaBuilder` and `SimpleBuilder` do **not** support lock files and throw
`UnsupportedOperationException`.

When no lock is supplied, behavior is unchanged: no strict flag is passed and `appose.json` omits
`lockHash`, so the snapshot is byte-identical to pre-lock-file builds (no spurious rebuilds).

## Related Projects

- appose-python: Python implementation of Appose (https://github.com/apposed/appose-python)
Expand Down
107 changes: 88 additions & 19 deletions src/main/java/org/apposed/appose/Builder.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,13 @@

package org.apposed.appose;

import java.io.ByteArrayOutputStream;
import org.apposed.appose.util.Downloads;
import org.apposed.appose.util.FilePaths;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -238,12 +236,7 @@ default T file(String path) throws BuildException {
*/
default T file(File file) throws BuildException {
try {
Path filePath = file.toPath();
String fileContent = new String(
Files.readAllBytes(filePath),
StandardCharsets.UTF_8
);
return content(fileContent);
return content(FilePaths.readText(file));
}
catch (IOException e) {
throw new BuildException(this, e);
Expand Down Expand Up @@ -276,14 +269,8 @@ default T url(String path) throws BuildException {
* @throws BuildException If the URL cannot be read
*/
default T url(URL url) throws BuildException {
try (InputStream stream = url.openStream()) {
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
int length;
while ((length = stream.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return content(result.toString(StandardCharsets.UTF_8.name()));
try {
return content(Downloads.readText(url));
}
catch (IOException e) {
throw new BuildException(this, e);
Expand All @@ -308,6 +295,88 @@ default T url(URL url) throws BuildException {
*/
T scheme(String scheme);

/**
* Specifies lock file content for reproducible builds. When provided, the
* lock file is copied into the environment directory, and the environment
* is installed strictly from it (via {@code --locked}), failing if the
* lock is out of date with the configuration file.
* <p>
* Not all builders support lock files; builders that do not will throw
* {@link UnsupportedOperationException}.
* </p>
*
* @param lockContent Lock file content (e.g., uv.lock, pixi.lock)
* @return This builder instance, for fluent-style programming.
* @throws UnsupportedOperationException If this builder does not support lock files.
*/
default T lockContent(String lockContent) {
throw new UnsupportedOperationException(
getClass().getSimpleName() + " does not support lock files");
}

/**
* Specifies a lock file path for reproducible builds.
* Reads the file content immediately and delegates to {@link #lockContent(String)}.
*
* @param path Path to the lock file (e.g., "uv.lock", "pixi.lock")
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the file cannot be read
*/
default T lockFile(String path) throws BuildException {
return lockFile(new File(path));
}

/**
* Specifies a lock file for reproducible builds.
* Reads the file content immediately and delegates to {@link #lockContent(String)}.
*
* @param file Lock file (e.g., uv.lock, pixi.lock)
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the file cannot be read
*/
default T lockFile(File file) throws BuildException {
try {
return lockContent(FilePaths.readText(file));
}
catch (IOException e) {
throw new BuildException(this, e);
}
}

/**
* Specifies a URL to fetch lock file content from for reproducible builds.
* Reads the URL content immediately and delegates to {@link #lockContent(String)}.
*
* @param path URL path of the lock file
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the URL cannot be read or is invalid
*/
default T lockUrl(String path) throws BuildException {
try {
return lockUrl(new URL(path));
}
catch (MalformedURLException e) {
throw new BuildException(this, e);
}
}

/**
* Specifies a URL to fetch lock file content from for reproducible builds.
* Reads the URL content immediately and delegates to {@link #lockContent(String)}.
*
* @param url URL to the lock file
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the URL cannot be read
*/
default T lockUrl(URL url) throws BuildException {
try {
return lockContent(Downloads.readText(url));
}
catch (IOException e) {
throw new BuildException(this, e);
}
}

/**
* Registers a callback method to be invoked when progress happens during environment building.
*
Expand Down
85 changes: 85 additions & 0 deletions src/main/java/org/apposed/appose/builder/BaseBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
Expand Down Expand Up @@ -73,6 +75,9 @@ public abstract class BaseBuilder<T extends BaseBuilder<T>> implements Builder<T
/** Configuration file content. */
protected String content;

/** Lock file content for reproducible builds (e.g., uv.lock, pixi.lock), or null when unset. */
protected String lockContent;

/** Explicit scheme (e.g., "pixi.toml", "environment.yml"). */
protected Scheme scheme;

Expand Down Expand Up @@ -155,6 +160,12 @@ public T content(String content) {
return typedThis();
}

@Override
public T lockContent(String lockContent) {
this.lockContent = lockContent;
return typedThis();
}

@Override
public T scheme(String scheme) {
this.scheme = Schemes.fromName(scheme);
Expand Down Expand Up @@ -208,6 +219,38 @@ protected void addStateFields(Map<String, Object> state) {
state.put("channels", channels);
state.put("flags", flags);
state.put("envVars", new TreeMap<>(envVars));
// Record a hash of the lock file content so that lock changes trigger a
// rebuild via the exact-match isUpToDate() comparison. Only added when a
// lock is supplied, so lock-less builds produce a byte-identical
// appose.json (backward compatibility).
if (lockContent != null) state.put("lockHash", computeLockHash(lockContent));
}

/**
* Computes a SHA-256 hash (lowercase hex) of the given content, or null if
* the content is null. Used to snapshot lock files into {@code appose.json}
* without storing the (potentially large) lock content verbatim.
*
* @param content The content to hash (e.g., lock file content).
* @return The SHA-256 hex hash, or null if content is null.
*/
protected static String computeLockHash(String content) {
if (content == null) return null;
try {
MessageDigest md = MessageDigest.getInstance("SHA-256");
byte[] digest = md.digest(content.getBytes(StandardCharsets.UTF_8));
char[] hex = new char[digest.length * 2];
for (int i = 0; i < digest.length; i++) {
int v = digest[i] & 0xff;
hex[i * 2] = Character.forDigit(v >>> 4, 16);
hex[i * 2 + 1] = Character.forDigit(v & 0x0f, 16);
}
return new String(hex);
}
catch (NoSuchAlgorithmException e) {
// SHA-256 is mandated by the JVM specification; this never happens.
throw new RuntimeException("SHA-256 algorithm not available", e);
}
}

/**
Expand Down Expand Up @@ -239,6 +282,48 @@ protected void writeApposeStateFile(File envDir) throws IOException {
Files.write(apposeJson.toPath(), buildStateString().getBytes(StandardCharsets.UTF_8));
}

/**
* Reads the builder state recorded in {@code appose.json} in the given directory.
*
* @param envDir The environment directory.
* @return The recorded state, or null if absent or unreadable.
* @throws IOException If reading {@code appose.json} fails.
*/
protected static Map<?, ?> readApposeState(File envDir) throws IOException {
File apposeJson = new File(envDir, "appose.json");
if (!apposeJson.isFile()) return null;
Object state;
try {
state = Json.parseJson(FilePaths.readText(apposeJson));
}
catch (RuntimeException e) {
return null; // Unreadable state; the env will just look stale.
}
return state instanceof Map ? (Map<?, ?>) state : null;
}

/**
* Restores the lock file content of a wrapped environment, so that
* {@link #rebuild()} reproduces it even after the directory is deleted.
* <p>
* Note: package managers write a lock file even for lock-less builds, so
* the lock is only restored if {@code appose.json} records that the
* environment was built from one.
* </p>
*
* @param envDir The environment directory.
* @param lockFileName Name of the lock file (e.g., "uv.lock", "pixi.lock").
* @throws IOException If reading the state or lock file fails.
*/
protected void restoreLockContent(File envDir, String lockFileName) throws IOException {
if (lockContent != null) return;
Map<?, ?> state = readApposeState(envDir);
if (state == null || !state.containsKey("lockHash")) return;
File lockFile = new File(envDir, lockFileName);
if (!lockFile.isFile()) return;
lockContent = FilePaths.readText(lockFile);
}

/**
* Tests whether the given directory actually contains a usable environment
* of this builder's type, regardless of who built it. Used by {@link #status()}.
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/org/apposed/appose/builder/DynamicBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ private void copyConfigToDelegate(Builder<?> delegate) {
if (envDir != null) delegate.base(envDir);
if (content != null) delegate.content(content);
if (scheme != null) delegate.scheme(scheme.name());
// Forward the lock file if one was provided. Builders that do not
// support locks (mamba/custom) override lockContent() to throw.
if (lockContent != null) delegate.lockContent(lockContent);
delegate.channels(channels);
progressSubscribers.forEach(delegate::subscribeProgress);
outputSubscribers.forEach(delegate::subscribeOutput);
Expand Down
6 changes: 6 additions & 0 deletions src/main/java/org/apposed/appose/builder/MambaBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,12 @@ public String envType() {
return "mamba";
}

@Override
public MambaBuilder lockContent(String lockContent) {
throw new UnsupportedOperationException(
"MambaBuilder does not yet support lock files");
}

@Override
protected boolean hasEnvironment(File envDir) {
return new File(envDir, "conda-meta").isDirectory();
Expand Down
Loading
Loading