Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
janickm
commented
Sep 18, 2026
| ) | ||
|
|
||
| @property | ||
| def camera_ray_dim(self) -> int: |
Collaborator
Author
There was a problem hiding this comment.
should this be a class-wide or even static property?
Collaborator
Author
There was a problem hiding this comment.
Class-wide, agreed — it never depends on instance state. Now a ClassVar[int], following the precedent at types.py:1568. Verified it works on an nn.Module subclass: readable from both class and instance, doesn't land in state_dict(), survives .to(). Test added.
Adds `IdealOrthographicCameraModel` / `IdealOrthographicCameraModelParameters`, a distortion-free parallel projection that drops the camera-frame depth instead of dividing by it. Serialized as `ideal-orthographic`. This is NCore's first *non-central* camera model: its rays are parallel and share no common origin, so a 3d direction no longer identifies a ray. Unprojection therefore returns 6d `[origin, direction]` rays, matching the layout `WorldRaysReturn` already uses, and `camera_ray_dim` reports which representation a model uses. Origins and directions alike are relative to the extrinsic camera frame's origin. The two directions of the ray interface are asymmetric, which is what keeps the change small: projection receives an unnormalized camera-frame *point* for every model, so the `world_points_to_*` family and the rolling-shutter solver work unchanged, and rolling shutter comes for free. Only the unprojection path needed generalizing, where per-ray origins are now carried through the sensor pose as `R @ origin + t` rather than broadcasting the pose translation. Central models are unaffected: their origins are zero, so the same expression reproduces the previous broadcast exactly. Two combinations are refused because a parallel ray bundle makes them ill-defined rather than merely unimplemented: - External distortion deflects each ray individually and so would not preserve the parallel bundle. Rejected when the model is constructed, since the forward path alone would not surface the mismatch. - Rectification against a central model has no depth-independent solution, as there is no common centre to pivot about. Rectifying two non-central models is well-defined but not implemented yet. `paraxial_pinhole_geometry()` raises via the documented opt-out: an orthographic camera's focal length is infinite, so it has no pinhole approximation. The intrinsics are `principal_point` plus `pixels_per_unit`, the orthographic counterpart of a pinhole's `focal_length`, differing in that its input carries scene units. Named per-unit rather than per-meter because coordinates may be `UNITLESS`. `window_min()` / `window_max()` / `from_window()` restate the same intrinsics as the metric window the camera views, for consumers working in normalized image coordinates. Renames the forward projection to say what it takes: camera_rays_to_image_points -> camera_points_to_image_points camera_rays_to_pixels -> camera_points_to_pixels The argument was always an unnormalized camera-frame point, never a ray. With only central models the distinction was unobservable, because every central projection is scale-invariant: the pinholes divide by depth and the FTheta and fisheye models take an angle about the projection centre, so any point along a given ray projects identically and the magnitude is discarded. An orthographic projection has no such freedom - the point's x and y are precisely the quantity being projected - so passing a normalized direction silently yields a different image location rather than an error. The old names are kept as documented-deprecated forwarders, following the docstring-only deprecation style already used in this module. Unprojection keeps its name, as `image_points_to_camera_rays` genuinely returns rays. Also refactors `ExternalDistortionModel` to guard the ray representation in its public methods and dispatch to `_distort_camera_rays_impl` / `_undistort_camera_rays_impl`, mirroring `CameraModel`, and extracts the image-domain scale/resolution prelude duplicated across all four `transform()` implementations. Note the `ExternalDistortionModel` refactor renames the two abstract methods an external distortion model implements. There are no out-of-tree implementations of the abstract base, and models deriving from a concrete class such as `BivariateWindshieldModel` inherit the new hooks unchanged. No serialized format changed.
janickm
force-pushed
the
dev/janickm/ideal-orthographic-camera-model
branch
from
September 18, 2026 10:04
e95d3ba to
75074c1
Compare
janickm
marked this pull request as ready for review
September 18, 2026 10:04
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an ideal orthographic camera model (
ideal-orthographic): adistortion-free parallel projection that drops the camera-frame depth instead of
dividing by it, so a point's image location does not depend on it.
This is NCore's first non-central camera model. Its rays are parallel and
share no common origin, so a 3d direction no longer identifies a ray:
unprojection returns 6d
[origin, direction]rays, matching the layoutWorldRaysReturnalready uses.CameraModel.camera_ray_dimreports whichrepresentation a model uses (
3or6).Why
Bird's-eye-view raster data is naturally described by a parallel projection, and
NCore had no way to express one: FTheta, pinhole and fisheye are all central
projections. Without it, consumers hand-roll their own projection alongside
NCore's camera models, which then does not participate in the shared pose,
rolling-shutter and serialization machinery.
Orthographic is a standard projection rather than a bespoke one, e.g. OpenUSD's
GfCamera::Projection {Perspective, Orthographic}and OpenGL'sglOrtho.Design notes
The ray interface is asymmetric, which keeps the change small.
camera_rays_to_image_pointsreceives an unnormalized camera-frame point forevery model (for a central model a point and a direction coincide up to scale;
a non-central model projects the point itself). So the
world_points_to_*family and the rolling-shutter solver work unchanged, and rolling shutter
comes for free. Only unprojection needed generalizing.
Central models are provably unaffected. Per-ray origins are now carried
through the sensor pose as
R @ origin + tinstead of broadcasting the posetranslation. A central model's origins are zero, so the same expression
reproduces the previous broadcast exactly, with no branch. There is a regression
test asserting this for all three central models.
Two combinations are refused, because a parallel bundle makes them
ill-defined rather than merely unimplemented:
the parallel bundle. Rejected at construction: a ray-dimensionality guard
alone would let the pairing survive every forward projection and only surface
on unprojection.
there is no common centre to pivot about. Rectifying two non-central models is
well-defined but not implemented yet (
NotImplementedError).paraxial_pinhole_geometry()raises via the documented opt-out: an orthographiccamera's focal length is infinite, so it has no pinhole approximation and
IdealPinholeCameraModelParameters.from_source()correctly refuses it.Naming.
pixels_per_unitis the orthographic counterpart of a pinhole'sfocal_length, occupying the same place in the projection but differing indimension: a pinhole consumes the dimensionless
x/z, this consumesxdirectly. Named per-unit rather than per-meter because coordinates may be
UNITLESS(e.g. SfM reconstructions). TheIdealprefix mirrorsIdealPinholeCameraModelParametersand leaves the unqualifiedOrthographicfamily name free for a distorted telecentric sibling, avoiding a repeat of the
retired
pinhole->opencv-pinholeidentifier alias.No serialized format changed. The model registers through the existing
register_camera_model/register_camera_model_parametershooks, and thediscriminator was already written from
type().Also in this PR
camera_points_to_image_points/camera_points_to_pixels(old names kept as documented-deprecatedforwarders). See below.
ExternalDistortionModelnow guards the ray representation in its publicmethods and dispatches to
_implhooks, mirroringCameraModel. See the notebelow.
camera_ray_dimis aClassVar, since the ray representation is a property ofthe model type rather than of an instance.
all four
transform()implementations (and dropped a redundant double.astype(np.uint64)in the FTheta one).Renaming
camera_rays_to_image_pointsRaised in review. The forward method's argument was always an unnormalized
camera-frame point, never a ray. With only central models the distinction
was unobservable, because every central projection is scale-invariant:
cam_rays[:,:2] / cam_rays[:,2:3]atan2(xy_norm, z), thendelta/xy_norm * xycam_rays[:,:2] * pixels_per_unitA non-central projection has no such freedom: the point's
xandyareprecisely what is being projected, so passing a normalized direction silently
yields a different image location rather than an error. That promotes the name
from merely loose to actively misleading, so it is fixed here rather than after
release:
camera_rays_to_image_points->camera_points_to_image_pointscamera_rays_to_pixels->camera_points_to_pixelsOld names remain as forwarders carrying a
.. deprecated::directive, matchingthe docstring-only deprecation style already used in this module (there is no
runtime
warnings.warnanywhere in the repo, so none was introduced).Note the unprojection side keeps its name:
image_points_to_camera_raysgenuinely returns rays, and for a non-central model those rays are the whole
point of the 6d representation.
Note on the
ExternalDistortionModelrefactorThe refactor renames the two abstract methods an external distortion model
implements (
distort_camera_rays/undistort_camera_rays->_distort_camera_rays_impl/_undistort_camera_rays_impl), so that the publicmethods can hold the ray-representation guard.
Not flagged as a breaking change: there are no out-of-tree implementations of
the abstract base, and models deriving from a concrete class such as
BivariateWindshieldModelinherit the new hooks unchanged. Were an out-of-treesubclass of the abstract base to exist, it would fail loudly at instantiation
(
Can't instantiate abstract class ...) rather than silently.Testing
invariance, out-of-window validity, and that points behind the camera stay
valid (no frustum, unlike every other model).
unprojection/projection round trip.
static and rolling-shutter paths; plus a regression guard that the three
central models are bit-identical.
paraxial_pinhole_geometry()andfrom_source()raising, non-central rays rejected by the distortion model,and both
Rectificatorcases.transform(), and factory dispatch.is not, and the deprecated aliases agree with the renamed methods. The alias
test covers the orthographic model specifically, since it is the only one
whose result would change if an alias perturbed its argument in passing.
bazel test //... --config=no-gpupasses (36/36, Python 3.8 and 3.11),bazel build //...is clean under thetyaspect, andbazel run //:format.checkreports no violations.Docs
New "Central and Non-Central Models" and "Ideal Orthographic Camera Model"
sections in
sensor_models.rst, a "Camera Rays" section inconventions.rstdocumenting the 3d/6d representations, caveats in the external-distortion and
rectification sections, and
formats.rstnow listsideal-orthographic(plusthe previously missing
ideal-pinhole).