Skip to content

Commit 8fad652

Browse files
aligneddevaligneddev
aligneddev
and
aligneddev
authored
013 csv ride import (#34)
* updated constitution for more modularity, etc * implementation up to US3 e2e tests * refactor large CsvRideImportService * Implement weekly CSV import enrichment * show import ride link * harden CSV import * skip empty rows * fixing bugs, example import * fix e2e tests --------- Co-authored-by: aligneddev <aligneddev@github.com>
1 parent dc47da8 commit 8fad652

File tree

74 files changed

+9863
-42
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

74 files changed

+9863
-42
lines changed

‎.specify/memory/constitution.md‎

Lines changed: 21 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,11 @@
11
# Bike Tracking Application Constitution
2-
<!-- Sync Impact Report v1.12.0
3-
Rationale: Added explicit modularity and contract-first collaboration governance so teams can deliver independently in parallel while preserving interoperability through stable interfaces and versioned contracts.
2+
<!-- Sync Impact Report v1.14.0
3+
Rationale: Strengthened Principle I from "Clean Architecture & DDD" to "Clean Architecture, DDD & Ports-and-Adapters". Added explicit requirements for: port/adapter boundaries, anti-corruption layers for third-party integrations, separation of concerns (no mixed orchestration + I/O methods), modularity (no god services), and abstraction over third-party libraries. Added references to Modern Software Engineering (Dave Farley) and Domain Modeling Made Functional (Scott Wlaschin). Added two new guardrails: mandatory port/adapter boundaries and no god services.
44
Modified Sections:
5-
- Principle IX: Added explicit Modularity, Interfaces & Contract-First Collaboration principle
6-
- Development Workflow: Added contract-first parallel delivery guidance
7-
- Definition of Done: Added interface/contract compatibility verification requirement
8-
- Testing Strategy: Added integration contract-compatibility testing expectation
9-
- Development Approval Gates: Added contract boundary freeze gate before implementation
10-
- Compliance Audit Checklist: Added modular boundary and contract compatibility checks
11-
- Guardrails: Added non-negotiable interface/contract boundary rules for cross-module integration
12-
Status: Approved — modular architecture and contract-first parallel delivery are now constitutional requirements
13-
Current Update (v1.13.0): Added Principle X — Trunk-Based Development, Continuous Integration & Delivery. Codified branching strategy (short-lived feature branches, git worktrees, PR-gated merges with validation builds), feature flag governance (max 5 active, mandatory cleanup), and PR completion policy (owner-only completion, GitHub issue linkage required).
5+
- Principle I: Expanded to include Ports and Adapters, ACLs, separation of concerns, modularity, and third-party abstraction requirements
6+
- Guardrails: Added port/adapter boundary enforcement and god-service prohibition rules
7+
Status: Approved — ports-and-adapters architecture, anti-corruption layers, and modularity are now constitutional requirements
8+
Previous Update (v1.13.0): Added Principle X — Trunk-Based Development, Continuous Integration & Delivery. Codified branching strategy (short-lived feature branches, git worktrees, PR-gated merges with validation builds), feature flag governance (max 5 active, mandatory cleanup), and PR completion policy (owner-only completion, GitHub issue linkage required).
149
Previous Update (v1.12.3): Added mandatory per-migration test coverage governance requiring each migration to include a new or updated automated test, enforced by a migration coverage policy test in CI.
1510
Previous Updates:
1611
- v1.12.2: Added mandatory spec-completion gate requiring database migrations to be applied and E2E tests to pass before a spec can be marked done.
@@ -51,11 +46,21 @@ For detailed amendment history, see [DECISIONS.md](./DECISIONS.md).
5146

5247
## Core Principles
5348

54-
### I. Clean Architecture & Domain-Driven Design
49+
### I. Clean Architecture, Domain-Driven Design & Ports-and-Adapters
5550

56-
Domain logic isolated from infrastructure concerns via layered architecture aligned with Biker Commuter aggregates: Rides, Expenses, Savings Calculations. Infrastructure dependencies (database, HTTP clients, external APIs) must be injectable and independently testable. Use domain models to express business rules explicitly; repositories and services should abstract data access. Repository pattern separates domain models from persistence details.
51+
Domain logic isolated from infrastructure concerns via **Ports and Adapters (Hexagonal Architecture)** aligned with Biker Commuter aggregates: Rides, Expenses, Savings Calculations. Infrastructure dependencies (database, HTTP clients, external APIs) must be injectable and independently testable. Use domain models to express business rules explicitly; repositories and services should abstract data access. Repository pattern separates domain models from persistence details.
5752

58-
**Rationale**: Testability without mocking infrastructure; business logic remains framework-agnostic and reusable; easier to reason about domain behavior independent of deployment environment.
53+
**Architectural Boundaries & Abstractions**:
54+
- **Ports**: Define explicit interfaces (`I*Repository`, `I*Gateway`, `I*Service`) at domain/application layer boundaries. These are the contracts between layers — stable, owned by the inner layer, and technology-agnostic.
55+
- **Adapters**: Infrastructure implementations (EF Core repositories, HTTP clients, file system access, notification channels) implement port interfaces. Adapters are replaceable without changing business logic.
56+
- **Anti-Corruption Layers (ACLs)**: When integrating with external systems or third-party libraries, wrap them behind dedicated abstractions that translate external models into domain language. No third-party types (EF entities, HTTP response models, SDK-specific types) may leak into application or domain layers. Reference: *Domain Modeling Made Functional* (Scott Wlaschin) — model boundaries as explicit translation layers.
57+
- **Separation of Concerns & Cohesion**: Each class/module has a single, well-defined responsibility. Orchestration logic (workflow coordination) is separated from business rules (validation, calculation) and from infrastructure I/O (persistence, notifications, external calls). Functions that mix orchestration, business rules, and I/O in the same method body violate this principle.
58+
- **Modularity**: Services are small, focused, and composable. Prefer multiple cohesive services over a single "god service" that handles an entire feature's workflow. Break large orchestration methods into named, testable pipeline steps.
59+
- **Abstraction over third-party libraries**: Never depend directly on concrete third-party types in application/domain code. Wrap `DbContext`, `HttpClient`, `ILogger`, SignalR hubs, and all external SDKs behind owned interfaces so they can be replaced, decorated, or tested in isolation.
60+
61+
**Reference**: *Modern Software Engineering* (Dave Farley) — manage complexity through modularity, separation of concerns, abstraction, and loose coupling. Optimize for learning and managing complexity, not for output.
62+
63+
**Rationale**: Testability without mocking infrastructure; business logic remains framework-agnostic and reusable; easier to reason about domain behavior independent of deployment environment. Port/adapter boundaries make dependencies explicit and replaceable. Anti-corruption layers protect domain integrity when external systems change. Correct cohesion and separation of concerns prevent "god classes" that are hard to test, extend, and reason about.
5964

6065
### II. Functional Programming (Pure & Impure Sandwich)
6166

@@ -555,6 +560,8 @@ Breaking these guarantees causes architectural decay and technical debt accrual:
555560
- **Expected-flow C# logic uses Result, not exceptions** — validation, not-found, conflict, and authorization business outcomes must be returned via typed Result objects (including error code/message metadata). Throwing exceptions for these expected outcomes is prohibited; exceptions are only for truly unexpected failures.
556561
- **Cross-module work is contract-first and interface-bound** — teams must integrate through explicit interfaces and versioned contracts only; direct coupling to another module's internal implementation is prohibited.
557562
- **No Entity Framework DbContext in domain layer** — domain must remain infrastructure-agnostic. If domain needs persistence logic, use repository pattern abstracting EF.
563+
- **Port/Adapter boundaries are mandatory** — application and domain layers define ports (interfaces); infrastructure provides adapters (implementations). No concrete infrastructure types (`DbContext`, `HttpClient`, third-party SDK classes) referenced directly in application orchestration or domain logic. All I/O goes through owned interfaces. A service that directly calls `dbContext.SaveChangesAsync()` alongside business logic violates this boundary — persistence must be behind a repository or unit-of-work port.
564+
- **No god services** — a single class must not own the entire workflow of a feature (parsing, validation, persistence, background processing, notifications, cancellation). Break into cohesive collaborators: one for orchestration, one for persistence, one for row processing, etc. If a class has more than ~3 injected dependencies or methods longer than ~30 lines of business logic, it likely violates separation of concerns and should be decomposed.
558565
- **Secrets management by deployment context** — **Cloud**: all secrets in Azure Key Vault; **Local**: User Secrets or environment variables. No connection strings, API keys, or OAuth secrets in appsettings.json, code, or GitHub. Pre-commit hooks enforce this. **⚠️ This repository is public on GitHub**: any committed secret is immediately and permanently exposed to the internet; treat any accidental secret commit as an immediate security incident requiring credential rotation.
559566
- **Event schema is append-only** — never mutate existing events. If schema changes needed, create new event type and version old events. Immutability is non-negotiable.
560567
- **F# domain types must marshal through EF Core value converters** — no raw EF entities exposed to C# API layer. C# records serve as API DTOs; converters handle F#-to-C# translation.

‎specs/013-csv-ride-import/checklists/requirements.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Specification Quality Checklist: CSV Ride Import
2+
3+
**Purpose**: Validate specification completeness and quality before proceeding to planning
4+
**Created**: 2026-04-08
5+
**Feature**: [spec.md](../spec.md)
6+
7+
## Content Quality
8+
9+
- [x] No implementation details (languages, frameworks, APIs)
10+
- [x] Focused on user value and business needs
11+
- [x] Written for non-technical stakeholders
12+
- [x] All mandatory sections completed
13+
14+
## Requirement Completeness
15+
16+
- [x] No [NEEDS CLARIFICATION] markers remain
17+
- [x] Requirements are testable and unambiguous
18+
- [x] Success criteria are measurable
19+
- [x] Success criteria are technology-agnostic (no implementation details)
20+
- [x] All acceptance scenarios are defined
21+
- [x] Edge cases are identified
22+
- [x] Scope is clearly bounded
23+
- [x] Dependencies and assumptions identified
24+
25+
## Feature Readiness
26+
27+
- [x] All functional requirements have clear acceptance criteria
28+
- [x] User scenarios cover primary flows
29+
- [x] Feature meets measurable outcomes defined in Success Criteria
30+
- [x] No implementation details leak into specification
31+
32+
## Notes
33+
34+
- All items passed initial validation.
35+
- The spec intentionally avoids naming specific technologies (SignalR, etc.) in requirements — implementation details will be addressed during planning.
36+
- "Real-time persistent connection" is used instead of naming a specific protocol, keeping the spec technology-agnostic.
37+
- Enrichment is scoped to cached data only (no new external calls during import), which simplifies the import pipeline and keeps processing time predictable.
38+
- The "Override All Duplicates" behavior creates new records alongside existing ones (additive), not a merge or replacement — this is documented in Assumptions.

‎specs/013-csv-ride-import/contracts/api-contracts.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# API Contracts: CSV Ride Import
2+
3+
**Feature**: 013-csv-ride-import
4+
**Date**: 2026-04-08
5+
**Base path**: `/api/imports`
6+
7+
## New Endpoints
8+
9+
### POST `/api/imports/preview`
10+
11+
Uploads and validates CSV, creates `ImportJob` + `ImportRow` records in `awaiting-confirmation` state.
12+
13+
Request: multipart form-data
14+
- `file`: CSV file (`.csv`, max 5 MB)
15+
16+
Response: `ImportPreviewResponse`
17+
18+
```csharp
19+
public sealed record ImportPreviewResponse(
20+
long JobId,
21+
string FileName,
22+
int TotalRows,
23+
int ValidRows,
24+
int InvalidRows,
25+
IReadOnlyList<ImportRowErrorView> Errors,
26+
IReadOnlyList<ImportDuplicateView> Duplicates,
27+
bool CanStartImport
28+
);
29+
30+
public sealed record ImportRowErrorView(
31+
int RowNumber,
32+
string Field,
33+
string Message
34+
);
35+
36+
public sealed record ImportDuplicateView(
37+
int RowNumber,
38+
DateOnly RideDate,
39+
decimal Miles,
40+
IReadOnlyList<ExistingRideMatchView> ExistingMatches
41+
);
42+
43+
public sealed record ExistingRideMatchView(
44+
long RideId,
45+
DateTime RideDateTimeLocal,
46+
decimal Miles,
47+
int? RideMinutes,
48+
decimal? Temperature,
49+
string? Tags,
50+
string? Notes
51+
);
52+
```
53+
54+
Errors:
55+
- `400` invalid/missing file, bad CSV schema, file too large
56+
- `401` unauthenticated
57+
58+
### POST `/api/imports/{jobId}/start`
59+
60+
Starts processing of a previewed import job.
61+
62+
```csharp
63+
public sealed record StartImportRequest(
64+
bool OverrideAllDuplicates,
65+
IReadOnlyList<DuplicateResolutionChoice> DuplicateChoices
66+
);
67+
68+
public sealed record DuplicateResolutionChoice(
69+
int RowNumber,
70+
string Resolution // "keep-existing" | "replace-with-import"
71+
);
72+
73+
public sealed record StartImportResponse(
74+
long JobId,
75+
string Status,
76+
int TotalRows
77+
);
78+
```
79+
80+
Rules:
81+
- Requires job ownership by authenticated rider
82+
- If `OverrideAllDuplicates=true`, per-row duplicate choices are optional
83+
- Rejects start when job is already processing/completed/cancelled
84+
85+
### GET `/api/imports/{jobId}/status`
86+
87+
Returns current persisted state for reconnect and polling fallback.
88+
89+
```csharp
90+
public sealed record ImportStatusResponse(
91+
long JobId,
92+
string Status,
93+
int TotalRows,
94+
int ProcessedRows,
95+
int ImportedRows,
96+
int SkippedRows,
97+
int FailedRows,
98+
int? ProgressPercent,
99+
int? EtaRoundedMinutes,
100+
ImportCompletionSummary? Summary
101+
);
102+
103+
public sealed record ImportCompletionSummary(
104+
int TotalRows,
105+
int ImportedRows,
106+
int SkippedRows,
107+
int FailedRows,
108+
int GasEnrichedRows,
109+
int WeatherEnrichedRows,
110+
bool Cancelled
111+
);
112+
```
113+
114+
### POST `/api/imports/{jobId}/cancel`
115+
116+
Requests cooperative cancellation for an in-progress import.
117+
118+
```csharp
119+
public sealed record CancelImportResponse(
120+
long JobId,
121+
string Status,
122+
string Message
123+
);
124+
```
125+
126+
Rules:
127+
- Cancellation keeps already-imported rows
128+
- Idempotent if already cancelled/completed
129+
130+
## Real-time Progress Contract
131+
132+
Progress notifications are emitted for 25% milestones.
133+
134+
```csharp
135+
public sealed record ImportProgressNotification(
136+
long JobId,
137+
int ProgressPercent, // 25, 50, 75, 100
138+
int ProcessedRows,
139+
int TotalRows,
140+
int? EtaRoundedMinutes,
141+
string Status
142+
);
143+
```
144+
145+
## Validation and behavior rules
146+
147+
- Required CSV columns: `Date`, `Miles` (case-insensitive)
148+
- Optional CSV columns: `Time`, `Temp`, `Tags`, `Notes`
149+
- Duplicate key: `Date + Miles + Temp` (Temp must also match when provided)
150+
- Enrichment rules:
151+
- Cache-first gas/weather lookup
152+
- On cache miss, perform external lookup
153+
- Retry once on external failure
154+
- If retry fails, skip enrichment field and continue row processing
155+
- External lookup throttle: max 4 calls/second
156+
157+
## Frontend TypeScript contract sketch
158+
159+
```typescript
160+
export interface ImportPreviewResponse {
161+
jobId: number;
162+
fileName: string;
163+
totalRows: number;
164+
validRows: number;
165+
invalidRows: number;
166+
errors: ImportRowErrorView[];
167+
duplicates: ImportDuplicateView[];
168+
canStartImport: boolean;
169+
}
170+
171+
export interface ImportStatusResponse {
172+
jobId: number;
173+
status: "pending" | "validating" | "awaiting-confirmation" | "processing" | "completed" | "cancelled" | "failed";
174+
totalRows: number;
175+
processedRows: number;
176+
importedRows: number;
177+
skippedRows: number;
178+
failedRows: number;
179+
progressPercent?: number;
180+
etaRoundedMinutes?: number;
181+
summary?: ImportCompletionSummary;
182+
}
183+
```
184+
185+
## Backward compatibility
186+
187+
- Existing ride create/edit/history contracts remain unchanged.
188+
- Import is additive through new endpoints and new frontend route.

0 commit comments

Comments
 (0)