Skip to content

feat(context-center): file visibility, tabular profiles on assets, files in query runner requests - #34173

Open
tomasmontielp wants to merge 10 commits into
mainfrom
feat/csv-analysis
Open

tomasmontielp wants to merge 10 commits into
mainfrom
feat/csv-analysis

Conversation

@tomasmontielp

@tomasmontielp tomasmontielp commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Schema and service groundwork for analyzing uploaded spreadsheets from AskCollate.

  • An Asset can carry a tabular profile of an uploaded CSV or Excel file: per column type, range, null share, exact distinct count and top values, read once at upload.
  • Asset queries for the chat's guardrails: a user's uploads in a period, and uploads older than a moment; documents by the asset they are a view of.
  • Object storage gains maxUploadsPerUserPerDay, the per-user daily upload cap the chat enforces.
  • A query runner request can name uploaded files (id, name, presigned URI, relation name), loaded next to a warehouse service or on their own.
  • Context files get a shareConfig, the same shape Context Memory uses. A file without one stays visible as before; a restricted one says so, and listing and search honour it.
  • Context Center's documents view lets the owner switch a file between private and workspace.

Needed by open-metadata/openmetadata-collate#6932, which merges after this one. Nothing here waits on another PR.

Fixes #34175

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

@github-actions github-actions Bot added the safe to test Add this label to run secure Github workflows on PRs label Sep 28, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

❌ UI Checkstyle Failed

❌ ESLint + Prettier + Organise Imports (src)

One or more source files have linting or formatting issues.

Affected files
  • openmetadata-ui/src/main/resources/ui/src/components/ContextCenter/DocumentsView/DocumentsView.component.tsx

Fix locally (fast - only checks files changed in this branch):

make ui-checkstyle-changed

@github-actions

Copy link
Copy Markdown
Contributor

✅ Generated Sources Auto-Updated

The generated TypeScript types (src/generated/) and dereferenced JSON
schemas (src/jsons/, public/jsons/) have been automatically updated
based on JSON schema changes in this PR.

Comment on lines 297 to 310
@Context SecurityContext securityContext,
@PathParam("id") UUID id,
@Valid jakarta.json.JsonPatch patch) {
// Sharing is edited through this endpoint, so the guard runs here too: a caller who cannot see
// a document must not be able to change who else can.
ContextFileVisibility.enforceVisibility(
getInternal(
uriInfo,
securityContext,
id,
ContextFileVisibility.guardFields(""),
Include.NON_DELETED),
securityContext);
return patchInternal(uriInfo, securityContext, id, patch);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 Bug: shareConfig changes via PATCH are never persisted

ContextFileUpdater.entitySpecificUpdate (ContextFileRepository.java:261-271) records fileType, processingStatus, extractedText, folder and a few more, but not shareConfig. When a PATCH changes only shareConfig, which is exactly what the new visibility menu sends, entityChanged and versionChanged both stay false. storeUpdate then takes the no-change branch and never writes the row. The UI shows a success toast, but the file keeps its old visibility in the DB and in search. ContextMemoryRepository's updater does record this field. The fix is to add recordChange("shareConfig", original.getShareConfig(), updated.getShareConfig(), true); to the file updater.

Record shareConfig changes so the updater persists them:

recordChange("pageCount", original.getPageCount(), updated.getPageCount());
recordChange("shareConfig", original.getShareConfig(), updated.getShareConfig(), true);
updateFolder();
  • Apply fix

Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎

Comment on lines +48 to +62
public static boolean isVisibleToUser(ContextFile file, String userName, boolean isAdmin) {
if (isAdmin) {
return true;
}
MemoryShareConfig share = file.getShareConfig();
if (share == null || share.getVisibility() == null) {
return true;
}
if (isOwnedBy(file, userName)) {
return true;
}
if (share.getVisibility() == MemoryVisibility.ENTITY) {
return true;
}
return share.getVisibility() == MemoryVisibility.SHARED

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Bug: Private locks out the uploader since Drive files have no owners

The upload endpoint builds the file with ContextFileMapper.createToEntity(createFile, user) and never sets owners, so Drive uploads are ownerless. ContextFileVisibility.isVisibleToUser only lets owners (or admins) through for Private, and the search filter works the same way. When a non-admin picks "Private — visible only to you" (DocumentsView.component.tsx:316-323), the document disappears for that user and for everyone else except admins. The same happens to any editor who restricts a file they don't own. Fix options: make the uploader the owner at upload time, add createdBy to the check, or refuse Private/Shared on a file that has no owner the caller matches.

Make the uploader the owner so Private stays visible to them:

ContextFile file = mapper.createToEntity(createFile, user);
if (nullOrEmpty(file.getOwners())) {
  file.setOwners(List.of(Entity.getEntityReferenceByName(Entity.USER, user, Include.NON_DELETED)));
}
  • Apply fix

Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎

Comment on lines +464 to +466
ContextFile file =
getInternal(uriInfo, securityContext, id, ContextFileVisibility.guardFields(""), include);
ContextFileVisibility.enforceVisibility(file, securityContext);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Security: Bulk download and move skip the new file visibility guard

This commit adds ContextFileVisibility.enforceVisibility to get, getByName, patch, the single-file download and delete. bulkDownloadFiles (around line 620) still calls getInternal(uriInfo, securityContext, id, "", include) and streams every file's bytes into the zip without checking visibility. A user who is not the owner or a named sharer can POST a Private file's id to /bulk/download and get its content, even though GET /{id}/download returns 403 for the same file. moveFile and bulkMoveFiles also only check EDIT_ALL, so a user who cannot see a private file can still move it. Bulk delete is safe because it goes through delete(). Fix: in the bulk-download loop, fetch with guardFields("") and call enforceVisibility, and add the same check before repository.moveContextFile in both move endpoints.

Enforce visibility in the bulk-download resolve loop (apply the same pattern before moveContextFile in moveFile/bulkMoveFiles):

for (UUID id : ids) {
  ContextFile file =
      getInternal(uriInfo, securityContext, id, ContextFileVisibility.guardFields(""), include);
  ContextFileVisibility.enforceVisibility(file, securityContext);
  Asset asset = resolveAsset(file);
  • Apply fix

Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎

@github-actions

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 74%
73.94% (107670/145608) 59.22% (65976/111391) 60.48% (21786/36019)

@sonarqubecloud

Copy link
Copy Markdown

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Uploaded spreadsheets in AskCollate: file profiles on assets, files in query runner requests, private chat uploads in Context Center

1 participant