Skip to content

feat: Make Python Virtual Environment Persistent: Add Environments to Left Panel #5577

Open
SarahAsad23 wants to merge 23 commits into
apache:mainfrom
SarahAsad23:add-pve-panel
Open

feat: Make Python Virtual Environment Persistent: Add Environments to Left Panel #5577
SarahAsad23 wants to merge 23 commits into
apache:mainfrom
SarahAsad23:add-pve-panel

Conversation

@SarahAsad23

@SarahAsad23 SarahAsad23 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

This PR introduces persistent Python Virtual Environments (PVEs) by moving them out of the Computing Unit (CU) lifecycle and storing them in the database.

Previously, PVEs were managed through Computing Units and existed only within the CU they were created in. As a result, PVEs were lost when the corresponding CU was terminated. This PR adds a new python_virtual_environments table to persist PVE configurations and introduces a dedicated dashboard interface for managing them.

Users can now create, view, update, and delete their own Python virtual environments through a new "Environments" page in the dashboard sidebar. PVE definitions are stored as user-owned resources in the database and can be managed independently of Computing Units.

Screenshot 2026-06-08 at 6 39 55 PM Screenshot 2026-06-08 at 6 40 19 PM

K8s configurations for this feature will be added in a future PR.

Any related issues, documentation, discussions?

Related discussions and issues: #5360, #5361.

How was this PR tested?

Tested manually and tests added to PveResourceSpec.

Was this PR authored or co-authored using generative AI tooling?

Co-authored using: Claude Code

@github-actions github-actions Bot added engine ddl-change Changes to the TexeraDB DDL frontend Changes related to the frontend GUI platform Non-amber Scala service paths labels Jun 9, 2026
@SarahAsad23 SarahAsad23 marked this pull request as draft June 9, 2026 00:12
@codecov-commenter

codecov-commenter commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 46.47887% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.38%. Comparing base (17607c5) to head (16769f8).

Files with missing lines Patch % Lines
...esource/pythonvirtualenvironment/PveResource.scala 0.00% 31 Missing ⚠️
...virtual-environment/virtual-environment.service.ts 0.00% 4 Missing ⚠️
...resource/pythonvirtualenvironment/PveManager.scala 93.10% 2 Missing ⚠️
...exera/service/resource/AccessControlResource.scala 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5577      +/-   ##
============================================
- Coverage     52.39%   52.38%   -0.01%     
- Complexity     2478     2480       +2     
============================================
  Files          1070     1070              
  Lines         41360    41428      +68     
  Branches       4441     4455      +14     
============================================
+ Hits          21671    21703      +32     
- Misses        18416    18456      +40     
+ Partials       1273     1269       -4     
Flag Coverage Δ *Carryforward flag
access-control-service 64.28% <0.00%> (-0.33%) ⬇️
agent-service 33.76% <ø> (ø) Carriedforward from b7b5ed0
amber 53.25% <45.00%> (+<0.01%) ⬆️
computing-unit-managing-service 1.65% <ø> (ø)
config-service 56.71% <ø> (ø)
file-service 38.21% <ø> (ø)
frontend 46.98% <60.00%> (-0.03%) ⬇️
pyamber 90.72% <ø> (ø) Carriedforward from b7b5ed0
python 90.75% <ø> (ø) Carriedforward from b7b5ed0
workflow-compiling-service 58.69% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@SarahAsad23 SarahAsad23 marked this pull request as ready for review June 9, 2026 02:17

@kunwp1 kunwp1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The feature looks good to me! I left a few comments. One main decision is the naming of this module. Do we still want to use Pve or use a different name like "Environment" or "Virtual Environment"? Need a discussion.

Comment thread sql/updates/24.sql
Comment thread sql/texera_ddl.sql Outdated
@SarahAsad23 SarahAsad23 requested a review from kunwp1 June 10, 2026 21:13

@kunwp1 kunwp1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Update the PR description to be up-to-date. I still see old table names.
  2. Add more test cases. Try to achieve the coverage report to be near 100%. In our Apache monthly meeting, we have a goal to achieve near 90% coverage and your PR is currently lowering the coverage.
  3. Update changelog.xml because you're changing the DDL.
  4. Please explicitly mention in the PR that you're only persisting the metadata of the virtual environments so that the users don't confuse their CRUD on the venv won't drive anything.

Comment on lines +139 to +147
try {
val packagesJson = mapper.writeValueAsString(payload.packages)
val veid = PveManager.savePve(sessionUser.getUid.intValue(), name, packagesJson)
Response.ok(Map("veid" -> veid).asJava).build()
} catch {
case e: Exception =>
logger.error("Failed to save PVE", e)
throw new InternalServerErrorException(s"Failed to save PVE: ${e.getMessage}")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to return 409 error for unique constraint violations instead of 500 error. Same for updatePve. Also, better to return 201 instead of 200 for success.

Comment on lines +60 to +61
def isValidPveName(name: String): Boolean =
name != null && SafePveName.pattern.matcher(name).matches()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The database column is VARCHAR(128) but isValidPveName only checks the charset. Add a length guard here (e.g. name.length <= 128).

<div
*ngIf="pves.length === 0"
class="python-env-page-empty">
No environments yet. Click <strong>Create Python Venv</strong> to create one.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says “Create Python Venv”, but the actual button (and the page title) is “Create Environment” / “Environments”. Suggest aligning the names.

nzWrapClassName="pve-modal"
nzClassName="pve-modal"
[nzVisible]="pveModalVisible"
nzTitle="Python Environments"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ddl-change Changes to the TexeraDB DDL engine frontend Changes related to the frontend GUI platform Non-amber Scala service paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants