Add public projects #2759

haydnli-shopify · 2025-06-05T15:42:04Z

Summary

Implements support for public projects to improve user onboarding experience. Users can now create public projects that are visible to all users, while maintaining backward compatibility with private projects as the default.

This PR is part of the larger self-serve membership management effort described in #2742.

Fixes #2670

Changes Made

Added is_public field to Project model (database, core model, API schemas)
Updated /api/projects/create endpoint to accept is_public parameter
Implemented access control: non-members see only public projects, members see all accessible projects
Optimized project listing with single efficient SQL query
Added comprehensive test coverage (5 new tests covering creation and listing)
Maintained backward compatibility (projects default to private)

API Usage

# Create public project
curl -X POST /api/projects/create -H "Content-Type: application/json" -d '{"project_name": "my-project", "is_public": true}'

# Create private project (default)
curl -X POST /api/projects/create -H "Content-Type: application/json" -d '{"project_name": "my-project"}'

Testing

All tests pass including new comprehensive coverage:

test_creates_public_project
test_creates_private_project_by_default
test_creates_private_project_explicitly
test_returns_public_projects_to_non_members
test_member_sees_both_public_and_private_projects

src/dstack/_internal/server/services/projects.py

src/dstack/_internal/server/models.py

r4victor

The PR currently does not work since it's missing a db migration. Please ensure you've tested the changes.

Also note that we release from master, so PRs should not introduce partially working features or should hide them behind feature flags. This PR only makes public projects visible in the list but what happens if I open the project? I assume it will error. I'm ok with adding join/leave functionality in a separate PR, but at least it should not error when opening.

r4victor · 2025-06-06T04:52:25Z

I suppose we should differentiate joined and not-joined projects in the UI, so maybe the API will return some field indicating that?

… that changing it to include public non-member projects could break existing code that assumes membership, so split into two functions

r4victor · 2025-06-11T04:42:08Z

@haydnli-shopify, now there is a problem that clicking on a public project (/api/projects/{project_name}/get) would result in 403 since the /get endpoint allows only project members. Is your plan to make public projects "non-clickable" in the UI before joining? Just want to confirm

Another option would be to show public project details before joining – it doesn't break any confidentiality since users can join and see anyway.

haydnli-shopify · 2025-06-11T13:43:04Z

@haydnli-shopify, now there is a problem that clicking on a public project (/api/projects/{project_name}/get) would result in 403 since the /get endpoint allows only project members. Is your plan to make public projects "non-clickable" in the UI before joining? Just want to confirm

Another option would be to show public project details before joining – it doesn't break any confidentiality since users can join and see anyway.

I went with your second option - the /api/projects/{project_name}/get endpoint now allows any authenticated user to view public project details before joining. Added a new ProjectMemberOrPublicAccess permission class that maintains existing member access while enabling public project discovery.
The 403 issue should be resolved. It should give a access denied here.

By the way, You could also check the PR for the second part of this issue here #2779, which handles works after we finish merging this current PR

r4victor · 2025-06-12T04:36:59Z

I recommend installing pre-commits so that linting is done automatically: https://github.com/dstackai/dstack/blob/master/contributing/DEVELOPMENT.md

r4victor · 2025-06-13T06:41:14Z

src/dstack/_internal/server/migrations/versions/35f732ee4cf5_add_projectmodel_is_public.py

+    with op.batch_alter_table("projects", schema=None) as batch_op:
+        batch_op.add_column(sa.Column("is_public", sa.Boolean(), nullable=True))
+
+    # Set is_public to False for existing projects


Please avoid WHAT comments that duplicate the code. If LLM generated, please remove.

r4victor · 2025-06-13T06:53:41Z

@haydnli-shopify, also please avoid force pushing PRs once under review – makes it unnecessary harder for others to review and contribute. If you need to pull changes from master, just do a regular merge into a PR branch.

r4victor · 2025-06-13T07:24:44Z

@haydnli-shopify Have you tested the UI? I found the public project page won't open if a user is not a member because the UI requests /api/project/{project_name}/gateways/list which is only for members. I'm strongly against making project endpoints available for non-members even for public projects since it would complicate permission system and would be harder to keep track of. So I suggest the UI changes:

Do not show project details to non-members at all.
Or show a different view that does not require calls to project-specific endpoints.

I think this PR can be merged to keep things going, so please address the issue in #2795.

haydnli-shopify changed the title ~~Add public projects (#2670)~~ Add public projects Jun 5, 2025

haydnli-shopify marked this pull request as draft June 5, 2025 15:45

colinjc reviewed Jun 5, 2025

View reviewed changes

src/dstack/_internal/server/services/projects.py Outdated Show resolved Hide resolved

haydnli-shopify marked this pull request as ready for review June 5, 2025 16:11

haydnli-shopify requested a review from colinjc June 5, 2025 16:11

dakota-shop reviewed Jun 5, 2025

View reviewed changes

src/dstack/_internal/server/services/projects.py Outdated Show resolved Hide resolved

src/dstack/_internal/server/services/projects.py Outdated Show resolved Hide resolved

dakota-shop reviewed Jun 5, 2025

View reviewed changes

src/dstack/_internal/server/services/projects.py Show resolved Hide resolved

src/dstack/_internal/server/services/projects.py Outdated Show resolved Hide resolved

peterschmidt85 requested review from r4victor and removed request for colinjc June 5, 2025 20:17

r4victor reviewed Jun 6, 2025

View reviewed changes

src/dstack/_internal/server/models.py Show resolved Hide resolved

r4victor requested changes Jun 6, 2025

View reviewed changes

haydnli-shopify mentioned this pull request Jun 9, 2025

Add batch of members endpoints to fix race conditions (#2760) #2779

Closed

haydnli-shopify force-pushed the PR1-add-is-public-property-to-projects branch 2 times, most recently from be888c4 to 84e140a Compare June 10, 2025 16:10

haydnli-shopify added 11 commits June 10, 2025 14:36

feat: Add support for public projects (dstackai#2670)

3f36a08

refactor: Extract complex SQL query into helper function

2c951f9

fix: Keep list_user_projects backward compatible - Dakota pointed out…

cf1458f

… that changing it to include public non-member projects could break existing code that assumes membership, so split into two functions

test: Add service-level tests for backward compatibility

325b0c9

refactor: Consolidate query logic with parameter approach

43228e7

Simplify project service functions for public project discovery

c55f183

Clean up inline comments in project service

64947e1

Fix code formatting for pre-commit compliance

6e4537f

Add database migration for ProjectModel.is_public

0387e52

Fix code formatting for pre-commit compliance

878759e

fix: clarify docstring for global admin behavior

366f9c0

haydnli-shopify force-pushed the PR1-add-is-public-property-to-projects branch from fdd03c6 to 366f9c0 Compare June 10, 2025 18:51

Fix public project access for non-members

761fba6

haydnli-shopify requested review from dakota-shop, colinjc and r4victor June 11, 2025 16:17

Apply pre-commit formatting fixes

ea840a6

r4victor reviewed Jun 13, 2025

View reviewed changes

Fix comments and docstrings

24306d1

r4victor approved these changes Jun 13, 2025

View reviewed changes

r4victor merged commit 2e75105 into dstackai:master Jun 13, 2025
25 checks passed

This was referenced Jun 13, 2025

Pr3 add join leave UI buttons #2795

Merged

Fix is_public allowing null #2798

Merged

haydnli-shopify added a commit to haydnli-shopify/dstack that referenced this pull request Jun 17, 2025

fix: match ProjectMemberOrPublicAccess structure with PR dstackai#2759

b935d10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add public projects #2759

Add public projects #2759

Uh oh!

haydnli-shopify commented Jun 5, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

r4victor left a comment

Uh oh!

r4victor commented Jun 6, 2025

Uh oh!

r4victor commented Jun 11, 2025

Uh oh!

haydnli-shopify commented Jun 11, 2025 •

edited

Loading

Uh oh!

r4victor commented Jun 12, 2025

Uh oh!

r4victor Jun 13, 2025

Uh oh!

r4victor commented Jun 13, 2025

Uh oh!

r4victor commented Jun 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Add public projects #2759

Add public projects #2759

Uh oh!

Conversation

haydnli-shopify commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes Made

API Usage

Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

r4victor left a comment

Choose a reason for hiding this comment

Uh oh!

r4victor commented Jun 6, 2025

Uh oh!

r4victor commented Jun 11, 2025

Uh oh!

haydnli-shopify commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

r4victor commented Jun 12, 2025

Uh oh!

r4victor Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

r4victor commented Jun 13, 2025

Uh oh!

r4victor commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

haydnli-shopify commented Jun 5, 2025 •

edited

Loading

haydnli-shopify commented Jun 11, 2025 •

edited

Loading

r4victor commented Jun 13, 2025 •

edited

Loading