Skip to content

S3: Cross-account access for buckets #6333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
May 31, 2023

Conversation

viren-nadkarni
Copy link
Contributor

@viren-nadkarni viren-nadkarni commented May 19, 2023

This PR is the intial attempt to introduce cross-account support to S3.

The scope is limited to:

  • Adding cross-account access support for GetObject, PutObject, ListObjects
  • Sharing the bucket name space across accounts i.e. bucket names must now be unique globally

Currently, no IAM/ACL/permission checks are performed when cross-account access happens. This could be tackled gradually given how complex the S3 service is.

@viren-nadkarni viren-nadkarni changed the title S3: Bucket names must be unique across accounts S3: Cross-account access for buckets May 23, 2023
@viren-nadkarni viren-nadkarni force-pushed the s3-cross-accounts branch 3 times, most recently from 6723f42 to 5408413 Compare May 26, 2023 12:46
@@ -39,10 +45,20 @@ class TestS3FileHandleClosures(TestCase):
def setUp(self) -> None:
if settings.TEST_SERVER_MODE:
raise SkipTest("No point in testing ServerMode, we're not using boto3")
self.s3 = s3model.S3Backend("us-west-1", "1234")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This had to be changed because now we make references to s3_backends from with the S3Backend class

@viren-nadkarni viren-nadkarni marked this pull request as ready for review May 30, 2023 06:11
@@ -1645,6 +1647,13 @@ def create_bucket(self, bucket_name: str, region_name: str) -> FakeBucket:
def list_buckets(self) -> List[FakeBucket]:
return list(self.buckets.values())

def get_bucket_global(self, bucket_name: str) -> FakeBucket:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to just rewrite get_bucket to include this? I think that would make the cross-account logic available for all methods.

If that's not possible, or it introduces complexity somewhere else, I'd be happy to merge this as is - just trying to understand the pros/cons of this approach 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was being cautious by having separate methods.

Changed in 329af1c, everythings seems to be working fine 👍

@codecov
Copy link

codecov bot commented May 31, 2023

Codecov Report

Merging #6333 (329af1c) into master (e677bee) will decrease coverage by 0.11%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #6333      +/-   ##
==========================================
- Coverage   96.27%   96.16%   -0.11%     
==========================================
  Files         792      794       +2     
  Lines       77221    77540     +319     
==========================================
+ Hits        74341    74567     +226     
- Misses       2880     2973      +93     
Flag Coverage Δ
servertests 36.89% <73.68%> (-0.03%) ⬇️
unittests 96.10% <100.00%> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
moto/s3/models.py 96.96% <100.00%> (+0.02%) ⬆️
moto/s3/responses.py 95.66% <100.00%> (+<0.01%) ⬆️

... and 25 files with indirect coverage changes

Copy link
Collaborator

@bblommers bblommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome - thanks @viren-nadkarni!

@bblommers bblommers added this to the 4.1.11 milestone May 31, 2023
@bblommers bblommers merged commit 7bdea26 into getmoto:master May 31, 2023
@viren-nadkarni viren-nadkarni deleted the s3-cross-accounts branch May 31, 2023 10:08
@github-actions
Copy link
Contributor

This is now part of moto >= 4.1.11.dev21

except KeyError:
raise MissingBucket(bucket=bucket_name)

if bucket_name in s3_backends.bucket_accounts:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @viren-nadkarni! I realize it's been a while.. but what whas the reasoning behind this part?

I guess it made sense to me at the time, but looking at it again, I wonder why we don't just throw an AccessDenied error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @bblommers, it's there to access a bucket that belongs to a different account

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. When I looked at it earlier, an AccessDenied error felt more intuitive.

But thinking about it a little more, all our cross-account is allowed by default, so this implementation makes more sense. Thanks @viren-nadkarni

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants