Skip to content

feat: Added rest-apis serving option for registry server #5342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 13, 2025
Merged

feat: Added rest-apis serving option for registry server #5342

merged 1 commit into from
May 13, 2025

Conversation

ntkathole
Copy link
Member

@ntkathole ntkathole commented May 9, 2025 •
edited
Loading

What this PR does / why we need it:

This PR add

  • option to serve HTTP REST apis for registry server via FastAPI.
  • implements REST endpoints corresponding to gRPC methods.
  • REST endpoints mirror the gRPC interface but use intuitive HTTP methods and routes.
  • tests for registry REST endpoints.

This will allow easier integration with browser based web apps/UIs.

Misc

# feast serve_registry --rest-api

Screenshot 2025-05-09 at 12 05 51 PM

@ntkathole
feat: Added rest-apis option for registry server
bf2ebd6 
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole ntkathole requested a review from a team as a code owner May 9, 2025 07:01
@tchughesiv
Copy link
Contributor

@ntkathole when this flag is passed, does the rest API run instead of, or in addition to, the gRPC server?

@ntkathole
Copy link
Member Author

@ntkathole when this flag is passed, does the rest API run instead of, or in addition to, the gRPC server?

@tchughesiv rest api run instead of grpc.

@redhatHameed
Copy link
Contributor

@ntkathole what about Feast RBAC ? does that also dependent on internal grpc implementation ?

franciscojavierarceo
franciscojavierarceo reviewed May 10, 2025
View reviewed changes
Copy link
Member

@franciscojavierarceo franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the permissions assertions anywhere here.

I do see the get permissions but it's not clear to me that's asserting the permissions like it's done in the feature server. Am I missing something?

@ntkathole
Copy link
Member Author

ntkathole commented May 10, 2025 •
edited
Loading

@redhatHameed @franciscojavierarceo yes, RBAC authorization is same and dependent on gPRC server permissions assertion. Overall it works as :
Authentication

  • RBAC authentication is handled by the inject_user_details dependency and the auth_manager.
  • FastAPI has dependencies=[Depends(inject_user_details)] i.e. every incoming request passes through inject_user_details. It also set user context sm.set_current_user(current_user) which is globally accessible during request lifecycle using get_security_manager().current_user

Authorization

  • happens outside of gRPC or REST api server and happens within Feast itself. Here we are calling same python method which is called by gRPC handlers (grpc_handler.ListEntities()) that already enforce RBAC using the current user from the security manager., which has assert permissions logic.

That's one of the important benefit of this approach, we do not need to duplicate the logic like RBAC/response structure again for REST APIs, everything works like gRPC maintaining consistency between them.

@redhatHameed
Copy link
Contributor

@ntkathole also if you can add an example just demonstrate this approach will be helpful for community, not in this PR maybe you can add once you get chance. Thanks

@franciscojavierarceo franciscojavierarceo merged commit 9740fd1 into feast-dev:master May 13, 2025
31 checks passed
@franciscojavierarceo
Copy link
Member

@ntkathole can you make a follow up PR with the documentation?

j-wine pushed a commit to j-wine/feast that referenced this pull request Jun 7, 2025
@ntkathole @j-wine
feat: Added rest-apis serving option for registry server (feast-dev#5342
abfc589 
)

feat: Added rest-apis option for registry server

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Jacob Weinhold <29459386+j-wine@users.noreply.github.com>
franciscojavierarceo pushed a commit that referenced this pull request Jun 30, 2025
@feast-ci-bot
chore(release): release 0.50.0
3f30a1b 
# [0.50.0](v0.49.0...v0.50.0) (2025-06-30)

### Bug Fixes

* Add asyncio to integration test ([#5418](#5418)) ([6765515](6765515))
* Add clickhouse to OFFLINE_STORE_CLASS_FOR_TYPE map ([#5251](#5251)) ([9ed2ffa](9ed2ffa))
* Add missing conn.commit() in SnowflakeOnlineStore.online_write_batch ([#5432](#5432)) ([a83dd85](a83dd85))
* Add transformers in required dependencies ([8cde460](8cde460))
* Allow custom annotations on Operator installed objects ([#5339](#5339)) ([44c7a76](44c7a76))
* Dask pulling of latest data ([#5229](#5229)) ([571d81f](571d81f))
* **dask:** preserve remote URIs (e.g. s3://) in DaskOfflineStore path resolution ([2561cfc](2561cfc))
* Fix Event loop is closed error on dynamodb test ([#5480](#5480)) ([fe0f671](fe0f671))
* Fix lineage entity filtering ([#5321](#5321)) ([0d05701](0d05701))
* Fix list saved dataset api ([833696c](833696c))
* Fix NumPy - PyArrow array type mapping in Trino offline store ([#5393](#5393)) ([9ba9ded](9ba9ded))
* Fix pandas 2.x compatibility issue of Trino offline store caused by removed Series.iteritems() method ([#5345](#5345)) ([61e3e02](61e3e02))
* Fix polling mechanism for TestApplyAndMaterialize ([#5451](#5451)) ([b512a74](b512a74))
* Fix remote rbac integration tests ([#5473](#5473)) ([10879ec](10879ec))
* Fix Trino offline store SQL in Jinja template ([#5346](#5346)) ([648c53d](648c53d))
* Fixed CurlGeneratorTab github theme type ([#5425](#5425)) ([5f15329](5f15329))
* Increase the Operator Manager memory limits and requests ([#5441](#5441)) ([6c94dbf](6c94dbf))
* Method signature for push_async is out of date ([#5413](#5413)) ([28c3379](28c3379)), closes [#5410](#5410) [#006BB4](https://github.com/feast-dev/feast/issues/006BB4)
* Operator - support securityContext override at Pod level ([#5325](#5325)) ([33ea0f5](33ea0f5))
* Pybuild-deps throws errors w/ latest pip version ([#5311](#5311)) ([f2d6a67](f2d6a67))
* Reopen for integration test about add s3 storage-based registry store in Go feature server ([#5352](#5352)) ([ef75f61](ef75f61))
* resolve Python logger warnings ([#5361](#5361)) ([37d5c19](37d5c19))
* The ignore_paths not taking effect duration feast apply ([#5353](#5353)) ([e4917ca](e4917ca))
* Update generate_answer function to provide correct parameter format to retrieve function ([dc5b2af](dc5b2af))
* Update milvus connect function to work with remote instance ([#5382](#5382)) ([7e5e7d5](7e5e7d5))
* Updating milvus connect function to work with remote instance ([#5401](#5401)) ([b89fadd](b89fadd))
* Upperbound limit for protobuf generation ([#5309](#5309)) ([a114aae](a114aae))

### Features

* Add CLI, SDK, and API documentation page to Feast UI ([#5337](#5337)) ([203e888](203e888))
* Add dark mode toggle to Feast UI ([#5314](#5314)) ([ad02e46](ad02e46))
* Add data labeling tabs to UI ([#5410](#5410)) ([389ceb7](389ceb7)), closes [#006BB4](https://github.com/feast-dev/feast/issues/006BB4)
* Add Decimal to allowed python scalar types ([#5367](#5367)) ([4777c03](4777c03))
* Add feast rag retriver functionality ([#5405](#5405)) ([0173033](0173033))
* Add feature view curl generator ([#5415](#5415)) ([7a5b48f](7a5b48f))
* Add feature view lineage tab and filtering to home page lineage ([#5308](#5308)) ([308255d](308255d))
* Add feature view tags to dynamo tags ([#5291](#5291)) ([3a787ac](3a787ac))
* Add HybridOnlineStore for multi-backend online store routing ([#5423](#5423)) ([ebd67d1](ebd67d1))
* Add max_file_size to Snowflake config ([#5377](#5377)) ([e8cdf5d](e8cdf5d))
* Add MCP (Model Context Protocol) support for Feast feature server ([#5406](#5406)) ([de650de](de650de)), closes [#5398](#5398) [#5382](#5382) [#5389](#5389) [#5401](#5401)
* Add rag project to default dev UI ([#5323](#5323)) ([3b3e1c8](3b3e1c8))
* Add s3 storage-based registry store in Go feature server ([#5336](#5336)) ([abe18df](abe18df))
* Add support for data labeling in UI ([#5409](#5409)) ([d183c4b](d183c4b)), closes [#27](#27)
* Added Lineage APIs to get registry objects relationships ([#5472](#5472)) ([be004ef](be004ef))
* Added rest-apis serving option for registry server ([#5342](#5342)) ([9740fd1](9740fd1))
* Added torch.Tensor as option for online and offline retrieval ([#5381](#5381)) ([0b4ae95](0b4ae95))
* Adding feast delete to CLI ([#5344](#5344)) ([19fe3ac](19fe3ac))
* Adding permissions to UI and refactoring some things ([#5320](#5320)) ([6f1b0cc](6f1b0cc))
* Allow to set registry server rest/grpc mode in operator ([#5364](#5364)) ([99afd6d](99afd6d))
* Allow to use env variable FEAST_FS_YAML_FILE_PATH and FEATURE_REPO_DIR ([#5420](#5420)) ([6a1b33a](6a1b33a))
* Enable materialization for ODFV Transform on Write ([#5459](#5459)) ([3d17892](3d17892))
* Improve search results formatting ([#5326](#5326)) ([18cbd7f](18cbd7f))
* Improvements to Lambda materialization engine ([#5379](#5379)) ([b486f29](b486f29))
* Make batch_source optional in PushSource ([#5440](#5440)) ([#5454](#5454)) ([ae7e20e](ae7e20e))
* Refactor materialization engine ([#5354](#5354)) ([f5c5360](f5c5360))
* Remote Write to Online Store completes client / server architecture ([#5422](#5422)) ([2368f42](2368f42))
* Serialization version 2 and below removed ([#5435](#5435)) ([9e50e18](9e50e18))
* SQLite online retrieval. Add timezone info into timestamp. ([#5386](#5386)) ([6b05153](6b05153))
* Support dual-mode REST and gRPC for Feast Registry Server ([#5396](#5396)) ([fd1f448](fd1f448))
* Support DynamoDB as online store in Go feature server ([#5464](#5464)) ([40d25c6](40d25c6))
* Update Spark Compute read source node to be able to use other data sources ([#5445](#5445)) ([a93d300](a93d300))

### Reverts

* Feat: Add CLI, SDK, and API documentation page to Feast UI" ([#5341](#5341)) ([b492f14](b492f14)), closes [#5337](#5337)
* Revert "feat: Add s3 storage-based registry store in Go feature server" ([#5351](#5351)) ([d5d6766](d5d6766)), closes [#5336](#5336)
* Revert "fix: Update milvus connect function to work with remote instance" ([#5398](#5398)) ([434dd92](434dd92)), closes [#5382](#5382)
franciscojavierarceo pushed a commit that referenced this pull request Jul 1, 2025
@feast-ci-bot
chore(release): release 0.50.0
a9dfac3 
# [0.50.0](v0.49.0...v0.50.0) (2025-07-01)

### Bug Fixes

* Add asyncio to integration test ([#5418](#5418)) ([6765515](6765515))
* Add clickhouse to OFFLINE_STORE_CLASS_FOR_TYPE map ([#5251](#5251)) ([9ed2ffa](9ed2ffa))
* Add missing conn.commit() in SnowflakeOnlineStore.online_write_batch ([#5432](#5432)) ([a83dd85](a83dd85))
* Add transformers in required dependencies ([8cde460](8cde460))
* Allow custom annotations on Operator installed objects ([#5339](#5339)) ([44c7a76](44c7a76))
* Dask pulling of latest data ([#5229](#5229)) ([571d81f](571d81f))
* **dask:** preserve remote URIs (e.g. s3://) in DaskOfflineStore path resolution ([2561cfc](2561cfc))
* Fix Event loop is closed error on dynamodb test ([#5480](#5480)) ([fe0f671](fe0f671))
* Fix lineage entity filtering ([#5321](#5321)) ([0d05701](0d05701))
* Fix list saved dataset api ([833696c](833696c))
* Fix NumPy - PyArrow array type mapping in Trino offline store ([#5393](#5393)) ([9ba9ded](9ba9ded))
* Fix pandas 2.x compatibility issue of Trino offline store caused by removed Series.iteritems() method ([#5345](#5345)) ([61e3e02](61e3e02))
* Fix polling mechanism for TestApplyAndMaterialize ([#5451](#5451)) ([b512a74](b512a74))
* Fix remote rbac integration tests ([#5473](#5473)) ([10879ec](10879ec))
* Fix Trino offline store SQL in Jinja template ([#5346](#5346)) ([648c53d](648c53d))
* Fixed CurlGeneratorTab github theme type ([#5425](#5425)) ([5f15329](5f15329))
* Increase the Operator Manager memory limits and requests ([#5441](#5441)) ([6c94dbf](6c94dbf))
* Method signature for push_async is out of date ([#5413](#5413)) ([28c3379](28c3379)), closes [#5410](#5410) [#006BB4](https://github.com/feast-dev/feast/issues/006BB4)
* Operator - support securityContext override at Pod level ([#5325](#5325)) ([33ea0f5](33ea0f5))
* Pybuild-deps throws errors w/ latest pip version ([#5311](#5311)) ([f2d6a67](f2d6a67))
* Reopen for integration test about add s3 storage-based registry store in Go feature server ([#5352](#5352)) ([ef75f61](ef75f61))
* resolve Python logger warnings ([#5361](#5361)) ([37d5c19](37d5c19))
* The ignore_paths not taking effect duration feast apply ([#5353](#5353)) ([e4917ca](e4917ca))
* Update generate_answer function to provide correct parameter format to retrieve function ([dc5b2af](dc5b2af))
* Update milvus connect function to work with remote instance ([#5382](#5382)) ([7e5e7d5](7e5e7d5))
* Updating milvus connect function to work with remote instance ([#5401](#5401)) ([b89fadd](b89fadd))
* Upperbound limit for protobuf generation ([#5309](#5309)) ([a114aae](a114aae))

### Features

* Add CLI, SDK, and API documentation page to Feast UI ([#5337](#5337)) ([203e888](203e888))
* Add dark mode toggle to Feast UI ([#5314](#5314)) ([ad02e46](ad02e46))
* Add data labeling tabs to UI ([#5410](#5410)) ([389ceb7](389ceb7)), closes [#006BB4](https://github.com/feast-dev/feast/issues/006BB4)
* Add Decimal to allowed python scalar types ([#5367](#5367)) ([4777c03](4777c03))
* Add feast rag retriver functionality ([#5405](#5405)) ([0173033](0173033))
* Add feature view curl generator ([#5415](#5415)) ([7a5b48f](7a5b48f))
* Add feature view lineage tab and filtering to home page lineage ([#5308](#5308)) ([308255d](308255d))
* Add feature view tags to dynamo tags ([#5291](#5291)) ([3a787ac](3a787ac))
* Add HybridOnlineStore for multi-backend online store routing ([#5423](#5423)) ([ebd67d1](ebd67d1))
* Add max_file_size to Snowflake config ([#5377](#5377)) ([e8cdf5d](e8cdf5d))
* Add MCP (Model Context Protocol) support for Feast feature server ([#5406](#5406)) ([de650de](de650de)), closes [#5398](#5398) [#5382](#5382) [#5389](#5389) [#5401](#5401)
* Add rag project to default dev UI ([#5323](#5323)) ([3b3e1c8](3b3e1c8))
* Add s3 storage-based registry store in Go feature server ([#5336](#5336)) ([abe18df](abe18df))
* Add support for data labeling in UI ([#5409](#5409)) ([d183c4b](d183c4b)), closes [#27](#27)
* Added Lineage APIs to get registry objects relationships ([#5472](#5472)) ([be004ef](be004ef))
* Added rest-apis serving option for registry server ([#5342](#5342)) ([9740fd1](9740fd1))
* Added torch.Tensor as option for online and offline retrieval ([#5381](#5381)) ([0b4ae95](0b4ae95))
* Adding feast delete to CLI ([#5344](#5344)) ([19fe3ac](19fe3ac))
* Adding permissions to UI and refactoring some things ([#5320](#5320)) ([6f1b0cc](6f1b0cc))
* Allow to set registry server rest/grpc mode in operator ([#5364](#5364)) ([99afd6d](99afd6d))
* Allow to use env variable FEAST_FS_YAML_FILE_PATH and FEATURE_REPO_DIR ([#5420](#5420)) ([6a1b33a](6a1b33a))
* Enable materialization for ODFV Transform on Write ([#5459](#5459)) ([3d17892](3d17892))
* Improve search results formatting ([#5326](#5326)) ([18cbd7f](18cbd7f))
* Improvements to Lambda materialization engine ([#5379](#5379)) ([b486f29](b486f29))
* Make batch_source optional in PushSource ([#5440](#5440)) ([#5454](#5454)) ([ae7e20e](ae7e20e))
* Refactor materialization engine ([#5354](#5354)) ([f5c5360](f5c5360))
* Remote Write to Online Store completes client / server architecture ([#5422](#5422)) ([2368f42](2368f42))
* Serialization version 2 and below removed ([#5435](#5435)) ([9e50e18](9e50e18))
* SQLite online retrieval. Add timezone info into timestamp. ([#5386](#5386)) ([6b05153](6b05153))
* Support dual-mode REST and gRPC for Feast Registry Server ([#5396](#5396)) ([fd1f448](fd1f448))
* Support DynamoDB as online store in Go feature server ([#5464](#5464)) ([40d25c6](40d25c6))
* Update Spark Compute read source node to be able to use other data sources ([#5445](#5445)) ([a93d300](a93d300))

### Reverts

* Chore Release "chore(release): release 0.50.0" ([#5483](#5483)) ([0eef391](0eef391))
* Feat: Add CLI, SDK, and API documentation page to Feast UI" ([#5341](#5341)) ([b492f14](b492f14)), closes [#5337](#5337)
* Revert "feat: Add s3 storage-based registry store in Go feature server" ([#5351](#5351)) ([d5d6766](d5d6766)), closes [#5336](#5336)
* Revert "fix: Update milvus connect function to work with remote instance" ([#5398](#5398)) ([434dd92](434dd92)), closes [#5382](#5382)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@franciscojavierarceo franciscojavierarceo franciscojavierarceo left review comments

@redhatHameed redhatHameed redhatHameed approved these changes

Assignees
No one assigned
Labels
ok-to-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ntkathole @tchughesiv @redhatHameed @franciscojavierarceo