Skip to content

Add 'postfixPath' to support time pattern in the middle of paths #858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Dec 2, 2022
Merged

Add 'postfixPath' to support time pattern in the middle of paths #858

merged 14 commits into from
Dec 2, 2022

Conversation

enya-yx
Copy link
Collaborator

@enya-yx enya-yx commented Nov 15, 2022 •
edited
Loading

Description

  • Support time pattern in the middle of data sources paths
  • Modify scala source codes and add related test cases
  • Expose the API to python client
  • Add functions to copy directory for both databricks and synapse
  • Add test cases with 'hourly' and 'postfix-path' for both feature join and feature generation
  • Add some documents

How was this PR tested?

  • Test cases added into pipeline
  • Conducted more test cases locally

@xiaoyongzhu xiaoyongzhu added the safe to test Tag to execute build pipeline for a PR from forked repo label Nov 16, 2022
xiaoyongzhu
xiaoyongzhu reviewed Nov 23, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 23, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 23, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 23, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 23, 2022
View reviewed changes
@xiaoyongzhu
Copy link
Member

Thanks @enya-yx for this PR, this looks good to me other than some minor comments on the duplicated code/naming conventions.

@enya-yx enya-yx mentioned this pull request Nov 25, 2022
Closed
2 tasks
xiaoyongzhu
xiaoyongzhu reviewed Nov 25, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 28, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 28, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu reviewed Nov 28, 2022
View reviewed changes
xiaoyongzhu
xiaoyongzhu approved these changes Dec 2, 2022
View reviewed changes
Copy link
Member

@xiaoyongzhu xiaoyongzhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test failures seem should to be irrelevant

@enya-yx enya-yx merged commit 2ee97ce into feathr-ai:main Dec 2, 2022
jaymo001 pushed a commit that referenced this pull request Dec 7, 2022
@enya-yx
Add 'postfixPath' to support time pattern in the middle of paths (#858)
3c48645 
* Add 'postfixPath' to support time pattern in the middle of paths
* Expose 'postfix_path' to datasource API
* Add test cases & documents
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@windoze windoze windoze approved these changes

@xiaoyongzhu xiaoyongzhu xiaoyongzhu approved these changes

@jaymo001 jaymo001 Awaiting requested review from jaymo001

@blrchen blrchen Awaiting requested review from blrchen

@edwincheung edwincheung Awaiting requested review from edwincheung

@jainr jainr Awaiting requested review from jainr

@mufajjul mufajjul Awaiting requested review from mufajjul

@Yuqing-cat Yuqing-cat Awaiting requested review from Yuqing-cat

Assignees
No one assigned
Labels
safe to test Tag to execute build pipeline for a PR from forked repo
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@enya-yx @xiaoyongzhu @windoze