Skip to content

feat(api): get experiment metrics #1148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Jul 29, 2025
Merged

feat(api): get experiment metrics #1148

merged 6 commits into from
Jul 29, 2025

Conversation

kites262
Copy link
Member

@kites262 kites262 commented Jul 6, 2025 •
edited
Loading

Description

Add get_metrics API for experiment charts.

Get Experiment Metrics

res = api.get_metrics(
    exp_id="2sfvs329b9w8qclqlez2w",
    keys=["loss", "loss2", "t/accuracy"]
)

res.data (pandas.DataFrame):

          loss  loss_timestamp     loss2  loss2_timestamp  t/accuracy  t/accuracy_timestamp
step                                                                                       
0     0.482057   1751825848103  0.461134    1751825848104    0.635561         1751825848103
1     0.443716   1751825849109  0.401702    1751825849111    0.630522         1751825849107
2     0.118565   1751825850116  0.228831    1751825850118    0.809097         1751825850114
3     0.256647   1751825851124  0.156915    1751825851127    0.909703         1751825851121
4     0.087223   1751825852132  0.120786    1751825852134    0.868021         1751825852130
5     0.080971   1751825853139  0.096484    1751825853142    0.820760         1751825853137
6     0.121501   1751825854147  0.123113    1751825854150    0.925957         1751825854145
7     0.034765   1751825855155  0.191367    1751825855157    0.962653         1751825855153
  • experiment_name refers to the name of the experiment indicated by exp_id
  • key is the chart key provided by the user

@kites262 kites262 requested a review from SAKURA-CAT July 6, 2025 18:43
@kites262 kites262 self-assigned this Jul 6, 2025
@kites262 kites262 force-pushed the feature/api/metrics branch from 325c45e to 7f5e4c4 Compare July 16, 2025 15:21
@kites262 kites262 force-pushed the feature/api/metrics branch from 7f5e4c4 to 44882cb Compare July 16, 2025 15:27
@Zeyi-Lin
Copy link
Member

@kites262 API的Python交互部分是否有变化？

@kites262
Copy link
Member Author

@kites262 API的Python交互部分是否有变化？

交互和之前相同, 通过api.get_metrics(exp_id, keys)得到的是一个pd.DataFrame对象, 每个key对应其中一列

pd.DataFrame的每个数据列带有列名索引, 可以通过df['xxx']获取数据, 进而直接画图或转为np.array

@Zeyi-Lin Zeyi-Lin self-requested a review July 29, 2025 05:19
@SAKURA-CAT
Merge branch 'main' into feature/api/metrics
18fb384
@SAKURA-CAT
Make pandas an optional dependency in API modules
eda3964 
Wrapped pandas DataFrame import in try-except blocks in experiment.py and main.py. This allows the modules to function even if pandas is not installed, improving flexibility for environments where pandas is not required.
@SAKURA-CAT
Deduplicate keys in get_metrics and add test
daedbad 
Updated ExperimentAPI.get_metrics to remove duplicate keys before processing. Added a unit test to verify correct behavior when duplicate keys are provided.
@SAKURA-CAT
Simplify ImportError message for pandas dependency
947d867 
Updated the ImportError message in ExperimentAPI to be more concise when pandas is not installed. The new message clarifies the requirement for the pandas module.
@SAKURA-CAT SAKURA-CAT merged commit 77e6b59 into main Jul 29, 2025
5 checks passed
@SAKURA-CAT SAKURA-CAT deleted the feature/api/metrics branch July 29, 2025 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Zeyi-Lin Zeyi-Lin Zeyi-Lin approved these changes

@SAKURA-CAT SAKURA-CAT SAKURA-CAT approved these changes

Assignees

@kites262 kites262

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kites262 @Zeyi-Lin @SAKURA-CAT