-
Notifications
You must be signed in to change notification settings - Fork 290
Description
I notice that the default row of dataFrame
of launch_gradio_widget
is set to 2
. Since the gradio does not provide delete_row
option in the webui interface, the calculation of some web demos (e.g., exact_match, I don't test all) includes the blank row, which may confuse the user.
For example, if we as the users only hope to calculate the exact match score of one sample, thus we only fill the first row in the webui and left the second unremovable row blank, the returned calculation results is always wrong since it counts the second blank text row as a 1.0
score (exactly match! because they are both 0 length '' str).
The above example is shown in this figure.
Actually, I have posted a discussion in huggingface web demo nearly one month ago and no one responsed.
Here are some potential useful suggestions to fix this issue:
-
Considering the upstream package gradio is not tending to add
delete a row
option (related issue discussed in here), a quick fix is set the default row to1
instead of2
inevaluate/utils/gradio.py/launch_gradio_widget/LINE 115
. -
Check the user's input data and discard these null inputs (blank text input), which could be done in
evaluate/utils/gradio.py/parse_test_cases
function.
And I would like to have a PR for fixing this if the the community welcome.😁