/kind feature
/area sdk

Our Katib Python SDK doesn't have an API to get Trial metrics from Katib DB.
Currently, user can see the Trial metrics only using Katib UI.
We should give an ability to query metrics using GetObservationLog gRPC API via Katib SDK.

From the security perspective user can run this gRPC API from any namespace and any experiment since our DB Manager doesn't have any auth checks, right ?
Should we investigate how to improve user isolation for Katib ("multi-user mode feature") ?
One solution could be to use Istio to allow traffic only from the appropriate user, as @apo-ger mentioned here: #1983 (comment).

What do you think @johnugeorge @gaocegege @tenzen-y @anencore94 @kimwnasptd @apo-ger ?

Love this feature? Give it a 👍 We prioritize the features with the most 👍