In #31464 (comment), when discussing rich previews, @javierarce suggested hiding the placeholders if there is no data returned for that placeholder from the API.

Here's the Figma for that https://www.figma.com/file/OXOmvYcX0Gpwgt7FB5LlKA/Rich-URL-metadata?node-id=0%3A1

This will need implementing as a follow up once In #31464 is merged.