Update

This Issue is now quite old and much of the original description is confusing given that half the interfaces we refer to have already shipped.

The purpose of this Issue is now solely to add an API endpoint to retrieve details about a remote URL (eg: <title> tag contents, favicon...etc). What we end up doing with this endpoint is open for discussion but one suggestion from my original Issue description below is still valid - showing details about the selected URL within the link creation UI.

TL;DR;

• I'm working on a new UI for adding a hyperlink in Gutenberg.
• It accepts URL input or free text searches for Posts and displays results as you type.
• For URL input, we need to display information about the remote URL (including site title and favicon).
• We cannot request this info using JavaScript due to cross-origin rules.
• We could do this server side in PHP via the REST API and relay info back to Gutenberg.
• Are there any known WP REST API endpoints that could help us retrieve this info from a remote site?
• How likely would it be for Gutenberg to allow us to add an endpoint such as shown in this draft PR?

What are we doing?

There is effort underway to add a new user interface (UI) for adding hyperlinks within the Block Editor. As the time of writing, it currently looks something like this:

As you can see from the above, the “search” input accepts either:

1. A free text search for a Post, Page or Category from your WordPress website.
2. A direct URL input (eg: https://make.wordpress.org).

The new component is designed to intelligently detect the type of input and respond with the most appropriate type of suggestions for the Link the user is trying to create. This is either:

• A list of search suggestions matching the search term the user has entered.
• A single “suggestion” corresponding to the URL the user has entered.

What’s the challenge?

For URL input, the original design for the component calls for a preview of the URL to be presented in the search suggestions area.

Specifically we need to display the contents of the remote website’s <title> tag:

Note: an earlier iteration of the design also displayed the website’s favicon here. We’ve omitted that from this discussion as the concepts are similar.

If we dive into the detail implied by the design mockup, we’ll see that the text Get Involved displayed in the search suggestion area, corresponds to the contents of the <title>tag from the URL https://make.wordpress.org.

What this design implies, therefore, is that we need to:

• Accept a URL from the Link UI search input.
• Make a request to retrieve the contents of the remote website resolved by that URL (eg: the HTML from make.wordpress.com).
• Parse the <title> tag from those contents.
• Display the resulting information in the suggestion area within the Link UI.
• Simple right? Well, no actually…

Why is this not simple?

You might think that achieving the above is as easy as making an AJAX request to the remote website from JavaScript.

Something like this perhaps?

fetch('https://make.wordpress.org')
    .then((response) => response.text())
    .then((html) => {
        console.log(html); // access title from DOM here
    });

But in trying this request you will violate cross-origin rules.

So does this mean we can't access our data?

Using server side requests to grab remote site data

Traditionally, to work around the limitations discussed above, developers have relied on server side code (such as PHP) to make cross origin requests on behalf of frontend code (eg: JS) and then return the requisite information to the browser.

So in PHP land (and forgive my pseudo code here) we could have a method which makes a request to a remote site.

$response = wp_remote_get( $some_url );

…and parses the response into a DOMDocument to retrieve the title:

$doc = new DOMDocument();
$doc->loadHTML( wp_remote_retrieve_body( $response ) );
// ... retrieve title tag
// ... send response back
// ...etc

Assuming this was all wired up to an AJAX request the resulting title tag information could be sent back in the response for use within the Block Editor.

So can we do this already?

Nope. The Block Editor interacts with WordPress via the REST API. I’ve searched high and low in the docs and there’s no endpoint that would allow us to do what we’ve outlined above.

That means to make this work we’re looking at adding a new custom REST API endpoint to Gutenberg.

Draft proposal - adding a new custom REST API endpoint to Gutenberg.

To get things moving forward, I've created a draft PR to show what adding a new REST API endpoint to Gutenberg might look like:

If accepted, then my intent would be to hook this up with the new Link UI to allow us to display the title of the URL entered by the user in the search results.

Where do we go from here?

The current questions I have can be summarised as follows:

• Is adding the new endpoint proposed above feasible?
• What security concerns might there be with such an approach?
• If this feature isn’t practically possible, then what fallback experience can we offer for URL inputs? (cc @shaunandrews @karmatosed ).