At present, the close button is just a <span>, so it can't have focus and therefore can't be navigated to with a keyboard. It should be made with a focusable element, probably a <button>. Further, that button needs a text label of some form. That could be done either with a span with class screen-reader-text or an aria-label attribute.