The output from pandoc's latex writer is incorrect when the markdown input contains ?‘ (i.e., a question mark plus a LEFT SINGLE QUOTATION MARK, Unicode: U+2018) in turn followed by “ or ‘.

With -t latex-smart, the actual output from
?‘‘ and ?‘“ is
?’‘ and ?’“
(i.e., a RIGHT SINGLE QUOTATION MARK, Unicode: U+2019, to the right of the question mark in both cases.).

Expected: ?‘‘ and ?‘“ (i.e., a LEFT SINGLE QUOTATION MARK, Unicode: U+2018, in both cases.)

With -t latex, the actual output from
?‘‘ and ?‘“ is
?'\,` and ?'\,``,
i.e., an APOSTROPHE, Unicode: U+0027 to the right of the question mark in both cases.

Expected, first approximation only: ?`\,` and ?`\,``, i.e., a GRAVE ACCENT, Unicode: U+0060, in both cases.

Second approximation, since ?` is latex (kernel) code for an inverted question mark (as is !` for an inverted exclamation mark), which arguably cannot represent a user’s intention in this context, pandoc should at least handle it the same way it handles the ?“ combination where it outputs ?{}`` and produce

?{}`\,` and ?{}`\,``

Ultimately, this, i.e., the current pandoc approach, still remains inadequate because, unlike pdflatex and xelatex, lualatex does not recognize {} for breaking ligatures (for some background, see here).

My tests show that only something like the following reliably suppresses the unwanted inverted question mark:

?\mbox{`}\,` and ?\mbox{`}\,``

This solution, or, in case latex experts come up with an even better idea that works for all latex engines, something equivalent, should of course also be applied to the handling of the corresponding double quotation marks (LEFT DOUBLE QUOTATION MARK, Unicode: U+201C), so that ?“ should produce ?\mbox{``}, and !“ !\mbox{``} when using -t latex.