I think it would be good if for errors that are raised due to lack of permisions, that it's somehow understandable from the error, as these are expected cases (Something like a "Sorry, you don't have permission to do this"). I'd also argue that just the HTTP status code isn't sufficient in these cases since it's also used for other things.

I think we're talking about different cases here. I updated the cases where we're handling unexpected errors. I hope that the case you described is actually handled as expected, and we don't randomly throw exceptions.

The fix here is that we still have (conceptually) a big try catch around engine execution, in case there are any unexpected errors. And before, we were passing back the complete exception message, which may include the complete serialized record.

So two things here:

1. This may leak details
2. Passing back the raw exception is unlikely to be helpful for the user. They will not only get a 500 anyway (because an unexpected exception is a bug, most likely), but it's very likely not actionable.

Does that make sense? For missing permissions, I would expect the processors themselves to send an explicit rejection, and not have it handled in the uncaught exception loop.

Thanks, yes, that makes sense. And I do recall that for permission problems we already had another format and that didn't expose the full record, so that fits your explanation :)

Successfully created backport PR for release-8.8.0-alpha8:

• [Backport release-8.8.0-alpha8] refactor: do not print out record in error responses #37222