I've seen this when implementing Calico NetworkPolicy against the Kubernetes API (as opposed to via Calico's etcd)

Since the Pod status reporting is not synchronous with Pod set up, it often takes a long time for the API to get updated with the IP address of a newly networked Pod, which means that it can take multiple seconds before any NetworkPolicy implementation based off the k8s API learns about the new Pod's assigned IP.

I've seen this have an impact on 99%ile time to first connectivity being measured in seconds (because it takes seconds for kubelet -> apiserver -> controller to occur).

A naive fix would be to write the pod status immediately after CNI execution completes based on the result returned by the CNI plugin (though this might have performance impacts due to increased number of writes to the API).

CC @thockin @freehan @dchen1107 @wojtek-t