Hi,
I see that data is sent in 64KB packets to the hdfs datanode in rpc requests. But if there is not data pending say 63KB then if it takes more than a minute for the data to arrive the rpc connection will be closed and the data will not be written. Would it be good to add a flush routine on the writer to help in such scenarios?

Thanks,
Murali