On Fri, 9 Sep 2022 20:39:44 +1000 David Gibson <david(a)gibson.dropbear.id.au> wrote:On Fri, Sep 09, 2022 at 11:26:58AM +0200, Stefano Brivio wrote:On Fri, 9 Sep 2022 14:27:13 +1000 David Gibson <david(a)gibson.dropbear.id.au> wrote:No, because with this change m[i] is entirely unreferenced by mm[].udp_tap_handler() currently skips outbound packets if they have a payload length of zero. This is not correct, since in a datagram protocol zero length packets still have meaning.Right, nice catch. As far as I can tell it's an issue I added with commit bb708111833e ("treewide: Packet abstraction with mandatory boundary checks").Adjust this to correctly forward the zero-length packets by using a msghdr with msg_iovlen == 0. Bugzilla: https://bugs.passt.top/show_bug.cgi?id=19 Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au> --- udp.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/udp.c b/udp.c index c4ebecc..caa852a 100644 --- a/udp.c +++ b/udp.c @@ -1075,19 +1075,19 @@ int udp_tap_handler(struct ctx *c, int af, const void *addr, uh_send = packet_get(p, i, 0, sizeof(*uh), &len); if (!uh_send) return p->count; + + mm[i].msg_hdr.msg_name = sa; + mm[i].msg_hdr.msg_namelen = sl; + count++; + if (!len) continue; m[i].iov_base = (char *)(uh_send + 1); m[i].iov_len = len;I haven't tested this yet, but: - shouldn't iov_len be set to 0 (moving also this line before)? Note that I'm not initialising m - shouldn't iov_base point to NULL to avoid noise from valgrind?Also:Here I'm relying on the fact that mm[] (unlike m[]) *is* initialized, so if we don't alter it here, msg_iov is NULL and msg_iovlen is 0.- mm[i].msg_hdr.msg_name = sa; - mm[i].msg_hdr.msg_namelen = sl; - mm[i].msg_hdr.msg_iov = m + i; mm[i].msg_hdr.msg_iovlen = 1;...I guess we should still go through those even if the size is zero, because we're appending a message. If we don't, I would expect some subsequent messages in the batch to be dropped (as many as zero sized packets we have).I was looking at removing that initialization, but I haven't gotten that working yet.Oops, I see now. So, I suppose that if you want to drop that initialisation, you might need to zero msg_hdr.controllen as well. And msg_hdr.control too: other than keeping valgrind happy, not leaking random stuff to the kernel might make this marginally more secure. That should be better than the huge memset() at the beginning, because we're already writing to msg_iovlen anyway. If you already tried that, though, I don't have any other quick idea. By the way, I had a mechanism in place, just for TCP though, to avoid reassigning those pointers and also length descriptors. I got rid of it in commit 38fbfdbcb95d ("tcp: Get rid of iov with cached MSS, drop sendmmsg(), add deferred flush") because it didn't really help with throughput. I don't see any significant "userspace" overhead on guest-to-host TCP paths with perf(1). ...maybe for UDP that's different, I haven't focused that much on UDP performance.Right. Maybe it was just me being thick, or perhaps that could use a comment: /* Zero-length packet: don't use any buffer, msg_iovlen is 0 */ if (!len) continue; -- StefanoThat is, I suppose we could just drop the continue statement on if (!len) above -- but, again, I haven't tested it.My first version actually did that, so it also works, but I think setting msg_iovlen to 0 is a bit neater.