Problem with kqueue/FreeBSD

[scopedog]

I'm trying to port msquic to FreeBSD and found a weird behavior.
The programs build but ListenerClose() doesn't return and blocks forever for the following reason:

• Add an event with EV_SET + kevent(), and block with kevent() and wait for the event occuring: this is a normal procedure for kqueue.
• However, platform/datapath_kqueue.c does like:
  • A thread calls kevent() and blocks, then a different thread adds an event with EV_SET + kevent().
    This doesn't basically work and I wonder if it is OK with MacOS.

I could fix this by using pipe() and investigate more but does ListenerClose() really work with MacOS?
Fundamentally, it is better to 'add an event and block' in the same thread but you will need to do something if you want to do in different threads.

See more details about what I want to say here: https://stackoverflow.com/questions/64245246/how-to-add-a-kevent-to-a-kqueuewhile-kevent-is-locked

Thanks.

[nibanks]

Are you calling ListenerClose on a QUIC callback? That's not allowed as it will create a deadlock such as what you describe.

[scopedog]

I'm using src/tools/sample/sample.c

[scopedog]

I have probably figured out what causes it.

In datapath_kqueue.c's CxPlatSocketContextUninitialize(),

EV_SET(&Event, SocketContext->SocketFd, EVFILT_USER, EV_ADD | EV_CLEAR,
NOTE_TRIGGER, 0, (void*)SocketContext);
kevent(....)

does not trigger an event in CxPlatDataPathRunEC()'s kevent() with FreeBSD.

I instead used pipe() and it seems to be working now but need to carefully check if it doesn't affect others.