Question: Is it okay to simply close a connection in the TCP_CLOSE state upon restore?

[aegiryy]

Description

We hit the following error when restoring a TCP connection:

(00.430332)    718: inet:       Restore: family AF_INET6   type SOCK_STREAM    proto IPPROTO_TCP      port 34622 state TCP_CLOSE        src_addr ::ffff:10.4.104.194
(00.430342)    718: tcp: Restoring TCP connection
(00.430347)    718: tcp: Restoring TCP connection id 17c8 ino 84d1d
(00.430375)    718: Error (soccr/soccr.c:501): Can't bind inet socket back (0.0.0.0): Cannot assign requested address
(00.430389)    718: Error (criu/files.c:1213): Unable to open fd=2053 id=0x17c8

What we observed is that the connection is already in the TCP_CLOSE state. So, instead of restoring it, can we simply close the connections upon restore? Would it leave any side effect that is noticeable to the restored application?

diff --git a/criu/sk-inet.c b/criu/sk-inet.c
index db11cfb64..1fe52321e 100644
--- a/criu/sk-inet.c
+++ b/criu/sk-inet.c
@@ -859,6 +859,13 @@ static int open_inet_sk(struct file_desc *d, int *new_fd)
                }
 
                mutex_lock(&ii->port->reuseaddr_lock);
+               if (ie->state == TCP_CLOSE) {
+                       if (shutdown(sk, SHUT_RDWR) && errno != ENOTCONN) {
+                               pr_perror("Unable to shutdown the socket id %x ino %x", ii->ie->id, ii->ie->ino);
+                       } else {
+                               pr_info("TCP_CLOSE connection is forcefully closed");
+                       }
+               }
                if (restore_one_tcp(sk, ii)) {
                        mutex_unlock(&ii->port->reuseaddr_lock);
                        goto err;

Steps to reproduce the issue:
1.
2.
3.

Describe the results you received:

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

CRIU logs and information:

CRIU full dump/restore logs:

Output of `criu --version`:

$ criu --version
Version: 3.17
GitID: v3.17-5-gea24496ac

Output of `criu check --all`:

Warn  (criu/kerndat.c:1453): CRIU was built without libnftables support
Warn  (criu/cr-check.c:1334): Nftables based locking requires libnftables and set concatenations support
Looks good but some kernel features are missing
which, depending on your process tree, may cause
dump or restore failure.

$ uname -r
5.15.0-1047-aws

[rst0git]

@aegiryy Have you tried using the --tcp-close option?

criu/criu/sk-tcp.c

Lines 458 to 459 in 711775f

	if (opts.tcp_close) {
	if (shutdown(fd, SHUT_RDWR) && errno != ENOTCONN) {

[aegiryy]

@rst0git I am aware of this option, but my understanding is that it will close all TCP connections regardless of their states. In my scenario, I want to keep all the active TCP connections as they are all through the loopback device (127.0.0.1). For the connections in TCP_CLOSE state, we are okay to have them closed upon restore, hence the question.

[aegiryy]

Based on the comment below, it seems to be a bug when we even try to call bind(...) when restoring a TCP connection in the TCP_CLOSE state:

criu/criu/sk-inet.c

Lines 195 to 197 in 711775f

	case TCP_CLOSE:
	/* Trivial case, we just need to create a socket on restore */
	break;

criu/soccr/soccr.c

Lines 498 to 501 in 711775f

	if (bind(sk->fd, &sk->src_addr->sa, addr_size)) {
	logerr("Can't bind inet socket back");
	return -1;
	}

Can you confirm if we expect a closed connection to ever go through bind on restore?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[avagin]

Can you confirm if we expect a closed connection to ever go through bind on restore?
yes, it is expected. getsockname returns a bound address and it works for closes sockets too, so we need to restore it.

(00.430332)    718: inet:       Restore: family AF_INET6   type SOCK_STREAM    proto IPPROTO_TCP      port 34622 state TCP_CLOSE        src_addr ::ffff:10.4.104.194
(00.430342)    718: tcp: Restoring TCP connection
(00.430347)    718: tcp: Restoring TCP connection id 17c8 ino 84d1d
(00.430375)    718: Error (soccr/soccr.c:501): Can't bind inet socket back (0.0.0.0): Cannot assign requested address
(00.430389)    718: Error (criu/files.c:1213): Unable to open fd=2053 id=0x17c8

Here, we see the AF_INET6 socket, but the address 0.0.0.0 is ipv4. It may be a dual-stack socket.
And it is an ANY address that looks weird in this case too.

[avagin]

@aegiryy do you have any clue how this socket has been created?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.