Broken LaunchAgents with launchctl 2.0

[lloeki]

This:

  launchd.agents."some.agent" = {

Results in a /Library/LaunchAgents/com.some.agent.plist, as expected.

Unfortunately it gets loaded by launchtl load in a root context (either because directly doing sudo darwin-rebuild or darwin-rebuild invoking a sudo password prompt):

$ [sudo] darwin-rebuild switch
[...]
setting up launchd services...
creating service com.some.agent
Warning: Expecting a LaunchDaemons path since the command was ran as root. Got LaunchAgents instead.
`launchctl bootstrap` is a recommended alternative.
[...]

The root cause here is that launchctl 2.0 (introduced in macOS 10.10), there's a more precise domain system:

system/[service-name]
Targets the system domain or a service within the system domain. The system domain manages the root Mach bootstrap and is considered a privileged execution context. Anyone may read or query the system
domain, but root privileges are required to make modifications.

user/<uid>/[service-name]
Targets the user domain for the given UID or a service within that domain. A user domain may exist independently of a logged-in user.

login/<asid>/[service-name]
Targets a user-login domain or service within that domain. A user-login domain is created when the user logs in at the GUI and is identified by the audit session identifier associated with that login. If a user domain has an associated login domain, the print subcommand will display the ASID of that login domain.

gui/<uid>/[service-name]
Another form of the login specifier. Rather than specifying a user-login domain by its ASID, this specifier targets the domain based on which user it is associated with and is generally more convenient.

Note: GUI domains and user domains share many resources. For the purposes of the Mach bootstrap name lookups, they are "flat", so they share the same set of registered names. But they still have discrete sets of services. So when printing the user domain's contents, you may see many Mach bootstrap name registrations from services that exist in the GUI domain for that user, but you will not see the services themselves in that list.

... and a corresponding more explicit CLI has been added (notably the domain must be explicitly provided):

launchctl enable <service-target>
launchctl disable <service-target>

Enables or disables the service in the requested domain. Once a service is disabled, it cannot be loaded in the specified domain until it is once again enabled. This state persists across boots of the device. This subcommand may only target services within the system domain or user and user-login domains.

launchctl bootstrap <domain-target> [service-path, service-path2, ...]
launchctl bootout <domain-target> [service-path1, service-path2, ...] | <service-target>

Bootstraps or removes domains and services. When service arguments are present, bootstraps and correspondingly removes their definitions into the domain. Services may be specified as a series of paths or a service identifier. Paths may point to XPC service bundles, launchd.plist(5) s, or a directories containing a collection of either. If there were one or more errors while bootstrapping or removing a collection of services, the problematic paths will be printed with the errors that occurred.

If no paths or service target are specified, these commands can either bootstrap or remove a domain specified as a domain target. Some domains will implicitly bootstrap pre-defined paths as part of their creation.

launchctl kickstart [-k] [-p] <service-target>

Instructs launchd to run the specified service immediately, regardless of its configured launch conditions.
-k If the service is already running, kill the running instance before restarting the service.
-p Upon success, print the PID of the new process or the already-running process to stdout.

launchctl print <domain-target> | <service-target>

Prints information about the specified service or domain. Domain output includes various properties about the domain as well as a list of services and endpoints in the domain with state pertaining to each. Service output includes various properties of the service, including information about its origin on-disk, its current state, execution context, and last exit status.

Legacy commands like launchctl load and launchctl unload attempt to guess the user intent:

Legacy subcommands select the target domain based on whether they are executed as root or not. When executed as root, they target the system domain.

The consequence of the current code is than in a root context launchctl load apparently mistakenly makes the LaunchAgent a LaunchDaemon on the system domain:

$ launchctl print user/`id -u`/com.some.agent
Bad request.
Could not find service "com.some.agent" in domain for uid: 501
$ launchctl print gui/`id -u`/com.some.agent
Bad request.
Could not find service "com.some.agent" in domain for user gui: 501
$ launchctl print system/com.some.agent
system/com.some.agent = {
	[...]
	type = LaunchDaemon
	[...]
	domain = system
	[...]
$ sudo launchctl bootout com.some.agent

Which is, well, wrong.

As far as the current code is concerned I found these:

nix-darwin/modules/system/launchd.nix

Lines 18 to 32 in ba9b317

	launchdActivation = basedir: target: ''
	if ! diff '${cfg.build.launchd}/Library/${basedir}/${target}' '/Library/${basedir}/${target}' &> /dev/null; then
	if test -f '/Library/${basedir}/${target}'; then
	echo "reloading service $(basename ${target} .plist)" >&2
	launchctl unload '/Library/${basedir}/${target}' || true
	else
	echo "creating service $(basename ${target} .plist)" >&2
	fi
	if test -L '/Library/${basedir}/${target}'; then
	rm '/Library/${basedir}/${target}'
	fi
	cp -f '${cfg.build.launchd}/Library/${basedir}/${target}' '/Library/${basedir}/${target}'
	launchctl load -w '/Library/${basedir}/${target}'
	fi
	'';

nix-darwin/modules/system/launchd.nix

Lines 34 to 48 in ba9b317

	userLaunchdActivation = target: ''
	if ! diff ${cfg.build.launchd}/user/Library/LaunchAgents/${target} ~/Library/LaunchAgents/${target} &> /dev/null; then
	if test -f ~/Library/LaunchAgents/${target}; then
	echo "reloading user service $(basename ${target} .plist)" >&2
	launchctl unload ~/Library/LaunchAgents/${target} || true
	else
	echo "creating user service $(basename ${target} .plist)" >&2
	fi
	if test -L ~/Library/LaunchAgents/${target}; then
	rm ~/Library/LaunchAgents/${target}
	fi
	cp -f '${cfg.build.launchd}/user/Library/LaunchAgents/${target}' ~/Library/LaunchAgents/${target}
	launchctl load -w ~/Library/LaunchAgents/${target}
	fi
	'';

And these:

nix-darwin/modules/system/launchd.nix

Lines 113 to 114 in ba9b317

	echo "removing service $(basename "$f" .plist)" >&2
	launchctl unload "/Library/LaunchAgents/$f" || true

nix-darwin/modules/system/launchd.nix

Lines 126 to 127 in ba9b317

	echo "removing service $(basename "$f" .plist)" >&2
	launchctl unload "/Library/LaunchDaemons/$f" || true

nix-darwin/modules/system/launchd.nix

Lines 151 to 152 in ba9b317

	echo "removing user service $(basename "$f" .plist)" >&2
	launchctl unload ~/Library/LaunchAgents/"$f" || true

[Samasaur1]

What's your macOS version? I'm pretty sure I've had all-user LaunchAgents work for me before

[lloeki]

TBF I'm still wrapping my head around this launchd stuff, e.g I'm currently facing this kind of terribly obtuse errors:

$ launchctl bootstrap gui/`id -u` /Library/LaunchAgents/com.some.agent.plist     
Bootstrap failed: 5: Input/output error

That said, the need to provide a uid kind of hints at a few things, e.g say you have:

  launchd.agents."some.agent" = {
    command = "${pkgs.some.thing}/bin/thing start --foreground";
    path = [ pkgs.some.package ];
    serviceConfig = {
      Label = "com.some.agent";
      RunAtLoad = true;
      KeepAlive = true;
    };
  };

In multi-user (say, user1 and user2) scenarios one may want to declaratively enable for one but not the other, or both, or neither and let the user kickstart. Combined with theneed for an explicit uid this leads to users.users.<user> having to contain whether launchd.agents."some.agent" is to be enabled/disabled/bootstrapped.

[Samasaur1]

I think the general consensus is that in an ideal world nix-darwin shouldn't manage per-user LaunchAgents (launchd.user.agents), and should only manage LaunchDaemons and system-wide LaunchAgents. My other question is what the correct service target is for a system-wide LaunchAgent, because we want to load/enable it for all users, not just the activating user

[lloeki]

What's your macOS version?

Latest Sequoia.

The thing is, the usage of the legacy commands make things wildly unpredictable: I've had a config working on one machine and the exact same config failing on another. The only difference I see is that I created and set up the plist manually on the first one before removing it and making it declarative, which left some state that made it work. On the clean one it failed, and now after a bit of mucking, it also fails on the new one.

[lloeki]

My other question is what the correct service target is for a system-wide LaunchAgent

That's the thing, there doesn't seem to be one, you have to pass the user id, which leads to having to declare users in config (otherwise you can't get a reliable list?)

[Samasaur1]

do you know whether that properly enables a service for users other than the activating user?

[lloeki]

Unknown. I'll sort my particular problem out eventually, but the core issue remains anyway:

• upon setting up launchd services... step, darwin-rebuild is (if invoked via sudo) or goes (otherwise, through a prompt) root
• launchctl load then uses the context to decide whether the plist is for a LaunchAgent or a LaunchDaemon
• since it is run under root launchctl decides it's a LaunchDaemon(whereas previously it used the path/some different method? therefore the need to use bsexec/asuser)
• hell breaks loose; it might work after a reboot, or it might not.

This makes systemwide LaunchAgents via launchd.agents.* sadly unusable. I'm not even sure it really worked that well before, as I remember having to load -w/start manually after a switch, even though it worked after a reboot because there was no system vs user domain. Today it's just adding the service into the wrong domain.

[spikespaz]

I can't figure out how to remove the plist manually and stop the service, perhaps I mucked it up by removing the file before disable/bootout?