Connecting to an existing - running - Jupyter kernel

[matteha]

Being able to connect to an existing Jupyter kernel (either a local one, or a remote kernel - on another server - with ports forwarded) would be tremendously helpful - rather than having to spawn a new local kernel for Quarto to obtain results.

In org-mode in Emacs you can specify an existing kernel (with the emacs-jupyter package) as follows:

#+BEGIN_SRC jupyter-python :session "~/Dropbox/big.json" 
print('connected')
#+END_SRC

where "big.json" contains the ports of the running Jupyter kernel, e.g.:

{
  "shell_port": 42439,
  "iopub_port": 34519,
  "stdin_port": 42989,
  "control_port": 38072,
  "hb_port": 56865,
  "ip": "127.0.0.1",
  "key": "122341321321-jupkey",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": "bigkernel"
}

Certain Neovim plugins also have this feature (e.g. 'nvim-ipy'), and VSCode has the option to hook into existing kernels.

This would allow one to report results locally using Quarto while executing code on a running kernel on a remote server (e.g. in an HPC environment, or a secure environment you necessarily need to connect to to work with certain data - through port forwarding) and to hook into running local kernels which may already have certain data loaded into memory.

Right now Quarto obtains results only - as far as I understand - by starting a separate new Jupyter kernel locally.

Or is there a way to 'trick' Quarto into thinking a kernel has already started so it won't start one by itself? e.g. by specifying ports beforehand, e.g.with the following kernel information specified in kernel.json from "kernelspec list" (I haven't gotten this to work though):

  {  "argv": [
            "/opt/homebrew/opt/[email protected]/bin/python3.10",
            "-m",
            "ipykernel_launcher",
            "--shell=52439",
            "--stdin=52989",
            "--iopub=44519",
            "--hb=26865",
            "--control=48072",
            "--Session.key='b\"afd95f290579\"'",
            "-f",
            "{connection_file}"
          ],
          "env": {},
          "display_name": "Python 3 (ipykernel)",
          "language": "python",
          "interrupt_mode": "signal",
          "metadata": {
            "debugger": true
          }
        }

[davidhaley]

I am wondering the same thing.