Describe Rules of Engagement

Src/IronPython.Modules/mmap.cs

@@ -1,4 +1,4 @@
-// Licensed to the .NET Foundation under one or more agreements.
+// Licensed to the .NET Foundation under one or more agreements.
 // The .NET Foundation licenses this file to you under the Apache 2.0 License.
 // See the LICENSE file in the project root for more information.
 
@@ -26,6 +26,155 @@
 using Microsoft.Scripting.Utils;
 using Microsoft.Win32.SafeHandles;
 
+/*
+MemoryMappedFile — Rules of Engagement on .NET
+==============================================
+
+In .NET, there are the following fields of `MemoryMappedFile` related to the lifetime management of
+resources.
+* `private readonly SafeMemoryMappedFileHandle _handle;` created in the constructor; necessary to
+  operate on the mmap, always disposed.
+* `private readonly bool _leaveOpen;` initialized to a constructor parameter value; it pertains to
+  `_fileHandle` not `_handle`
+* `private readonly SafeFileHandle? _fileHandle;` may be provided to the constructor, created by the
+  constructor, or null.
+
+Note that there is no field that captures `FileStream`. If a `FileStream` instance is provided to
+the factory method, it will only be used once to get its file handle, which fate is controlled by
+`_leaveOpen`. The `FileStream` instance itself is not disposed by `MemoryMappedFile`. A bit strange,
+since `FileStream` has a destructor and may be lingering around. However, when its `Dispose` is
+called from within the finalizer, it will not try to dispose the file handle, which is the whole
+point.
+
+`MemoryMappedFile` itself is `IDisposable` and its `Dispose` does:
+* dispose `_handle`, unless `_handle.IsClosed` already.
+* if not `_leaveOpen` and `_fileHandle` is not null, dispose `_fileHandle`.
+
+There are several factory/constructor groups of `MemoryMappedFile`:
+
+## Factory Method Group #1 (Windows only)
+
+Opens an existing named memory mapped file by name. In this case, only `_handle` is initialized;
+there is no underlying `_fileHandle`. It delegates opening to `OpenCore(mapName, inheritability,
+desiredAccessRights, false);` **This group functions only on Windows.**
+
+## Factory Method Group #2
+
+Creates a new memory mapped file where the content is taken from an existing file on disk.
+
+If the factory method is given a file path, it creates its own `FileStream`, stores its handle in
+`_fileHandle`, and ensures that the file handle gets closed on dispose (`_leaveOpen` is false).
+
+If the factory method is given a file handle, it is stored in `_fileHandle` and its lifetime is
+controlled by parameter `leaveOpen` given to the same method. If `leaveOpen` is true, the caller is
+responsible of disposing the file handle.
+
+If the factory method is given a `fileStream`, it is used to get the file length, flush the stream,
+and to extract the file handle into `_fileHandle`. Whether the extracted file handle is disposed
+depend on parameter `leaveOpen`. `FileStream` itself is never disposed.
+
+It delegates the opening to `CreateCore(fileHandle, mapName, HandleInheritability.None, access,
+MemoryMappedFileOptions.None, capacity, fileSize);` (see below for mode details on POSIX).
+
+**On POSIX, mapName must be null.**
+
+## Factory Method Group #3 (not POSIX)
+
+Creates a new empty memory mapped file. It only accepts a map name, and never creates/uses an
+existing file from the file system. It delegates the creation to `CreateCore(fileHandle: null,
+mapName, inheritability, access, options, capacity, -1);`
+
+**On POSIX, mapName must be null so practically this group cannot be used on POSIX.**
+
+## Factory Method Group #4 (Windows only)
+
+Creates a new empty memory mapped file or opens an existing memory mapped file if one exists with
+the same name. In this factory method/constructor, there is no file stream or file handle involved;
+If the map of the requested name exists, it is like opening from group #1, if it doesn't, it is like
+group #3 **This group functions only on Windows.**
+
+## Behaviour on POSIX
+
+Only Group #2 can be used so it means that the factory/constructor must be given one of:
+* file path (`string`)
+* file handle (`SafeFileHandle`), may be null for an anonymous empty map
+* file stream (`FileStream`)
+
+The actual work is done by `CreateCore` (POSIX-specific). If given a null file handle (from factory
+method `CreateNew`), `CreateCore` may create its own file stream if needed. This file stream/handle
+is not saved in a field `_fileHandle` of the map itself, but when the handle is passed to the
+constructor of `SafeMemoryMappedFileHandle`, it is also marked as `ownsFileStream`, so disposing the
+map will dispose the file handle. In all normal cases, i.e. `filehandle` is not null but passed by
+the factory method, the lifetime of the file handle is controlled by `_leaveOpen` of
+`MemoryMappedFile` and `SafeMemoryMappedFileHandle` is created with the argument `ownsFileStream`
+set to false. The constructor to `SafeMemoryMappedFileHandle` does `DangerousAddRef` to the given
+file stream handle (if any), so that when the original file is disposed, the handle is still valid.
+On POSIX, the mmap handle value will be set to the same value as file handle value, which is the
+file descriptor. For mmaps without the underlying file stream, the handle is set originally to
+`IntPtr.MaxValue` so that it is valid but does not collide with any existing file descriptor. When
+the mmap handle is released, it also does `DangerousRelease` of the underlying file stream handle
+(if any), plus `Dispose` of it if it owned the file stream.
+
+`SafeFileHandle` closes the underlying file descriptor on dispose and sets it to invalid. It is OK
+to close the handle several times, or even if it add-reffed and in use somewhere else. The
+descriptor will be closed ass soon as the refcount is released, and in the meantime, it will prevent
+future addrefs.
+
+MmapDefault - Rules of Engagement
+=================================
+
+`MmapDefault` is the workhorse for Python's `mmap`. It contains all the code necessary to run on all
+supported platforms. The two subclasses `MmapWindows` and `MmapPosix` only contain platform specific
+constructors, to adhere to Python API.
+
+The relevant lifetime-sensitive (disposable) fields are:
+* `MemoryMappedFile _file;` created in the constructor, may be recreated on resize
+* `MemoryMappedViewAccessor _view;` created in the constructor, may be recreated on resize
+* `FileStream _sourceStream;` the underlying file object, may be null
+* `SafeFileHandle _handle;` the handle of the underlying file object, only used on some POSIX
+  platforms, null otherwise
+
+## .NET 8.0+/POSIX
+
+When the constructor is given a file descriptor, it is duplicated, saved in `_handle` and used to
+create the memory-mapped `_file`. The duplication of the file descriptor is CPython's behaviour;
+since Python 3.13 the constructor accepts a keyword-only argument `trackfd` prevents the duplication
+but it is not implemented here. `_handle` always owns the (duplicated) file descriptor, so it has to
+be disposed appropriately, if created. `_file` is created instructed to leave the file handle open,
+so it is possible to dispose it and recreate again on `resize`.
+
+## .NET 6.0/POSIX
+
+The factory method to create a memory-mapped file from a file descriptor is not available. The
+descriptor is still duplicated and saved in `_handle` like on .NET 8.0 but it is used to create a
+`FileStream` which is then used to create the memory-mapped file. The created `FileStream` is saved
+in `_fileStream` since it is useful to perform various file operations. However, it does not own the
+file descriptor, so the rules of engagement for `_handle` from .NET 8.0 still apply. Because of
+that, it is not essential to dispose `_sourceStream` in this case, but a good practice since it will
+suppress its finalizer. Also the memory-mapped `_file` is created instructed to leave the file
+handle open to prevent the closure of the file descriptor when the memory-mapped file is re-created.
+
+## Windows, all frameworks
+
+On Windows, the file descriptor is emulated by `PythonFileManager`, the file handle is not
+duplicated and field `_handle` is always null. The associated file stream is retrieved from
+`PythonFileManager` and used to create the memory-mapped file. The file stream is saved in
+`_sourceStream`, but since it comes from somewhere else, it must not be disposed here. Therefore the
+memory-mapped file is created instructed to leave the file handle open and there is no
+`_sourceStream.Dispose` call on disposing `MmapDefault`. The `MemoryMappedFile` constructor will
+addref the actual file handle internally, so it is safe to keep using `mmap` even if the original
+file stream is closed prematurely. Of course, it is still important to dispose the `mmap` object to
+release the reference to the file handle.
+
+## Mono
+
+Mono uses genuine file descriptors, however due to bugs and limitations, it cannot use the
+.NET/POSIX mechanics. Therefore, to prevent regressions, it follows the Windows way (to the extent
+that it is feasible), but more advanced scenarios will not behave correctly.
+
+*/
+
+
 [assembly: PythonModule("mmap", typeof(IronPython.Modules.MmapModule))]
 namespace IronPython.Modules {
     public static class MmapModule {
@@ -137,7 +286,7 @@ public class MmapDefault : IWeakReferenceable {
             private readonly long _offset;
             private readonly string _mapName;
             private readonly MemoryMappedFileAccess _fileAccess;
-            private readonly SafeFileHandle _handle;  // only used on some POSIX platforms, null otherwise
+            private readonly SafeFileHandle _handle;
 
             private volatile bool _isClosed;
             private int _refCount = 1;