# Memory Management

`FEX.Core.dll` exposes three distinct memory ownership patterns. Mixing them
up is the single biggest source of crashes in callers, so this page is the
short version every integrator should read once.

- [The three patterns at a glance](#the-three-patterns-at-a-glance)
- [Pattern 1 — DLL-allocated single buffer (V2)](#pattern-1--dll-allocated-single-buffer-v2)
- [Pattern 2 — DLL-allocated columnar arrays (V3)](#pattern-2--dll-allocated-columnar-arrays-v3)
- [Pattern 3 — Caller-allocated buffer (raw read)](#pattern-3--caller-allocated-buffer-raw-read)
- [Pitfalls](#pitfalls)
- [Thread safety](#thread-safety)

---

## The three patterns at a glance

| Pattern | Used by | DLL allocates | Caller frees with | Wraps cleanly with `using` |
|---|---|:---:|---|:---:|
| **1 — Single buffer (V2)** | `GetVersionAsJSON`, `GetLicenseInfoAsJSON`, `GetImageInfoByFilenameAsJSON`, `GetImageInfoByIdAsJSON`, `GetPartitionInfoAsJSON`, `GetFilePath`, `GetItemInfo`, `GetFileSystemRecords_V2`, `GetAvailableProperties`, `GetFilePropertiesFromPathAsJSON`, `GetFilePropertiesAsJSON`, `GetImageTypeAsString` | yes | `FreeAllocatedBuffer(IntPtr)` | yes |
| **2 — Spec-array columns (V3)** | `CreatePropertySpecsFromJSON`, `GetFileSystemRecordsCustom_V2`, `GetFileSystemRecordsCustom_V3` | yes | `FreePropertySpecArray(IntPtr, int)` | yes |
| **3 — Caller-allocated raw read** | `ReadFileData`, `ReadImageData`, `GetVersion` (legacy) | no | caller's own free / GC | n/a |

Treat the V2 and V3 patterns as the standard. Pattern 3 only appears for raw
byte streaming and exists because pre-allocating into a caller-owned buffer
avoids per-chunk allocation churn.

---

## Pattern 1 — DLL-allocated single buffer (V2)

```text
out IntPtr Buffer       (the DLL writes the pointer here)
out uint   BufferSize   (the DLL writes the byte count here)
```

The DLL allocates a fresh buffer for each call and writes the pointer into
your `out IntPtr`. You must call `FreeAllocatedBuffer` exactly once per
successful call, even if you've already copied the data out. The size is
**inclusive of the trailing null** for string buffers.

### Canonical C# pattern

```csharp
public static string GetVersion()
{
    IntPtr buffer = IntPtr.Zero;
    try
    {
        int rc = NativeMethods.GetVersionAsJSON(out buffer, out uint size);
        if (rc != NativeMethods.RESULT_OK)
            throw new FexCoreException("GetVersionAsJSON failed", rc);

        // Buffer is UTF-8 with a null terminator counted in size; strip it.
        return Marshal.PtrToStringUTF8(buffer, (int)size)?.TrimEnd('\0')
            ?? throw new FexCoreException("Empty buffer", NativeMethods.RESULT_ERROR);
    }
    finally
    {
        if (buffer != IntPtr.Zero)
            NativeMethods.FreeAllocatedBuffer(buffer);
    }
}
```

Three rules:

1. Initialise `buffer = IntPtr.Zero` before the call.
2. Free in `finally` so an exception during marshalling can't leak.
3. Don't free a non-`RESULT_OK` buffer — but defensively guarding with
   `buffer != IntPtr.Zero` covers both cases and is harmless.

### When the buffer holds a struct array

`GetFileSystemRecords_V2` returns an array of `TFileRecord`. The single
`Buffer` allocation owns:

- the array itself
- every `filename` UTF-8 string pointed to from the records
- every `children` list pointed to from directory records

You free the **whole graph** with one call to `FreeAllocatedBuffer`. Do not
attempt to free individual `filename` or `children` pointers.

```csharp
IntPtr records = IntPtr.Zero;
try
{
    NativeMethods.ReadFileSystem(imageId, out _);
    int rc = NativeMethods.GetFileSystemRecords_V2(
        imageId, out uint count, out records, out _);
    if (rc != NativeMethods.RESULT_OK)
        throw new FexCoreException("GetFileSystemRecords_V2", rc);

    int stride = Marshal.SizeOf<FileRecordNative>();
    IntPtr cur  = records;
    for (int i = 0; i < count; i++)
    {
        var native = Marshal.PtrToStructure<FileRecordNative>(cur);
        // copy whatever you need into managed types here
        cur = IntPtr.Add(cur, stride);
    }
}
finally
{
    if (records != IntPtr.Zero)
        NativeMethods.FreeAllocatedBuffer(records);
}
```

Once you've called `FreeAllocatedBuffer`, every `IntPtr` you copied out of
those records — including any cached `filename` pointers — is dangling.
Resolve them to managed `string`s *before* the free.

---

## Pattern 2 — DLL-allocated columnar arrays (V3)

V3 returns columnar data: each `TPropertySpec` ends up holding an array of
`FNumberValue` values, each `FValueSize` bytes wide, in `FDataPtr`.

The cleanup boundary is the **whole spec array**, not individual columns.
`FreePropertySpecArray(specPtr, specCount)` walks the array, frees every
`FDataPtr` (and any string slots inside string columns), and finally frees
the spec array itself.

```csharp
IntPtr specs = IntPtr.Zero;
int    count = 0;
try
{
    int rc = NativeMethods.CreatePropertySpecsFromJSON(
        @"[{""name"":""EntryName""},{""name"":""LogicalSize""}]",
        out specs, out count);
    if (rc != NativeMethods.RESULT_OK)
        throw new FexCoreException("CreatePropertySpecsFromJSON", rc);

    rc = NativeMethods.GetFileSystemRecordsCustom_V3(
        imageId, specs, count,
        startRecord: 0, requestedCount: 100_000,
        out uint total);
    if (rc != NativeMethods.RESULT_OK)
        throw new FexCoreException("GetFileSystemRecordsCustom_V3", rc);

    // ... read the columns out of each spec's FDataPtr ...
}
finally
{
    if (specs != IntPtr.Zero)
        NativeMethods.FreePropertySpecArray(specs, count);
}
```

Key consequences:

- **Never** call `FreeAllocatedBuffer` on a column's `FDataPtr` — that
  pointer is owned by the spec array, not by the V2 allocator.
- **Never** call `FreePropertySpecArray` on a buffer returned by a V2
  function — it would try to interpret the bytes as `TPropertySpec`
  records.
- Each call to `GetFileSystemRecordsCustom_V3` re-populates the columns
  in-place, so you **must** call `FreePropertySpecArray` (or call the V3
  function again) before discarding the spec pointer; otherwise you leak
  the previous page's columns.

For paginated reads, the typical loop frees once at the end:

```csharp
try
{
    for (int page = 0; ; page++)
    {
        int got = ReadOnePage(specs, count, page * pageSize, pageSize);
        if (got == 0) break;
        // ... process this page's columns ...
    }
}
finally
{
    NativeMethods.FreePropertySpecArray(specs, count);
}
```

Each iteration of `ReadOnePage` reuses the same spec array; the DLL frees
the previous columns internally before re-populating.

---

## Pattern 3 — Caller-allocated buffer (raw read)

`ReadFileData` and `ReadImageData` write into a buffer you supply:

```csharp
[DllImport(DllName, CallingConvention = CallingConvention.Cdecl)]
public static extern int ReadFileData(
    int    imageId,
    int    fileIndex,
    long   offset,
    IntPtr buffer,
    int    bufferSize,
    out int bytesRead);
```

Allocation is yours; release is yours. The DLL writes up to `bufferSize`
bytes to `buffer` and reports the actual count in `bytesRead`. There is
no `FreeAllocatedBuffer` here.

The FexViewer `extract` command reads in 1 MB chunks:

```csharp
const int CHUNK = 1024 * 1024;
IntPtr scratch = Marshal.AllocHGlobal(CHUNK);
try
{
    long offset = 0;
    while (offset < logicalSize)
    {
        int want = (int)Math.Min(CHUNK, logicalSize - offset);
        int rc = NativeMethods.ReadFileData(
            imageId, fileIndex, offset, scratch, want, out int got);
        if (rc != NativeMethods.RESULT_OK || got <= 0) break;

        var slice = new byte[got];
        Marshal.Copy(scratch, slice, 0, got);
        outputStream.Write(slice, 0, got);

        offset += got;
    }
}
finally
{
    Marshal.FreeHGlobal(scratch);
}
```

`AllocHGlobal` is paired with `FreeHGlobal`. `Marshal.AllocCoTaskMem` works
too if you prefer that allocator. Don't use `stackalloc` for buffers larger
than a few KB.

---

## Pitfalls

### Double-free

Calling `FreeAllocatedBuffer` twice on the same pointer is undefined
behaviour. Either zero the pointer after freeing, or rely on `try/finally`
flowing through exactly once.

### Mixing free functions

| If the buffer came from… | Free with… |
|---|---|
| any V2 export with `out IntPtr Buffer` | `FreeAllocatedBuffer` |
| `CreatePropertySpecsFromJSON` | `FreePropertySpecArray` |
| `GetFileSystemRecordsCustom_V2` / `_V3` | `FreePropertySpecArray` |
| `Marshal.AllocHGlobal` (yours) | `Marshal.FreeHGlobal` |

If you are looking at an `IntPtr` and you can't immediately answer "where
did this come from", you have a bug.

### Reading after free

Every `Marshal.PtrToStructure<T>`, `Marshal.PtrToStringUTF8`, and direct
pointer dereference must happen **before** the cleanup call. Once
`FreeAllocatedBuffer` returns, every secondary pointer that was inside the
freed graph is dangling. The compiler can't catch this; structure your
code so reads precede frees.

### Caching pointers across calls

The `FDataPtr` columns in a V3 spec are only valid until the next call to
`GetFileSystemRecordsCustom_V3` on the same spec array. Re-resolve after
each page.

### Returning DLL pointers from your own functions

Don't. Convert to managed types (`string`, `byte[]`, your own DTOs) inside
the function that owns the cleanup. Letting an `IntPtr` from the DLL escape
into application code makes the lifetime contract impossible to follow.

### `RESULT_PASSWORD_REQUIRED` / `RESULT_PASSWORD_INCORRECT` cleanup

When `OpenImage` returns one of these, no `ImageID` is allocated — there is
nothing to close. Don't call `CloseImage`. Re-attempt `OpenImage` with a
different password instead.

---

## Thread safety

`FEX.Core.dll` is fully thread-safe: callers do not need external locks.
You can issue any combination of V2 / V3 reads against the same image from
multiple threads concurrently — internal locking serialises the metadata
mutation paths and lets readers run in parallel.

The one rule: each allocation belongs to whichever thread frees it. If
thread A obtains a V2 buffer, thread B can read from it, but the call to
`FreeAllocatedBuffer` must be made exactly once total — pick one thread
and stick to it. The simplest model is "the same thread that called the
allocating function calls the matching free", which is what `try/finally`
naturally enforces.
