Search Results for

# Memory Management

`FEX.Core.dll` exposes three distinct memory ownership patterns. Mixing them up is the single biggest source of crashes in callers, so this page is the short version every integrator should read once.

- The three patterns at a glance
- Pattern 1 — DLL-allocated single buffer (V2)
- Pattern 2 — DLL-allocated columnar arrays (V3)
- Pattern 3 — Caller-allocated buffer (raw read)
- Pitfalls
- Thread safety

## The three patterns at a glance

Pattern | Used by | DLL allocates | Caller frees with | Wraps cleanly with `using`
--- | --- | --- | --- | ---
1 — Single buffer (V2) | `GetVersionAsJSON`, `GetLicenseInfoAsJSON`, `GetImageInfoByFilenameAsJSON`, `GetImageInfoByIdAsJSON`, `GetPartitionInfoAsJSON`, `GetFilePath`, `GetItemInfo`, `GetFileSystemRecords_V2`, `GetAvailableProperties`, `GetFilePropertiesFromPathAsJSON`, `GetFilePropertiesAsJSON`, `GetImageTypeAsString` | yes | `FreeAllocatedBuffer(IntPtr)` | yes
2 — Spec-array columns (V3) | `CreatePropertySpecsFromJSON`, `GetFileSystemRecordsCustom_V2`, `GetFileSystemRecordsCustom_V3` | yes | `FreePropertySpecArray(IntPtr, int)` | yes
3 — Caller-allocated raw read | `ReadFileData`, `ReadImageData`, `GetVersion` (legacy) | no | caller's own free / GC | n/a

Treat the V2 and V3 patterns as the standard. Pattern 3 only appears for raw byte streaming and exists because pre-allocating into a caller-owned buffer avoids per-chunk allocation churn.

## Pattern 1 — DLL-allocated single buffer (V2)

    out IntPtr Buffer       (the DLL writes the pointer here)
    out uint   BufferSize   (the DLL writes the byte count here)

The DLL allocates a fresh buffer for each call and writes the pointer into your `out IntPtr`. You must call `FreeAllocatedBuffer` exactly once per successful call, even if you've already copied the data out. The size is inclusive of the trailing null for string buffers.

### Canonical C# pattern

    public static string GetVersion()
    {
        IntPtr buffer = IntPtr.Zero;
        try
        {
            int rc = NativeMethods.GetVersionAsJSON(out buffer, out uint size);
            if (rc != NativeMethods.RESULT_OK)
                throw new FexCoreException("GetVersionAsJSON failed", rc);

            // Buffer is UTF-8 with a null terminator counted in size; strip it.
            return Marshal.PtrToStringUTF8(buffer, (int)size)?.TrimEnd('\0')
                ?? throw new FexCoreException("Empty buffer", NativeMethods.RESULT_ERROR);
        }
        finally
        {
            if (buffer != IntPtr.Zero)
                NativeMethods.FreeAllocatedBuffer(buffer);
        }
    }

Three rules:

1. Initialise `buffer = IntPtr.Zero` before the call.
2. Free in `finally` so an exception during marshalling can't leak.
3. Don't free a non-`RESULT_OK` buffer — but defensively guarding with `buffer != IntPtr.Zero` covers both cases and is harmless.

### When the buffer holds a struct array

`GetFileSystemRecords_V2` returns an array of `TFileRecord`. The single `Buffer` allocation owns:

- the array itself
- every `filename` UTF-8 string pointed to from the records
- every `children` list pointed to from directory records

You free the whole graph with one call to `FreeAllocatedBuffer`. Do not attempt to free individual `filename` or `children` pointers.

    IntPtr records = IntPtr.Zero;
    try
    {
        NativeMethods.ReadFileSystem(imageId, out _);
        int rc = NativeMethods.GetFileSystemRecords_V2(
            imageId, out uint count, out records, out _);
        if (rc != NativeMethods.RESULT_OK)
            throw new FexCoreException("GetFileSystemRecords_V2", rc);

        int stride = Marshal.SizeOf<FileRecordNative>();
        IntPtr cur  = records;
        for (int i = 0; i < count; i++)
        {
            var native = Marshal.PtrToStructure<FileRecordNative>(cur);
            // copy whatever you need into managed types here
            cur = IntPtr.Add(cur, stride);
        }
    }
    finally
    {
        if (records != IntPtr.Zero)
            NativeMethods.FreeAllocatedBuffer(records);
    }

Once you've called `FreeAllocatedBuffer`, every `IntPtr` you copied out of those records — including any cached `filename` pointers — is dangling. Resolve them to managed `string`s before the free.

## Pattern 2 — DLL-allocated columnar arrays (V3)

V3 returns columnar data: each `TPropertySpec` ends up holding an array of `FNumberValue` values, each `FValueSize` bytes wide, in `FDataPtr`.

The cleanup boundary is the whole spec array, not individual columns. `FreePropertySpecArray(specPtr, specCount)` walks the array, frees every `FDataPtr` (and any string slots inside string columns), and finally frees the spec array itself.

    IntPtr specs = IntPtr.Zero;
    int    count = 0;
    try
    {
        int rc = NativeMethods.CreatePropertySpecsFromJSON(
            @"[{""name"":""EntryName""},{""name"":""LogicalSize""}]",
            out specs, out count);
        if (rc != NativeMethods.RESULT_OK)
            throw new FexCoreException("CreatePropertySpecsFromJSON", rc);

        rc = NativeMethods.GetFileSystemRecordsCustom_V3(
            imageId, specs, count,
            startRecord: 0, requestedCount: 100_000,
            out uint total);
        if (rc != NativeMethods.RESULT_OK)
            throw new FexCoreException("GetFileSystemRecordsCustom_V3", rc);

        // ... read the columns out of each spec's FDataPtr ...
    }
    finally
    {
        if (specs != IntPtr.Zero)
            NativeMethods.FreePropertySpecArray(specs, count);
    }

Key consequences:

- Never call `FreeAllocatedBuffer` on a column's `FDataPtr` — that pointer is owned by the spec array, not by the V2 allocator.
- Never call `FreePropertySpecArray` on a buffer returned by a V2 function — it would try to interpret the bytes as `TPropertySpec` records.
- Each call to `GetFileSystemRecordsCustom_V3` re-populates the columns in-place, so you must call `FreePropertySpecArray` (or call the V3 function again) before discarding the spec pointer; otherwise you leak the previous page's columns.

For paginated reads, the typical loop frees once at the end:

    try
    {
        for (int page = 0; ; page++)
        {
            int got = ReadOnePage(specs, count, page * pageSize, pageSize);
            if (got == 0) break;
            // ... process this page's columns ...
        }
    }
    finally
    {
        NativeMethods.FreePropertySpecArray(specs, count);
    }

Each iteration of `ReadOnePage` reuses the same spec array; the DLL frees the previous columns internally before re-populating.

## Pattern 3 — Caller-allocated buffer (raw read)

`ReadFileData` and `ReadImageData` write into a buffer you supply:

    [DllImport(DllName, CallingConvention = CallingConvention.Cdecl)]
    public static extern int ReadFileData(
        int    imageId,
        int    fileIndex,
        long   offset,
        IntPtr buffer,
        int    bufferSize,
        out int bytesRead);

Allocation is yours; release is yours. The DLL writes up to `bufferSize` bytes to `buffer` and reports the actual count in `bytesRead`. There is no `FreeAllocatedBuffer` here.

The FexViewer `extract` command reads in 1 MB chunks:

    const int CHUNK = 1024 * 1024;
    IntPtr scratch = Marshal.AllocHGlobal(CHUNK);
    try
    {
        long offset = 0;
        while (offset < logicalSize)
        {
            int want = (int)Math.Min(CHUNK, logicalSize - offset);
            int rc = NativeMethods.ReadFileData(
                imageId, fileIndex, offset, scratch, want, out int got);
            if (rc != NativeMethods.RESULT_OK || got <= 0) break;

            var slice = new byte[got];
            Marshal.Copy(scratch, slice, 0, got);
            outputStream.Write(slice, 0, got);

            offset += got;
        }
    }
    finally
    {
        Marshal.FreeHGlobal(scratch);
    }

`AllocHGlobal` is paired with `FreeHGlobal`. `Marshal.AllocCoTaskMem` works too if you prefer that allocator. Don't use `stackalloc` for buffers larger than a few KB.

## Pitfalls

### Double-free

Calling `FreeAllocatedBuffer` twice on the same pointer is undefined behaviour. Either zero the pointer after freeing, or rely on `try/finally` flowing through exactly once.

### Mixing free functions

If the buffer came from… | Free with…
--- | ---
any V2 export with `out IntPtr Buffer` | `FreeAllocatedBuffer`
`CreatePropertySpecsFromJSON` | `FreePropertySpecArray`
`GetFileSystemRecordsCustom_V2` / `_V3` | `FreePropertySpecArray`
`Marshal.AllocHGlobal` (yours) | `Marshal.FreeHGlobal`

If you are looking at an `IntPtr` and you can't immediately answer "where did this come from", you have a bug.

### Reading after free

Every `Marshal.PtrToStructure<T>`, `Marshal.PtrToStringUTF8`, and direct pointer dereference must happen before the cleanup call. Once `FreeAllocatedBuffer` returns, every secondary pointer that was inside the freed graph is dangling. The compiler can't catch this; structure your code so reads precede frees.

### Caching pointers across calls

The `FDataPtr` columns in a V3 spec are only valid until the next call to `GetFileSystemRecordsCustom_V3` on the same spec array. Re-resolve after each page.

### Returning DLL pointers from your own functions

Don't. Convert to managed types (`string`, `byte[]`, your own DTOs) inside the function that owns the cleanup. Letting an `IntPtr` from the DLL escape into application code makes the lifetime contract impossible to follow.

### `RESULT_PASSWORD_REQUIRED` / `RESULT_PASSWORD_INCORRECT` cleanup

When `OpenImage` returns one of these, no `ImageID` is allocated — there is nothing to close. Don't call `CloseImage`. Re-attempt `OpenImage` with a different password instead.

## Thread safety

`FEX.Core.dll` is fully thread-safe: callers do not need external locks. You can issue any combination of V2 / V3 reads against the same image from multiple threads concurrently — internal locking serialises the metadata mutation paths and lets readers run in parallel.

The one rule: each allocation belongs to whichever thread frees it. If thread A obtains a V2 buffer, thread B can read from it, but the call to `FreeAllocatedBuffer` must be made exactly once total — pick one thread and stick to it. The simplest model is "the same thread that called the allocating function calls the matching free", which is what `try/finally` naturally enforces.