V3 API Reference — Custom Column Extraction

The V3 surface is for the cases where GetFileSystemRecords_V2 is the wrong tool: huge datasets where you can't materialise every record at once, or queries where you only need a few columns out of the rich record schema and don't want to pay for the rest.

V3 lets you:

• Pick the columns you want by name (EntryName, LogicalSize, Modified, …)
• Window the read with a StartRecord / RequestedCount pair, so you can page through a million-file image at constant memory.
• Get columnar output — one contiguous array per column rather than an array of structs — which is friendlier to CSV export, Arrow / Parquet pipelines, and most analytical workloads.

Exports covered on this page: GetAvailableProperties, CreatePropertySpecsFromJSON, GetFileSystemRecordsCustom_V2, GetFileSystemRecordsCustom_V3, FreePropertySpecArray.

• When to choose V3 over V2
• The four-step workflow
• GetAvailableProperties
• CreatePropertySpecsFromJSON
• GetFileSystemRecordsCustom_V3
• GetFileSystemRecordsCustom_V2 (no windowing)
• FreePropertySpecArray
• End-to-end C# example
• Reading column data by type
• Common patterns

---

When to choose V3 over V2

Question	If yes, use…
Filesystem has more than ~500K records and you need every record?	V3 with pagination
You only need a handful of columns (e.g. EntryName, LogicalSize, Modified)?	V3 — skip the cost of materialising the rest
You're streaming records into a CSV / Parquet / database writer?	V3 — column-major output maps directly
You need every column on a small-to-medium image?	V2 (GetFileSystemRecords_V2) — convenience wins
You're prototyping or exploring an unknown image?	V2 — fewer moving parts

The V2 path materialises the entire TFileRecord array in one allocation sized roughly RecordCount × ~80 bytes + sum(filename lengths) + sum(children list bytes). For a 10 M-record image that runs into the gigabytes; on 32-bit-clean allocators it can hit the 4 GB cap and return RESULT_BUFFER_OVERFLOW. V3 windowing avoids that ceiling entirely.

---

The four-step workflow

 1. GetAvailableProperties      → JSON list of every column the DLL knows
 2. CreatePropertySpecsFromJSON → build a TPropertySpec[] from a name list
 3. GetFileSystemRecordsCustom_V3 (loop with StartRecord += page)
                                → DLL writes columnar data into each spec
 4. FreePropertySpecArray       → release the whole graph in one call

Step 1 is optional once you know the column names you want. Steps 2-4 are the per-job sequence.

For tiny / one-shot workloads where you don't need pagination, GetFileSystemRecordsCustom_V2 is V3 minus the windowing — same spec shape, same cleanup. See below.

---

GetAvailableProperties

Returns a JSON array of every property the V3 path can extract. Use this to discover available column names without hard-coding.

Pascal signature:

function GetAvailableProperties(out Buffer: PByte;
                                out BufferSize: uint32): int32; cdecl;

C# binding:

[DllImport("FEX.Core.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int GetAvailableProperties(out IntPtr buffer, out uint bufferSize);

Returns: RESULT_OK, RESULT_BUFFER_OVERFLOW.

Buffer ownership: caller frees buffer with FreeAllocatedBuffer.

JSON shape:

[
  {"name": "EntryName",   "type": "String",   "typeKind": "tkUString"},
  {"name": "LogicalSize", "type": "Int64",    "typeKind": "tkInt64"},
  {"name": "Modified",    "type": "DateTime", "typeKind": "tkFloat"},
  {"name": "Created",     "type": "DateTime", "typeKind": "tkFloat"},
  {"name": "Status",      "type": "UInt64",   "typeKind": "tkInt64"},
  ...
]

The type column maps onto PROPERTY_TYPE_* constants — see the property type table in types-and-constants.md.

This is also the starting point for the JSON you'll hand to CreatePropertySpecsFromJSON.

---

CreatePropertySpecsFromJSON

Parses a JSON spec into a TPropertySpec array suitable for V3 extraction.

Pascal signature:

function CreatePropertySpecsFromJSON(JSONString: PUTF8Char;
                                     out PropertySpecs: Pointer;
                                     out PropertyCount: int32): int32; cdecl;

C# binding:

[DllImport("FEX.Core.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
public static extern int CreatePropertySpecsFromJSON(
    [MarshalAs(UnmanagedType.LPStr)] string jsonString,
    out IntPtr propertySpecs,
    out int    propertyCount);

Parameters:

Name	Type	Dir	Purpose
jsonString	UTF-8 JSON	in	Array of property specs (see below).
propertySpecs	IntPtr	out	DLL-allocated TPropertySpec[propertyCount]. Free with FreePropertySpecArray.
propertyCount	int32	out	Number of specs parsed.

Returns: RESULT_OK, RESULT_INVALID_JSON, RESULT_OUT_OF_MEMORY.

JSON spec shape:

[
  {"name": "EntryName"},
  {"name": "LogicalSize"},
  {"name": "Modified", "fieldName": "lastModified"}
]

Each entry needs at least name (a property name from GetAvailableProperties). Optional fields:

• fieldName — output alias if you want to rename the column in your consumer (the DLL still uses name to look up the source).
• type — force a PROPERTY_TYPE_* value if you want a different representation than the auto-detected one (rarely needed).
• transform — reserved for future transformation hints. Leave empty.

Limits: propertyCount is capped at 100. Need more columns? Run multiple V3 passes against the same dataset.

---

GetFileSystemRecordsCustom_V3

The windowed extractor. Each call writes a chunk of the dataset into the spec array and reports the dataset's full size — so the caller knows when to stop paging.

Pascal signature:

function GetFileSystemRecordsCustom_V3(ImageID: int32;
                                       PropertySpecs: Pointer;
                                       PropertyCount: int32;
                                       StartRecord: int32;
                                       RequestedCount: int32;
                                       out TotalRecords: uint32): int32; cdecl;

C# binding:

[DllImport("FEX.Core.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int GetFileSystemRecordsCustom_V3(
    int     imageId,
    IntPtr  propertySpecs,
    int     propertyCount,
    int     startRecord,
    int     requestedCount,
    out uint totalRecords);

Parameters:

Name	Type	Dir	Purpose
imageId	int32	in	Image-session ID. ReadFileSystem must have been called first.
propertySpecs	IntPtr	in/out	Spec array from CreatePropertySpecsFromJSON. The DLL writes column data into each spec's FDataPtr.
propertyCount	int32	in	Length of the spec array (max 100).
startRecord	int32	in	Zero-based start offset in the dataset. Pass -1 to mean 0 (legacy convenience).
requestedCount	int32	in	Number of records to extract. Pass -1 to mean "all remaining from startRecord".
totalRecords	uint32	out	Full dataset count, independent of windowing — useful for paging UI ("page 4 of 73").

Returns: RESULT_OK, RESULT_IMAGEIDNOTFOUND (-5), RESULT_INVALID_PARAMETERS (-18) — passed when propertyCount <= 0, propertyCount > 100, or startRecord ≥ totalRecords.

Output layout:

After a successful call, each spec has been populated:

Field	Meaning
FDataPtr	Pointer to a contiguous array of FNumberValue values.
FValueSize	Element size in bytes (e.g. 8 for Int64, 8 for pointer-to-string).
FNumberValue	Number of elements written — equal to the lesser of RequestedCount and TotalRecords - StartRecord.

The columns are aligned by row: column[0].DataPtr[0] and column[1].DataPtr[0] describe the same record. Reading values out is type-specific — see Reading column data by type.

Buffer ownership: the spec array and every FDataPtr it points to are owned by the DLL. Free the whole graph at the end with FreePropertySpecArray(specs, count). Never call FreeAllocatedBuffer on a column pointer.

A second call to GetFileSystemRecordsCustom_V3 against the same spec array re-uses it: the DLL frees the previous page's columns internally before writing the new ones. Don't try to keep two pages alive simultaneously through one spec array — copy the values you need into managed types first.

---

GetFileSystemRecordsCustom_V2 (no windowing)

Same shape as V3 minus the start/requested parameters. Returns every record in one shot. Use this when you know the dataset is small enough.

Pascal signature:

function GetFileSystemRecordsCustom_V2(ImageID: int32;
                                       PropertySpecs: Pointer;
                                       PropertyCount: int32;
                                       out RecordCount: uint32): int32; cdecl;

C# binding:

[DllImport("FEX.Core.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int GetFileSystemRecordsCustom_V2(
    int     imageId,
    IntPtr  propertySpecs,
    int     propertyCount,
    out uint recordCount);

Returns: identical to V3 minus RESULT_INVALID_PARAMETERS for the windowing-specific cases.

Buffer ownership: identical to V3 — free with FreePropertySpecArray.

V3 is preferred for new code; the V2 column variant exists mainly for small workloads where the extra arguments would just be noise.

---

FreePropertySpecArray

Releases the entire spec graph: every FDataPtr, every string slot inside string columns, every children list inside ARRAYUINT32 columns, and finally the spec array itself.

Pascal signature:

function FreePropertySpecArray(PropertySpecs: Pointer;
                               PropertyCount: int32): int32; cdecl;

C# binding:

[DllImport("FEX.Core.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int FreePropertySpecArray(IntPtr propertySpecs, int propertyCount);

Returns: RESULT_OK, RESULT_ERROR (only if the underlying free raised an exception).

Pair every CreatePropertySpecsFromJSON with exactly one FreePropertySpecArray, regardless of how many V3 calls happened in between.

---

End-to-end C# example

The example below paginates a million-record dataset, extracting just three columns into CSV. The pattern scales to any column count and any page size.

using System.Globalization;
using System.Runtime.InteropServices;

const int PageSize = 100_000;

[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct PropertySpec
{
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 64)] public string PropertyName;
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 64)] public string FieldName;
    public int    PropertyID;
    public int    PropertyType;
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)] public string Transform;

    public IntPtr DataPtr;
    public int    ValueSize;
    public int    NumberValue;
}

public static void ExportFileNamesAndSizesToCsv(int imageId, string csvPath)
{
    NativeMethods.ReadFileSystem(imageId, out _);

    string specJson = """
        [
          {"name": "EntryName"},
          {"name": "LogicalSize"},
          {"name": "Modified"}
        ]
        """;

    IntPtr specs = IntPtr.Zero;
    int    count = 0;
    try
    {
        int rc = NativeMethods.CreatePropertySpecsFromJSON(specJson, out specs, out count);
        if (rc != NativeMethods.RESULT_OK)
            throw new FexCoreException("CreatePropertySpecsFromJSON", rc);

        using var writer = new StreamWriter(csvPath);
        writer.WriteLine("name,logicalSize,modifiedUtc");

        int   stride       = Marshal.SizeOf<PropertySpec>();
        int   start        = 0;
        uint  totalRecords = 0;

        while (true)
        {
            rc = NativeMethods.GetFileSystemRecordsCustom_V3(
                imageId, specs, count, start, PageSize, out totalRecords);
            if (rc != NativeMethods.RESULT_OK)
                throw new FexCoreException("GetFileSystemRecordsCustom_V3", rc);

            // Re-read each spec to get this page's column pointers
            var specArray = new PropertySpec[count];
            IntPtr cur = specs;
            for (int i = 0; i < count; i++)
            {
                specArray[i] = Marshal.PtrToStructure<PropertySpec>(cur);
                cur = IntPtr.Add(cur, stride);
            }

            int rowsThisPage = specArray[0].NumberValue;
            if (rowsThisPage <= 0) break;

            // Column 0: EntryName (string column — IntPtr per element)
            // Column 1: LogicalSize (int64)
            // Column 2: Modified (int64 FILETIME)
            for (int row = 0; row < rowsThisPage; row++)
            {
                IntPtr namePtr = Marshal.ReadIntPtr(
                    specArray[0].DataPtr, row * specArray[0].ValueSize);
                long size      = Marshal.ReadInt64(
                    specArray[1].DataPtr, row * specArray[1].ValueSize);
                long modFt     = Marshal.ReadInt64(
                    specArray[2].DataPtr, row * specArray[2].ValueSize);

                string name    = namePtr != IntPtr.Zero
                    ? Marshal.PtrToStringUTF8(namePtr) ?? ""
                    : "";
                string modIso  = modFt > 0
                    ? DateTime.FromFileTimeUtc(modFt).ToString("o", CultureInfo.InvariantCulture)
                    : "";

                writer.WriteLine($"{Csv(name)},{size},{modIso}");
            }

            start += rowsThisPage;
            if ((uint)start >= totalRecords) break;
        }
    }
    finally
    {
        if (specs != IntPtr.Zero)
            NativeMethods.FreePropertySpecArray(specs, count);
    }
}

private static string Csv(string s) =>
    s.Contains(',') || s.Contains('"') || s.Contains('\n')
        ? "\"" + s.Replace("\"", "\"\"") + "\""
        : s;

Key points to notice:

1. Re-read the spec array each iteration — the DLL updates DataPtr, ValueSize, and NumberValue in place every call.
2. Use Marshal.ReadIntPtr / ReadInt64 / ReadInt32 to dereference column pointers; Marshal.PtrToStructure is overkill for primitives.
3. One cleanup, in the outer finally. Don't try to free anything between pages.
4. Resolve every UTF-8 string before the next page or the cleanup — the column's pointer is in the DLL-owned region.

---

Reading column data by type

FValueSize tells you how many bytes per slot, but you still need to know the in-memory shape. The mapping below covers every type the DLL emits.

PROPERTY_TYPE_*	FValueSize (typical)	Slot contains	Read with
STRING (1)	8	IntPtr to UTF-8 null-terminated string	Marshal.ReadIntPtr(col, i*size) → Marshal.PtrToStringUTF8
INTEGER (2) / INT32 (11)	4	int32	Marshal.ReadInt32(col, i*size)
INT64 (3)	8	int64	Marshal.ReadInt64(col, i*size)
BOOLEAN (4)	1	byte (0 or 1)	Marshal.ReadByte(col, i*size) != 0
DATETIME (5)	8	int64 Windows FILETIME	Marshal.ReadInt64 → DateTime.FromFileTimeUtc (skip if <= 0)
GUID (6)	16	16 raw bytes	read 16 bytes → new Guid(bytes)
BYTES (7)	8	IntPtr to a byte buffer (length is format-specific)	inspect via Marshal.Copy once you know the length convention
UINT16 (8) / INT16 (9)	2	uint16 / int16	Marshal.ReadInt16
UINT32 (10)	4	uint32	Marshal.ReadInt32 then cast
UINT64 (12)	8	uint64	Marshal.ReadInt64 then cast
ARRAYUINT32 (13)	8	IntPtr to a TChildrenList (length-prefixed)	read the first int32 for count, then count × int32 for IDs

Always check FNumberValue for the live row count of this page — it can be less than RequestedCount near the end of the dataset.

---

Common patterns

CSV export, three columns, paginated

See the end-to-end example above.

Just the file count

NativeMethods.ReadFileSystem(imageId, out _);

IntPtr specs = IntPtr.Zero; int count = 0;
try
{
    NativeMethods.CreatePropertySpecsFromJSON(@"[{""name"":""EntryName""}]", out specs, out count);

    NativeMethods.GetFileSystemRecordsCustom_V3(
        imageId, specs, count,
        startRecord: 0, requestedCount: 0,    // request zero rows…
        out uint total);                       // …but the total is still reported

    Console.WriteLine($"{total:N0} records in image");
}
finally
{
    if (specs != IntPtr.Zero) NativeMethods.FreePropertySpecArray(specs, count);
}

A RequestedCount of zero is a valid query — the DLL fills in TotalRecords without populating any column. Cheap way to size up an unfamiliar image.

Filtering on the C# side

V3 doesn't push filter predicates into the DLL. Pull the columns you need, filter in your loop:

for (int row = 0; row < rowsThisPage; row++)
{
    long status = (long)Marshal.ReadInt64(specArray[3].DataPtr, row * specArray[3].ValueSize);
    if ((status & (long)StatusFlags.FILESTATUS_DELETED) == 0) continue;
    // ... handle deleted record ...
}

If you find yourself iterating millions of rows just to filter most of them out, consider whether your column set should include the filter predicate — at that point the cost is cache-friendly sequential access rather than per-record DLL calls.

Reusing one spec array across multiple images

Spec arrays are per-process and not bound to an image — the same (specs, count) pair can be passed to V3 calls against any open ImageID. Allocate once, reuse, free when you're done with the whole batch.