Search Results for

# FEX Core SDK - Best Practices

This guide helps you make optimal choices for API version, memory management, performance, and security.

## API Version Selection

FEX Core provides three API versions. Choose based on your use case.

### Decision Matrix

Scenario | Recommended API | Reason
--- | --- | ---
New development | V2 | Safest, simplest, no buffer sizing issues
Legacy migration | V1 | Compatibility with existing V1 code
Performance-critical | V3 | Custom field extraction reduces memory
Large-scale processing | V3 | Only fetch needed fields
Simple utilities | V2 | Easy to implement correctly

### V1 API (Legacy)

When to use:

- Maintaining existing V1 code
- Migrating from older FEX Core versions
- Specific compatibility requirements

Example:

    # V1: Caller allocates buffer
    buffer = ctypes.create_string_buffer(4096)
    result = dll.DllVersionAsJSON(buffer, len(buffer))
    # Risk: Buffer might be too small

Limitations:

- Must guess buffer sizes
- Retry loop if buffer too small
- Risk of buffer overflows

### V2 API (Recommended)

When to use:

- All new development
- When simplicity matters
- When safety is priority

Example:

    # V2: DLL allocates buffer
    buffer = ctypes.c_void_p()
    size = ctypes.c_uint32()
    result = dll.GetVersionAsJSON(byref(buffer), byref(size))
    # Always works, no size guessing
    dll.FreeAllocatedBuffer(buffer)

Benefits:

- No buffer sizing guesswork
- No overflow risks
- Cleaner code

### V3 API (Performance)

When to use:

- Processing millions of files
- Only need specific metadata
- Memory is constrained

Example:

    # V3: Request only needed fields
    fields = "Name,Size,Modified"
    result = dll.GetFileSystemRecordsCustom(
        image_id, 0, fields.encode(),
        byref(buffer), byref(size)
    )
    # Returns only requested fields, not full records

Benefits:

- Reduced memory usage
- Faster for large filesystems
- Customizable output

## Memory Management Patterns

### The Golden Rule

Always free DLL-allocated buffers with `FreeAllocatedBuffer()`.

### Reading file content at scale — use the handle-based stream API

`ReadFileData` looks convenient but leaks an internal reader per `FileIndex` that lives until `CloseImage()` — it is the #1 cause of runaway memory in bulk-extraction scripts. For any workload that touches more than a handful of files, or runs on multiple threads, use `RequestFileStream` / `ReadFileStream` / `ReleaseFileStream` instead.

    handle = ctypes.c_int32(0)
    if dll.RequestFileStream(image_id, file_index, ctypes.byref(handle)) != 0:
        return
    try:
        # chunked ReadFileStream calls here
        ...
    finally:
        # Single finally block — the pool slot is held until you release.
        dll.ReleaseFileStream(image_id, handle)

`ReleaseFileStream` failing with `RESULT_INVALID_HANDLE` (−30) in a cleanup path is benign — treat it as already-released. See V2 API / Handle-Based Stream API for full signatures.

### Python: try/finally Pattern

    buffer = ctypes.c_void_p()
    size = ctypes.c_uint32()

    result = dll.GetImageInfoByIdAsJSON(image_id, byref(buffer), byref(size))

    if result == 0:
        try:
            # Use the buffer
            data = ctypes.string_at(buffer, size.value)
            process_data(data)
        finally:
            # ALWAYS free, even if exception occurs
            dll.FreeAllocatedBuffer(buffer)

### C#: using/IDisposable Pattern

    public class FexBuffer : IDisposable
    {
        private IntPtr _buffer;
        private bool _disposed;

        public FexBuffer(IntPtr buffer)
        {
            _buffer = buffer;
        }

        public void Dispose()
        {
            if (!_disposed && _buffer != IntPtr.Zero)
            {
                FexCore.FreeAllocatedBuffer(_buffer);
                _buffer = IntPtr.Zero;
                _disposed = true;
            }
        }
    }

    // Usage
    using (var buffer = new FexBuffer(resultBuffer))
    {
        // Use buffer
        string json = Marshal.PtrToStringUTF8(buffer.Ptr, size);
    }
    // Automatically freed

### C++: RAII Pattern

    class FexBuffer {
    private:
        void* buffer_;

    public:
        explicit FexBuffer(void* buffer) : buffer_(buffer) {}

        ~FexBuffer() {
            if (buffer_) {
                FreeAllocatedBuffer(buffer_);
            }
        }

        // Non-copyable
        FexBuffer(const FexBuffer&) = delete;
        FexBuffer& operator=(const FexBuffer&) = delete;

        void* get() const { return buffer_; }
    };

    // Usage
    void* rawBuffer;
    uint32_t size;

    if (GetVersionAsJSON(&rawBuffer, &size) == 0) {
        FexBuffer buffer(rawBuffer);  // RAII takes ownership
        // Use buffer.get()
    }  // Automatically freed when scope exits

### Memory Leak Detection

Signs of memory leaks:

- Increasing memory usage over time
- Application slows down during long runs
- Out-of-memory errors

Debug approach:

    import tracemalloc

    tracemalloc.start()

    # Your FEX Core operations
    for i in range(100):
        # ... operations that might leak ...
        pass

    current, peak = tracemalloc.get_traced_memory()
    print(f"Current memory: {current / 1024 / 1024:.1f} MB")
    print(f"Peak memory: {peak / 1024 / 1024:.1f} MB")

    tracemalloc.stop()

## Performance Optimization

### 1. Use V3 for Large Filesystems

When processing millions of files, request only needed fields:

    # Slow: V2 returns all fields
    result = dll.GetFileSystemRecords_V2(image_id, 0, byref(buffer), byref(size))

    # Fast: V3 returns only requested fields
    fields = "Name,Size"  # Only what you need
    result = dll.GetFileSystemRecordsCustom(
        image_id, 0, fields.encode(),
        byref(buffer), byref(size)
    )

### 2. Process Files in Batches

For large operations, process in manageable chunks:

    def process_in_batches(files, batch_size=1000):
        """Process files in batches to manage memory."""
        for i in range(0, len(files), batch_size):
            batch = files[i:i + batch_size]
            yield process_batch(batch)

            # Optional: Force garbage collection between batches
            import gc
            gc.collect()

### 3. Reuse Image Handles

Opening images is expensive. Open once, use many times:

    # Bad: Open/close for each operation
    for file_path in file_paths:
        image_id = dll.ImageOpen(image_path, key)
        # ... process one file ...
        dll.ImageClose(image_id)

    # Good: Open once, process all, close once
    image_id = dll.ImageOpen(image_path, key)
    for file_path in file_paths:
        # ... process file ...
    dll.ImageClose(image_id)

### 4. Cache Frequently Used Data

    class ImageCache:
        def __init__(self, image_id):
            self.image_id = image_id
            self._file_list = None
            self._image_info = None

        @property
        def files(self):
            if self._file_list is None:
                self._file_list = list_files(self.image_id)
            return self._file_list

        @property
        def info(self):
            if self._image_info is None:
                self._image_info = get_image_info(self.image_id)
            return self._image_info

## Security Considerations

### Input Validation

Always validate paths before passing to the DLL:

    import os

    def safe_open_image(image_path, license_key):
        """Open image with path validation."""

        # Normalize and validate path
        normalized = os.path.normpath(image_path)

        # Check for path traversal attempts
        if '..' in normalized:
            raise ValueError("Path traversal not allowed")

        # Verify file exists
        if not os.path.isfile(normalized):
            raise FileNotFoundError(f"Image not found: {normalized}")

        # Check file extension
        valid_extensions = {'.dd', '.e01', '.l01', '.aff4', '.raw', '.img'}
        ext = os.path.splitext(normalized)[1].lower()
        if ext not in valid_extensions:
            raise ValueError(f"Unsupported extension: {ext}")

        return dll.ImageOpen(normalized.encode(), license_key.encode())

### Evidence Integrity

FEX Core is read-only by design. Maintain integrity by:

1. Never modify source images
2. Work on copies when possible
3. Log all operations for audit trail
4. Hash images before and after processing

    import hashlib

    def hash_file(path):
        """Calculate SHA-256 hash of file."""
        sha256 = hashlib.sha256()
        with open(path, 'rb') as f:
            for chunk in iter(lambda: f.read(8192), b''):
                sha256.update(chunk)
        return sha256.hexdigest()

    # Verify integrity
    original_hash = hash_file(image_path)
    # ... do processing ...
    final_hash = hash_file(image_path)

    assert original_hash == final_hash, "Image was modified!"

### License Key Handling

Never log or display license keys:

    # BAD - exposes key in logs
    print(f"Opening with key: {license_key}")
    logging.info(f"License: {license_key}")

    # GOOD - mask the key
    def mask_key(key):
        if len(key) > 8:
            return key[:4] + '*' * (len(key) - 8) + key[-4:]
        return '*' * len(key)

    print(f"Opening with key: {mask_key(license_key)}")

Store keys securely:

- Environment variables
- Secure credential stores
- Encrypted configuration files
- Never in source code

## Error Handling Strategies

### Check All Return Values

    def safe_api_call(func, *args, operation_name="operation"):
        """Wrapper for FEX Core API calls with error handling."""
        result = func(*args)

        if result < 0:
            error_map = {
                -1: "General error",
                -4: "Image format not supported",
                -5: "Image ID not found",
                -6: "Could not open image",
                -20: "Invalid license key"
            }
            error_msg = error_map.get(result, f"Unknown error: {result}")
            raise RuntimeError(f"{operation_name} failed: {error_msg}")

        return result

### Graceful Degradation

    def get_file_metadata(image_id, file_path):
        """Get file metadata with fallback for missing fields."""

        try:
            # Try to get full metadata
            metadata = get_full_metadata(image_id, file_path)
        except Exception as e:
            logging.warning(f"Full metadata unavailable: {e}")
            # Fall back to basic info
            metadata = {
                'name': os.path.basename(file_path),
                'path': file_path,
                'size': None,
                'modified': None
            }

        return metadata

### Logging Best Practices

    import logging

    # Configure logging
    logging.basicConfig(
        level=logging.INFO,
        format='%(asctime)s - %(levelname)s - %(message)s'
    )

    def process_image(path, key):
        logging.info(f"Opening image: {path}")

        image_id = dll.ImageOpen(path.encode(), key.encode())
        if image_id < 0:
            logging.error(f"Failed to open image: error {image_id}")
            return None

        logging.info(f"Image opened successfully, ID: {image_id}")

        try:
            # ... processing ...
            logging.debug(f"Processing {file_count} files")
        finally:
            dll.ImageClose(image_id)
            logging.info(f"Image closed: {path}")

## Anti-Patterns to Avoid

### 1. Memory Leaks

    # BAD - buffer never freed
    buffer = ctypes.c_void_p()
    size = ctypes.c_uint32()
    dll.GetVersionAsJSON(byref(buffer), byref(size))
    # buffer is leaked!

    # GOOD - always free with try/finally
    try:
        data = ctypes.string_at(buffer, size.value)
    finally:
        dll.FreeAllocatedBuffer(buffer)

### 2. Ignoring Error Codes

    # BAD - ignores errors
    image_id = dll.ImageOpen(path, key)
    files = list_files(image_id)  # Might fail silently

    # GOOD - check all return values
    image_id = dll.ImageOpen(path, key)
    if image_id < 0:
        raise RuntimeError(f"Failed to open: {image_id}")

### 3. Not Closing Resources

    # BAD - image stays open
    def quick_check(path, key):
        image_id = dll.ImageOpen(path, key)
        info = get_info(image_id)
        return info  # Image never closed!

    # GOOD - always close with try/finally
    def quick_check(path, key):
        image_id = dll.ImageOpen(path, key)
        try:
            return get_info(image_id)
        finally:
            dll.ImageClose(image_id)

### 4. Hardcoded Paths

    # BAD - hardcoded paths
    dll = ctypes.CDLL("C:\\Program Files\\FEX\\bin\\FEX.Core.dll")

    # GOOD - relative or configurable paths
    import os
    script_dir = os.path.dirname(__file__)
    dll_path = os.path.join(script_dir, "..", "bin", "Win64", "FEX.Core.dll")
    dll = ctypes.CDLL(dll_path)

### 5. Swallowing Exceptions

    # BAD - hides problems
    try:
        process_image(path)
    except:
        pass  # Silent failure

    # GOOD - log and handle appropriately
    try:
        process_image(path)
    except FileNotFoundError:
        logging.warning(f"Image not found: {path}")
    except RuntimeError as e:
        logging.error(f"Processing failed: {e}")
        raise

## Summary Checklist

Before deploying your FEX Core integration:

- [ ] Using V2 API for new development
- [ ] All buffers freed with `FreeAllocatedBuffer()`
- [ ] Using try/finally or RAII for cleanup
- [ ] All return codes checked
- [ ] License keys not logged or exposed
- [ ] Input paths validated
- [ ] Images closed when done
- [ ] Error handling in place
- [ ] Memory usage tested with large datasets

Back to Documentation Index