# FEX Core SDK - Best Practices

This guide helps you make optimal choices for API version, memory management, performance, and security.

## API Version Selection

FEX Core provides three API versions. Choose based on your use case.

### Decision Matrix

| Scenario | Recommended API | Reason |
|----------|----------------|--------|
| New development | **V2** | Safest, simplest, no buffer sizing issues |
| Legacy migration | V1 | Compatibility with existing V1 code |
| Performance-critical | **V3** | Custom field extraction reduces memory |
| Large-scale processing | **V3** | Only fetch needed fields |
| Simple utilities | **V2** | Easy to implement correctly |

### V1 API (Legacy)

**When to use:**
- Maintaining existing V1 code
- Migrating from older FEX Core versions
- Specific compatibility requirements

**Example:**
```python
# V1: Caller allocates buffer
buffer = ctypes.create_string_buffer(4096)
result = dll.DllVersionAsJSON(buffer, len(buffer))
# Risk: Buffer might be too small
```

**Limitations:**
- Must guess buffer sizes
- Retry loop if buffer too small
- Risk of buffer overflows

### V2 API (Recommended)

**When to use:**
- All new development
- When simplicity matters
- When safety is priority

**Example:**
```python
# V2: DLL allocates buffer
buffer = ctypes.c_void_p()
size = ctypes.c_uint32()
result = dll.GetVersionAsJSON(byref(buffer), byref(size))
# Always works, no size guessing
dll.FreeAllocatedBuffer(buffer)
```

**Benefits:**
- No buffer sizing guesswork
- No overflow risks
- Cleaner code

### V3 API (Performance)

**When to use:**
- Processing millions of files
- Only need specific metadata
- Memory is constrained

**Example:**
```python
# V3: Request only needed fields
fields = "Name,Size,Modified"
result = dll.GetFileSystemRecordsCustom(
    image_id, 0, fields.encode(),
    byref(buffer), byref(size)
)
# Returns only requested fields, not full records
```

**Benefits:**
- Reduced memory usage
- Faster for large filesystems
- Customizable output

---

## Memory Management Patterns

### The Golden Rule

**Always free DLL-allocated buffers with `FreeAllocatedBuffer()`.**

### Reading file content at scale — use the handle-based stream API

`ReadFileData` looks convenient but leaks an internal reader per `FileIndex`
that lives until `CloseImage()` — it is the #1 cause of runaway memory in
bulk-extraction scripts. For any workload that touches more than a handful
of files, or runs on multiple threads, use `RequestFileStream` /
`ReadFileStream` / `ReleaseFileStream` instead.

```python
handle = ctypes.c_int32(0)
if dll.RequestFileStream(image_id, file_index, ctypes.byref(handle)) != 0:
    return
try:
    # chunked ReadFileStream calls here
    ...
finally:
    # Single finally block — the pool slot is held until you release.
    dll.ReleaseFileStream(image_id, handle)
```

`ReleaseFileStream` failing with `RESULT_INVALID_HANDLE` (−30) in a cleanup
path is benign — treat it as already-released. See [V2 API / Handle-Based
Stream API](../api/v2-api.md#handle-based-stream-api-recommended-for-bulk-reads)
for full signatures.


### Python: try/finally Pattern

```python
buffer = ctypes.c_void_p()
size = ctypes.c_uint32()

result = dll.GetImageInfoByIdAsJSON(image_id, byref(buffer), byref(size))

if result == 0:
    try:
        # Use the buffer
        data = ctypes.string_at(buffer, size.value)
        process_data(data)
    finally:
        # ALWAYS free, even if exception occurs
        dll.FreeAllocatedBuffer(buffer)
```

### C#: using/IDisposable Pattern

```csharp
public class FexBuffer : IDisposable
{
    private IntPtr _buffer;
    private bool _disposed;

    public FexBuffer(IntPtr buffer)
    {
        _buffer = buffer;
    }

    public void Dispose()
    {
        if (!_disposed && _buffer != IntPtr.Zero)
        {
            FexCore.FreeAllocatedBuffer(_buffer);
            _buffer = IntPtr.Zero;
            _disposed = true;
        }
    }
}

// Usage
using (var buffer = new FexBuffer(resultBuffer))
{
    // Use buffer
    string json = Marshal.PtrToStringUTF8(buffer.Ptr, size);
}
// Automatically freed
```

### C++: RAII Pattern

```cpp
class FexBuffer {
private:
    void* buffer_;

public:
    explicit FexBuffer(void* buffer) : buffer_(buffer) {}

    ~FexBuffer() {
        if (buffer_) {
            FreeAllocatedBuffer(buffer_);
        }
    }

    // Non-copyable
    FexBuffer(const FexBuffer&) = delete;
    FexBuffer& operator=(const FexBuffer&) = delete;

    void* get() const { return buffer_; }
};

// Usage
void* rawBuffer;
uint32_t size;

if (GetVersionAsJSON(&rawBuffer, &size) == 0) {
    FexBuffer buffer(rawBuffer);  // RAII takes ownership
    // Use buffer.get()
}  // Automatically freed when scope exits
```

### Memory Leak Detection

Signs of memory leaks:
- Increasing memory usage over time
- Application slows down during long runs
- Out-of-memory errors

Debug approach:
```python
import tracemalloc

tracemalloc.start()

# Your FEX Core operations
for i in range(100):
    # ... operations that might leak ...
    pass

current, peak = tracemalloc.get_traced_memory()
print(f"Current memory: {current / 1024 / 1024:.1f} MB")
print(f"Peak memory: {peak / 1024 / 1024:.1f} MB")

tracemalloc.stop()
```

---

## Performance Optimization

### 1. Use V3 for Large Filesystems

When processing millions of files, request only needed fields:

```python
# Slow: V2 returns all fields
result = dll.GetFileSystemRecords_V2(image_id, 0, byref(buffer), byref(size))

# Fast: V3 returns only requested fields
fields = "Name,Size"  # Only what you need
result = dll.GetFileSystemRecordsCustom(
    image_id, 0, fields.encode(),
    byref(buffer), byref(size)
)
```

### 2. Process Files in Batches

For large operations, process in manageable chunks:

```python
def process_in_batches(files, batch_size=1000):
    """Process files in batches to manage memory."""
    for i in range(0, len(files), batch_size):
        batch = files[i:i + batch_size]
        yield process_batch(batch)

        # Optional: Force garbage collection between batches
        import gc
        gc.collect()
```

### 3. Reuse Image Handles

Opening images is expensive. Open once, use many times:

```python
# Bad: Open/close for each operation
for file_path in file_paths:
    image_id = dll.ImageOpen(image_path, key)
    # ... process one file ...
    dll.ImageClose(image_id)

# Good: Open once, process all, close once
image_id = dll.ImageOpen(image_path, key)
for file_path in file_paths:
    # ... process file ...
dll.ImageClose(image_id)
```

### 4. Cache Frequently Used Data

```python
class ImageCache:
    def __init__(self, image_id):
        self.image_id = image_id
        self._file_list = None
        self._image_info = None

    @property
    def files(self):
        if self._file_list is None:
            self._file_list = list_files(self.image_id)
        return self._file_list

    @property
    def info(self):
        if self._image_info is None:
            self._image_info = get_image_info(self.image_id)
        return self._image_info
```

---

## Security Considerations

### Input Validation

Always validate paths before passing to the DLL:

```python
import os

def safe_open_image(image_path, license_key):
    """Open image with path validation."""

    # Normalize and validate path
    normalized = os.path.normpath(image_path)

    # Check for path traversal attempts
    if '..' in normalized:
        raise ValueError("Path traversal not allowed")

    # Verify file exists
    if not os.path.isfile(normalized):
        raise FileNotFoundError(f"Image not found: {normalized}")

    # Check file extension
    valid_extensions = {'.dd', '.e01', '.l01', '.aff4', '.raw', '.img'}
    ext = os.path.splitext(normalized)[1].lower()
    if ext not in valid_extensions:
        raise ValueError(f"Unsupported extension: {ext}")

    return dll.ImageOpen(normalized.encode(), license_key.encode())
```

### Evidence Integrity

FEX Core is read-only by design. Maintain integrity by:

1. **Never modify source images**
2. **Work on copies when possible**
3. **Log all operations** for audit trail
4. **Hash images** before and after processing

```python
import hashlib

def hash_file(path):
    """Calculate SHA-256 hash of file."""
    sha256 = hashlib.sha256()
    with open(path, 'rb') as f:
        for chunk in iter(lambda: f.read(8192), b''):
            sha256.update(chunk)
    return sha256.hexdigest()

# Verify integrity
original_hash = hash_file(image_path)
# ... do processing ...
final_hash = hash_file(image_path)

assert original_hash == final_hash, "Image was modified!"
```

### License Key Handling

**Never log or display license keys:**

```python
# BAD - exposes key in logs
print(f"Opening with key: {license_key}")
logging.info(f"License: {license_key}")

# GOOD - mask the key
def mask_key(key):
    if len(key) > 8:
        return key[:4] + '*' * (len(key) - 8) + key[-4:]
    return '*' * len(key)

print(f"Opening with key: {mask_key(license_key)}")
```

**Store keys securely:**
- Environment variables
- Secure credential stores
- Encrypted configuration files
- Never in source code

---

## Error Handling Strategies

### Check All Return Values

```python
def safe_api_call(func, *args, operation_name="operation"):
    """Wrapper for FEX Core API calls with error handling."""
    result = func(*args)

    if result < 0:
        error_map = {
            -1: "General error",
            -4: "Image format not supported",
            -5: "Image ID not found",
            -6: "Could not open image",
            -20: "Invalid license key"
        }
        error_msg = error_map.get(result, f"Unknown error: {result}")
        raise RuntimeError(f"{operation_name} failed: {error_msg}")

    return result
```

### Graceful Degradation

```python
def get_file_metadata(image_id, file_path):
    """Get file metadata with fallback for missing fields."""

    try:
        # Try to get full metadata
        metadata = get_full_metadata(image_id, file_path)
    except Exception as e:
        logging.warning(f"Full metadata unavailable: {e}")
        # Fall back to basic info
        metadata = {
            'name': os.path.basename(file_path),
            'path': file_path,
            'size': None,
            'modified': None
        }

    return metadata
```

### Logging Best Practices

```python
import logging

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)

def process_image(path, key):
    logging.info(f"Opening image: {path}")

    image_id = dll.ImageOpen(path.encode(), key.encode())
    if image_id < 0:
        logging.error(f"Failed to open image: error {image_id}")
        return None

    logging.info(f"Image opened successfully, ID: {image_id}")

    try:
        # ... processing ...
        logging.debug(f"Processing {file_count} files")
    finally:
        dll.ImageClose(image_id)
        logging.info(f"Image closed: {path}")
```

---

## Anti-Patterns to Avoid

### 1. Memory Leaks

```python
# BAD - buffer never freed
buffer = ctypes.c_void_p()
size = ctypes.c_uint32()
dll.GetVersionAsJSON(byref(buffer), byref(size))
# buffer is leaked!

# GOOD - always free with try/finally
try:
    data = ctypes.string_at(buffer, size.value)
finally:
    dll.FreeAllocatedBuffer(buffer)
```

### 2. Ignoring Error Codes

```python
# BAD - ignores errors
image_id = dll.ImageOpen(path, key)
files = list_files(image_id)  # Might fail silently

# GOOD - check all return values
image_id = dll.ImageOpen(path, key)
if image_id < 0:
    raise RuntimeError(f"Failed to open: {image_id}")
```

### 3. Not Closing Resources

```python
# BAD - image stays open
def quick_check(path, key):
    image_id = dll.ImageOpen(path, key)
    info = get_info(image_id)
    return info  # Image never closed!

# GOOD - always close with try/finally
def quick_check(path, key):
    image_id = dll.ImageOpen(path, key)
    try:
        return get_info(image_id)
    finally:
        dll.ImageClose(image_id)
```

### 4. Hardcoded Paths

```python
# BAD - hardcoded paths
dll = ctypes.CDLL("C:\\Program Files\\FEX\\bin\\FEX.Core.dll")

# GOOD - relative or configurable paths
import os
script_dir = os.path.dirname(__file__)
dll_path = os.path.join(script_dir, "..", "bin", "Win64", "FEX.Core.dll")
dll = ctypes.CDLL(dll_path)
```

### 5. Swallowing Exceptions

```python
# BAD - hides problems
try:
    process_image(path)
except:
    pass  # Silent failure

# GOOD - log and handle appropriately
try:
    process_image(path)
except FileNotFoundError:
    logging.warning(f"Image not found: {path}")
except RuntimeError as e:
    logging.error(f"Processing failed: {e}")
    raise
```

---

## Summary Checklist

Before deploying your FEX Core integration:

- [ ] Using V2 API for new development
- [ ] All buffers freed with `FreeAllocatedBuffer()`
- [ ] Using try/finally or RAII for cleanup
- [ ] All return codes checked
- [ ] License keys not logged or exposed
- [ ] Input paths validated
- [ ] Images closed when done
- [ ] Error handling in place
- [ ] Memory usage tested with large datasets

---

**[Back to Documentation Index](../index.md)**
