Search Results for

# FEX Core - Architecture Overview

This document explains how FEX Core works, helping you understand the key concepts before integrating with the SDK.

## High-Level Architecture

FEX Core is a forensic image processing library that reads data from forensic disk images without modifying them. Here's how data flows through the system:

    +------------------+     +------------------+     +---------------------+     +-----------+
    |  Forensic Image  | --> |   FEX Core DLL   | --> | Filesystem Parser   | --> | File Data |
    |  (E01, DD, L01)  |     | (Container Layer)|     | (NTFS, FAT, EXT4)   |     |           |
    +------------------+     +------------------+     +---------------------+     +-----------+
            |                        |                         |                       |
            v                        v                         v                       v
       Raw bytes in            Decompress/               Parse directory            Return file
       container format        decrypt if needed         structures                 contents

### Layer Overview

Layer | Responsibility | Examples
--- | --- | ---
Container Layer | Open forensic image files, handle compression/encryption | E01, L01, DD, AFF4
Filesystem Layer | Parse directory structures, file metadata | NTFS, FAT32, EXT4, HFS+
API Layer | Expose functions to applications | V1, V2, V3 APIs
Your Application | Call API functions, display results | Python, C#, C++

### Read-Only Principle

FEX Core operates in read-only mode to preserve forensic integrity. The library:

- Never modifies the source image
- Never writes to the filesystem being examined
- Maintains evidence integrity for legal proceedings

This is critical for forensic applications where evidence must remain unaltered.

## Image Containers vs Filesystems

Understanding the distinction between containers and filesystems is essential for working with FEX Core.

### Containers (The Envelope)

A container is the file format that wraps the raw disk data. Think of it as an envelope containing a letter.

Container | Description | Common Use
--- | --- | ---
DD/RAW | Uncompressed byte-for-byte copy | Simple acquisitions, maximum compatibility
E01 | EnCase format with compression, hashing, and metadata | Law enforcement, enterprise forensics
L01 | Logical Evidence File (file-level, not full disk) | Targeted collections, mobile devices
AFF4 | Advanced Forensic Format v4 (if enabled) | Modern acquisitions, streaming

Key Point: The container format doesn't affect what files you can access - it only affects how the raw disk data is stored and transported.

### Filesystems (The Contents)

A filesystem is the organizational structure on the disk that defines folders, files, and metadata. Think of it as the letter inside the envelope.

Filesystem | Platform | Notes
--- | --- | ---
NTFS | Windows | Full support including alternate data streams, timestamps
FAT12/16/32 | Universal | Legacy systems, USB drives, memory cards
exFAT | Universal | Large files, modern removable media
EXT2/3/4 | Linux | Standard Linux filesystem family
HFS+ | macOS | Legacy Mac filesystem
APFS | macOS | Modern Apple filesystem (limited support)

### Example Combinations

You might encounter any container+filesystem combination:

- E01 containing NTFS - Windows server disk imaged with EnCase
- DD containing FAT32 - USB drive raw copy
- L01 containing files - Logical collection from mobile device
- DD containing EXT4 - Linux server raw disk image

FEX Core handles this automatically - you open the container, and the library detects and parses whatever filesystem is inside.

## Supported Image Formats

### DD/RAW Format

    +------------------------------------------+
    | Raw disk bytes (no headers or metadata)  |
    +------------------------------------------+

- Extension: `.dd`, `.raw`, `.img`, `.001`
- Compression: None (can be split into segments)
- Metadata: None stored in file
- Use Case: Maximum compatibility, simple tools

### E01 (EnCase Expert Witness Format)

    +--------+------------------+--------+------------------+-----+
    | Header | Compressed Chunk | CRC    | Compressed Chunk | ... |
    +--------+------------------+--------+------------------+-----+

- Extension: `.E01`, `.E02`, ... (multi-file)
- Compression: zlib compression per chunk
- Metadata: Case info, acquisition details, hash values
- Use Case: Professional forensics, chain of custody

### L01 (Logical Evidence File)

    +--------+------------------------+
    | Header | Logical file contents  |
    +--------+------------------------+

- Extension: `.L01`, `.L02`, ...
- Compression: Optional
- Metadata: File paths, timestamps
- Use Case: Targeted collections, mobile forensics

### AFF4 (Advanced Forensic Format)

- Extension: `.aff4`
- Compression: Various algorithms
- Metadata: RDF-based, extensible
- Use Case: Modern acquisitions

## Supported Filesystems

### NTFS (Windows)

Coverage: Full support

- Directory enumeration
- File content extraction
- Timestamps (created, modified, accessed, MFT modified)
- File attributes (hidden, system, archive, etc.)
- Alternate data streams (ADS)
- Deleted file recovery (where MFT entry intact)

### FAT12/16/32

Coverage: Full support

- Directory enumeration
- File content extraction
- 8.3 and long filenames
- Timestamps (limited precision)
- Deleted file markers

### exFAT

Coverage: Full support

- Large file support (>4GB)
- Modern timestamp precision
- Directory enumeration and extraction

### EXT2/3/4 (Linux)

Coverage: Full support

- Inode-based enumeration
- Extended attributes
- Journal recovery (EXT3/4)
- Unix permissions and ownership

### HFS+ (macOS Legacy)

Coverage: Partial support

- Directory enumeration
- File extraction
- Resource forks
- Note: APFS has limited support

### Cross-Platform Availability

All filesystems are supported on all platforms where FEX Core runs:

Filesystem | Windows | Linux
--- | --- | ---
NTFS | Yes | Yes
FAT32 | Yes | Yes
EXT4 | Yes | Yes
HFS+ | Yes | Yes

## Memory Management Architecture

FEX Core provides two API patterns for memory management. V2 is recommended for all new development.

### V1 API Pattern (Legacy)

In V1, the caller allocates buffers and passes them to the DLL:

    Caller                              DLL
      |                                  |
      |-- Guess buffer size ------------>|
      |-- Allocate buffer ---------------|
      |-- Call function(buffer, size) -->|
      |<-- Fill buffer, return size -----|
      |-- If too small, resize & retry ->|
      |-- Use data ----------------------|
      |-- Free buffer -------------------|

Problems with V1:

- Must guess buffer sizes
- Risk of buffer overflows
- Retry loop if buffer too small
- More complex error handling

### V2 API Pattern (Recommended)

In V2, the DLL allocates buffers and returns pointers:

    Caller                              DLL
      |                                  |
      |-- Call function(&ptr, &size) --->|
      |<-- Allocate buffer, return ptr --|
      |-- Use data ----------------------|
      |-- FreeAllocatedBuffer(ptr) ----->|
      |<-- Free buffer ------------------|

Benefits of V2:

- No buffer sizing guesswork
- No buffer overflow risk
- Simpler code, fewer errors
- DLL manages memory lifecycle

### FreeAllocatedBuffer() Usage

Critical: Any buffer returned by a V2 function must be freed with `FreeAllocatedBuffer()`:

    # Python example
    buffer = ctypes.c_void_p()
    size = ctypes.c_uint32()

    result = dll.GetVersionAsJSON(ctypes.byref(buffer), ctypes.byref(size))
    if result == 0:  # RESULT_OK
        try:
            data = ctypes.string_at(buffer, size.value)
            # Use data...
        finally:
            dll.FreeAllocatedBuffer(buffer)  # Always free!

Memory Leak Warning: Forgetting to call `FreeAllocatedBuffer()` causes memory leaks. Always use try/finally (Python), using/IDisposable (C#), or RAII (C++) patterns.

### V3 API (Custom Field Extraction)

V3 extends V2 with custom field selection for performance optimization:

    V2: GetFileSystemRecords_V2()    -> Returns ALL fields for ALL files
    V3: GetFileSystemRecordsCustom() -> Returns SPECIFIED fields only

Use V3 when:

- Processing millions of files
- Only need specific metadata (name, size, date)
- Optimizing memory usage

## Threading Model

### Thread Safety Guarantees

FEX Core provides the following thread safety guarantees:

Operation | Thread Safe | Notes
--- | --- | ---
Opening different images | Yes | Each image has independent state
Reading from same image (different threads) | Yes | Read operations are safe
Concurrent API calls (different images) | Yes | Fully supported
Single image from multiple threads | Limited | Use external synchronization

### Concurrent Image Processing

For processing multiple images in parallel:

    Thread 1: Open Image A -> Read files -> Close Image A
    Thread 2: Open Image B -> Read files -> Close Image B
    Thread 3: Open Image C -> Read files -> Close Image C

Each image gets its own handle/ID, and operations on different images don't interfere.

### Performance Characteristics

Image Size | Typical Open Time | File Listing (10K files)
--- | --- | ---
< 1 GB | < 1 second | < 1 second
1-10 GB | 1-5 seconds | 1-3 seconds
10-100 GB | 5-30 seconds | 2-10 seconds
> 100 GB | 30+ seconds | Varies by filesystem

Tips for Large Images:

- Use V3 API to request only needed fields
- Process files in batches if memory constrained
- Consider parallel processing for multiple images

### Single Image Concurrency

If multiple threads must access the same image:

1. Open the image once, share the handle
2. Use external locking (mutex) for write-like operations
3. Read operations are generally safe without locking
4. Close the image only after all threads complete

## Next Steps

Now that you understand FEX Core's architecture:

1. C# Quick Start or Python Quick Start — run your first example in 15 minutes
2. V2 API Reference - Detailed function documentation
3. Common Workflows - Step-by-step tutorials
4. Best Practices - Optimization and patterns

## Version Information

- Document Version: 1.1
- Last Updated: 2026-04-17
- Applies To: FEX Core SDK v1.3.0+

Back to Documentation Index