# FEX Core - Architecture Overview

This document explains how FEX Core works, helping you understand the key concepts before integrating with the SDK.

## High-Level Architecture

FEX Core is a forensic image processing library that reads data from forensic disk images without modifying them. Here's how data flows through the system:

```
+------------------+     +------------------+     +---------------------+     +-----------+
|  Forensic Image  | --> |   FEX Core DLL   | --> | Filesystem Parser   | --> | File Data |
|  (E01, DD, L01)  |     | (Container Layer)|     | (NTFS, FAT, EXT4)   |     |           |
+------------------+     +------------------+     +---------------------+     +-----------+
        |                        |                         |                       |
        v                        v                         v                       v
   Raw bytes in            Decompress/               Parse directory            Return file
   container format        decrypt if needed         structures                 contents
```

### Layer Overview

| Layer                | Responsibility                                           | Examples                |
| -------------------- | -------------------------------------------------------- | ----------------------- |
| **Container Layer**  | Open forensic image files, handle compression/encryption | E01, L01, DD, AFF4      |
| **Filesystem Layer** | Parse directory structures, file metadata                | NTFS, FAT32, EXT4, HFS+ |
| **API Layer**        | Expose functions to applications                         | V1, V2, V3 APIs         |
| **Your Application** | Call API functions, display results                      | Python, C#, C++         |

### Read-Only Principle

FEX Core operates in **read-only mode** to preserve forensic integrity. The library:

- Never modifies the source image
- Never writes to the filesystem being examined
- Maintains evidence integrity for legal proceedings

This is critical for forensic applications where evidence must remain unaltered.

## Image Containers vs Filesystems

Understanding the distinction between **containers** and **filesystems** is essential for working with FEX Core.

### Containers (The Envelope)

A **container** is the file format that wraps the raw disk data. Think of it as an envelope containing a letter.

| Container  | Description                                           | Common Use                                 |
| ---------- | ----------------------------------------------------- | ------------------------------------------ |
| **DD/RAW** | Uncompressed byte-for-byte copy                       | Simple acquisitions, maximum compatibility |
| **E01**    | EnCase format with compression, hashing, and metadata | Law enforcement, enterprise forensics      |
| **L01**    | Logical Evidence File (file-level, not full disk)     | Targeted collections, mobile devices       |
| **AFF4**   | Advanced Forensic Format v4 (if enabled)              | Modern acquisitions, streaming             |

**Key Point:** The container format doesn't affect what files you can access - it only affects how the raw disk data is stored and transported.

### Filesystems (The Contents)

A **filesystem** is the organizational structure on the disk that defines folders, files, and metadata. Think of it as the letter inside the envelope.

| Filesystem      | Platform  | Notes                                                     |
| --------------- | --------- | --------------------------------------------------------- |
| **NTFS**        | Windows   | Full support including alternate data streams, timestamps |
| **FAT12/16/32** | Universal | Legacy systems, USB drives, memory cards                  |
| **exFAT**       | Universal | Large files, modern removable media                       |
| **EXT2/3/4**    | Linux     | Standard Linux filesystem family                          |
| **HFS+**        | macOS     | Legacy Mac filesystem                                     |
| **APFS**        | macOS     | Modern Apple filesystem (limited support)                 |

### Example Combinations

You might encounter any container+filesystem combination:

- **E01 containing NTFS** - Windows server disk imaged with EnCase
- **DD containing FAT32** - USB drive raw copy
- **L01 containing files** - Logical collection from mobile device
- **DD containing EXT4** - Linux server raw disk image

FEX Core handles this automatically - you open the container, and the library detects and parses whatever filesystem is inside.

## Supported Image Formats

### DD/RAW Format

```
+------------------------------------------+
| Raw disk bytes (no headers or metadata)  |
+------------------------------------------+
```

- **Extension:** `.dd`, `.raw`, `.img`, `.001`
- **Compression:** None (can be split into segments)
- **Metadata:** None stored in file
- **Use Case:** Maximum compatibility, simple tools

### E01 (EnCase Expert Witness Format)

```
+--------+------------------+--------+------------------+-----+
| Header | Compressed Chunk | CRC    | Compressed Chunk | ... |
+--------+------------------+--------+------------------+-----+
```

- **Extension:** `.E01`, `.E02`, ... (multi-file)
- **Compression:** zlib compression per chunk
- **Metadata:** Case info, acquisition details, hash values
- **Use Case:** Professional forensics, chain of custody

### L01 (Logical Evidence File)

```
+--------+------------------------+
| Header | Logical file contents  |
+--------+------------------------+
```

- **Extension:** `.L01`, `.L02`, ...
- **Compression:** Optional
- **Metadata:** File paths, timestamps
- **Use Case:** Targeted collections, mobile forensics

### AFF4 (Advanced Forensic Format)

- **Extension:** `.aff4`
- **Compression:** Various algorithms
- **Metadata:** RDF-based, extensible
- **Use Case:** Modern acquisitions

## Supported Filesystems

### NTFS (Windows)

**Coverage:** Full support

- Directory enumeration
- File content extraction
- Timestamps (created, modified, accessed, MFT modified)
- File attributes (hidden, system, archive, etc.)
- Alternate data streams (ADS)
- Deleted file recovery (where MFT entry intact)

### FAT12/16/32

**Coverage:** Full support

- Directory enumeration
- File content extraction
- 8.3 and long filenames
- Timestamps (limited precision)
- Deleted file markers

### exFAT

**Coverage:** Full support

- Large file support (>4GB)
- Modern timestamp precision
- Directory enumeration and extraction

### EXT2/3/4 (Linux)

**Coverage:** Full support

- Inode-based enumeration
- Extended attributes
- Journal recovery (EXT3/4)
- Unix permissions and ownership

### HFS+ (macOS Legacy)

**Coverage:** Partial support

- Directory enumeration
- File extraction
- Resource forks
- Note: APFS has limited support

### Cross-Platform Availability

All filesystems are supported on all platforms where FEX Core runs:

| Filesystem | Windows | Linux |
| ---------- | ------- | ----- |
| NTFS       | Yes     | Yes   |
| FAT32      | Yes     | Yes   |
| EXT4       | Yes     | Yes   |
| HFS+       | Yes     | Yes   |

## Memory Management Architecture

FEX Core provides two API patterns for memory management. **V2 is recommended for all new development.**

### V1 API Pattern (Legacy)

In V1, the **caller allocates** buffers and passes them to the DLL:

```
Caller                              DLL
  |                                  |
  |-- Guess buffer size ------------>|
  |-- Allocate buffer ---------------|
  |-- Call function(buffer, size) -->|
  |<-- Fill buffer, return size -----|
  |-- If too small, resize & retry ->|
  |-- Use data ----------------------|
  |-- Free buffer -------------------|
```

**Problems with V1:**

- Must guess buffer sizes
- Risk of buffer overflows
- Retry loop if buffer too small
- More complex error handling

### V2 API Pattern (Recommended)

In V2, the **DLL allocates** buffers and returns pointers:

```
Caller                              DLL
  |                                  |
  |-- Call function(&ptr, &size) --->|
  |<-- Allocate buffer, return ptr --|
  |-- Use data ----------------------|
  |-- FreeAllocatedBuffer(ptr) ----->|
  |<-- Free buffer ------------------|
```

**Benefits of V2:**

- No buffer sizing guesswork
- No buffer overflow risk
- Simpler code, fewer errors
- DLL manages memory lifecycle

### FreeAllocatedBuffer() Usage

**Critical:** Any buffer returned by a V2 function must be freed with `FreeAllocatedBuffer()`:

```python
# Python example
buffer = ctypes.c_void_p()
size = ctypes.c_uint32()

result = dll.GetVersionAsJSON(ctypes.byref(buffer), ctypes.byref(size))
if result == 0:  # RESULT_OK
    try:
        data = ctypes.string_at(buffer, size.value)
        # Use data...
    finally:
        dll.FreeAllocatedBuffer(buffer)  # Always free!
```

**Memory Leak Warning:** Forgetting to call `FreeAllocatedBuffer()` causes memory leaks. Always use try/finally (Python), using/IDisposable (C#), or RAII (C++) patterns.

### V3 API (Custom Field Extraction)

V3 extends V2 with custom field selection for performance optimization:

```
V2: GetFileSystemRecords_V2()    -> Returns ALL fields for ALL files
V3: GetFileSystemRecordsCustom() -> Returns SPECIFIED fields only
```

Use V3 when:

- Processing millions of files
- Only need specific metadata (name, size, date)
- Optimizing memory usage

## Threading Model

### Thread Safety Guarantees

FEX Core provides the following thread safety guarantees:

| Operation                                   | Thread Safe | Notes                            |
| ------------------------------------------- | ----------- | -------------------------------- |
| Opening different images                    | Yes         | Each image has independent state |
| Reading from same image (different threads) | Yes         | Read operations are safe         |
| Concurrent API calls (different images)     | Yes         | Fully supported                  |
| Single image from multiple threads          | Limited     | Use external synchronization     |

### Concurrent Image Processing

For processing multiple images in parallel:

```
Thread 1: Open Image A -> Read files -> Close Image A
Thread 2: Open Image B -> Read files -> Close Image B
Thread 3: Open Image C -> Read files -> Close Image C
```

Each image gets its own handle/ID, and operations on different images don't interfere.

### Performance Characteristics

| Image Size | Typical Open Time | File Listing (10K files) |
| ---------- | ----------------- | ------------------------ |
| < 1 GB     | < 1 second        | < 1 second               |
| 1-10 GB    | 1-5 seconds       | 1-3 seconds              |
| 10-100 GB  | 5-30 seconds      | 2-10 seconds             |
| > 100 GB   | 30+ seconds       | Varies by filesystem     |

**Tips for Large Images:**

- Use V3 API to request only needed fields
- Process files in batches if memory constrained
- Consider parallel processing for multiple images

### Single Image Concurrency

If multiple threads must access the same image:

1. Open the image once, share the handle
2. Use external locking (mutex) for write-like operations
3. Read operations are generally safe without locking
4. Close the image only after all threads complete

## Next Steps

Now that you understand FEX Core's architecture:

1. **[C# Quick Start](quick-start-csharp.md)** or **[Python Quick Start](quick-start-python.md)** — run your first example in 15 minutes
2. **[V2 API Reference](api/v2-api.md)** - Detailed function documentation
3. **[Common Workflows](tutorials/common-workflows.md)** - Step-by-step tutorials
4. **[Best Practices](tutorials/best-practices.md)** - Optimization and patterns

## Version Information

- **Document Version:** 1.1
- **Last Updated:** 2026-04-17
- **Applies To:** FEX Core SDK v1.3.0+

---

**[Back to Documentation Index](index.md)**
