> ## Documentation Index
> Fetch the complete documentation index at: https://docs.standardagentbuilder.com/llms.txt
> Use this file to discover all available pages before exploring further.

# @standardagents/sip

> Ultra memory-efficient image processing for edge computing - process 100MP+ images with less than 1MB peak memory

## Overview

The `@standardagents/sip` package (**S**mall **I**mage **P**rocessor) provides memory-efficient image processing designed for memory-constrained edge computing environments.

<Card title="Key Features" icon="image">
  * Process 100MP+ images with less than 1MB peak memory
  * DCT-based scaling for JPEG (decode at reduced resolution)
  * Scanline-by-scanline processing (never holds full image in memory)
  * Native WASM codecs (libjpeg-turbo, libspng)
  * Works in edge runtimes, browsers, and Node.js
</Card>

## Why sip?

Edge computing environments typically have strict memory limits (e.g., 128MB). Traditional image processing libraries decode the entire image into memory:

| Image Size         | Decoded Memory |
| ------------------ | -------------- |
| 12MP (4000×3000)   | \~36MB         |
| 25MP (6000×4166)   | \~75MB         |
| 50MP (8688×5792)   | \~151MB        |
| 100MP (11375×8992) | \~307MB        |

<Warning>
  A single 25MP image upload can crash your edge function without sip's streaming approach.
</Warning>

**sip's solution:**

1. **DCT Scaling** - Decode JPEG at 1/2, 1/4, or 1/8 scale during decompression
2. **Scanline Streaming** - Process one row at a time (\~50KB peak for any image size)
3. **Native WASM** - libjpeg-turbo and libspng compiled to WebAssembly

## Installation

```bash npm theme={null}
npm install @standardagents/sip
```

```bash pnpm theme={null}
pnpm add @standardagents/sip
```

```bash yarn theme={null}
yarn add @standardagents/sip
```

<Warning>
  JPEG and PNG processing requires the WASM module to be built. See [WASM Build](#wasm-build) below.
</Warning>

## Quick Start

```typescript theme={null}
import { sip, probe } from "@standardagents/sip"
import { readFile, writeFile } from "fs/promises"

// Read an image file
const imageBuffer = await readFile("my-image.jpg")

// Get image info without decoding
const info = probe(imageBuffer)
console.log(info)
// { format: 'jpeg', width: 4000, height: 3000, hasAlpha: false }

// Process image: resize and convert to JPEG
const result = await sip.process(imageBuffer, {
  maxWidth: 2048,
  maxHeight: 2048,
  quality: 85,
})

console.log(result)
// {
//   data: ArrayBuffer,
//   width: 2048,
//   height: 1536,
//   mimeType: 'image/jpeg',
//   originalFormat: 'jpeg'
// }

// Save the resized image
await writeFile("my-image-resized.jpg", Buffer.from(result.data))
```

## Functions

### probe

Detect image format and dimensions by reading only the header bytes. No full decode occurs.

```typescript theme={null}
import { probe } from "@standardagents/sip"

const info = probe(imageBuffer)
// { format: 'jpeg', width: 4000, height: 3000, hasAlpha: false }
```

**Parameters:**

| Parameter | Type                        | Description |
| --------- | --------------------------- | ----------- |
| `data`    | `ArrayBuffer \| Uint8Array` | Image data  |

**Returns:** `ProbeResult`

```typescript theme={null}
interface ProbeResult {
  format: "jpeg" | "png" | "webp" | "avif" | "unknown"
  width: number   // 0 if unknown
  height: number  // 0 if unknown
  hasAlpha: boolean
}
```

**Format Detection:**

| Format | Magic Bytes                         |
| ------ | ----------------------------------- |
| JPEG   | `FF D8 FF`                          |
| PNG    | `89 50 4E 47 0D 0A 1A 0A`           |
| WebP   | `RIFF....WEBP`                      |
| AVIF   | `....ftyp` with `avif`/`avis` brand |

### sip.process

Process an image: decode, resize, and encode to JPEG.

```typescript theme={null}
import { sip } from "@standardagents/sip"

const result = await sip.process(imageBuffer, {
  maxWidth: 2048,
  maxHeight: 2048,
  maxBytes: 1.5 * 1024 * 1024,
  quality: 85,
})
```

**Parameters:**

| Parameter | Type             | Description              |
| --------- | ---------------- | ------------------------ |
| `data`    | `ArrayBuffer`    | Input image data         |
| `options` | `ProcessOptions` | Processing configuration |

**Options:**

| Option      | Type     | Default           | Description                     |
| ----------- | -------- | ----------------- | ------------------------------- |
| `maxWidth`  | `number` | `4096`            | Maximum output width in pixels  |
| `maxHeight` | `number` | `4096`            | Maximum output height in pixels |
| `maxBytes`  | `number` | `1572864` (1.5MB) | Target maximum file size        |
| `quality`   | `number` | `85`              | JPEG quality (1-100)            |

**Returns:** `Promise<ProcessResult>`

```typescript theme={null}
interface ProcessResult {
  data: ArrayBuffer        // Output JPEG data
  width: number            // Output width
  height: number           // Output height
  mimeType: "image/jpeg"   // Always JPEG
  originalFormat: ImageFormat  // Input format detected
}
```

**Behavior:**

* **Aspect Ratio**: Always preserved. Output fits within `maxWidth×maxHeight` box
* **Quality Reduction**: If output exceeds `maxBytes`, retries with lower quality
* **Size Reduction**: If still over `maxBytes` at minimum quality, resizes smaller

### initStreaming

Initialize the WASM module. Optional but recommended for reducing first-call latency.

```typescript theme={null}
import { initStreaming } from "@standardagents/sip"

// Call early in your application startup
const available = await initStreaming()
console.log(`WASM available: ${available}`)
```

**Returns:** `Promise<boolean>` - `true` if WASM loaded successfully.

## Format Support

### Input Formats

| Format   | Decoder              | Method                 | Memory Usage |
| -------- | -------------------- | ---------------------- | ------------ |
| **JPEG** | libjpeg-turbo (WASM) | DCT scaling + scanline | Minimal      |
| **PNG**  | libspng (WASM)       | Progressive scanline   | Minimal      |
| **WebP** | @jsquash/webp        | Full decode            | Higher       |
| **AVIF** | @jsquash/avif        | Full decode            | Higher       |

### Output Format

<Info>
  Output is always **JPEG**. This simplifies the encoder and provides universal browser compatibility.
</Info>

## Memory Model

### DCT Scaling (JPEG only)

JPEG uses Discrete Cosine Transform (DCT) for compression. libjpeg-turbo can decode at reduced resolution **during decompression**:

| Scale | Output Size          | Memory Savings |
| ----- | -------------------- | -------------- |
| 1/1   | Full size            | None           |
| 1/2   | 50% each dimension   | 4× less        |
| 1/4   | 25% each dimension   | 16× less       |
| 1/8   | 12.5% each dimension | 64× less       |

**Example:** 6800×4500 image (30.6MP)

| Scale | Output    | Decode Memory |
| ----- | --------- | ------------- |
| 1/1   | 6800×4500 | \~92MB        |
| 1/2   | 3400×2250 | \~23MB        |
| 1/4   | 1700×1125 | \~5.7MB       |
| 1/8   | 850×563   | \~1.4MB       |

sip automatically selects the optimal scale based on target dimensions.

### Scanline Streaming

Instead of decoding the entire image, sip processes row-by-row:

```
Input Image (any size)
        ↓
Decode Row 0 → Resize → Encode  ← Only these rows in memory
Decode Row 1 → Resize → Encode
        ↓
Output JPEG (streamed)
```

**Memory per row** (2000px width): \~24KB total, constant regardless of image height.

## WASM Build

### Prerequisites

Install the Emscripten SDK:

```bash theme={null}
# Clone emsdk
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk

# Install and activate latest version
./emsdk install latest
./emsdk activate latest

# Add to current shell
source ./emsdk_env.sh
```

### Building

```bash theme={null}
git clone git@github.com:standardagents/sip.git
cd sip
pnpm install
pnpm build:wasm
```

This will:

1. Download libjpeg-turbo, libspng, and miniz to `wasm/libs/`
2. Compile with Emscripten
3. Output `dist/sip.js` and `dist/sip.wasm`

<Warning>
  The WASM files are not committed to git. Your CI/CD pipeline must run `pnpm build:wasm`.
</Warning>

## Configuration Examples

**Recommended Settings by Use Case:**

| Use Case     | maxWidth | maxHeight | maxBytes | quality |
| ------------ | -------- | --------- | -------- | ------- |
| Thumbnails   | 400      | 400       | 50KB     | 80      |
| Preview      | 1024     | 1024      | 200KB    | 82      |
| Standard     | 2048     | 2048      | 1.5MB    | 85      |
| High Quality | 4096     | 4096      | 5MB      | 90      |

## Examples

### Thumbnail Generation

```typescript theme={null}
import { sip } from "@standardagents/sip"

async function createThumbnail(buffer: ArrayBuffer): Promise<ArrayBuffer> {
  const result = await sip.process(buffer, {
    maxWidth: 200,
    maxHeight: 200,
    maxBytes: 30 * 1024, // 30KB max
    quality: 80,
  })
  return result.data
}
```

### Upload Handler with Validation

```typescript theme={null}
import { sip, probe } from "@standardagents/sip"

const ALLOWED_FORMATS = ["jpeg", "png", "webp", "avif"]
const MAX_INPUT_SIZE = 50 * 1024 * 1024 // 50MB

async function handleImageUpload(buffer: ArrayBuffer) {
  // Validate input size
  if (buffer.byteLength > MAX_INPUT_SIZE) {
    throw new Error(`Image too large: ${buffer.byteLength} bytes`)
  }

  // Validate format
  const info = probe(buffer)
  if (!ALLOWED_FORMATS.includes(info.format)) {
    throw new Error(`Unsupported format: ${info.format}`)
  }

  // Process
  return await sip.process(buffer, {
    maxWidth: 4096,
    maxHeight: 4096,
    maxBytes: 2 * 1024 * 1024,
    quality: 85,
  })
}
```

### Edge Function Integration

```typescript theme={null}
import { sip, probe, initStreaming } from "@standardagents/sip"

// Warm up WASM on startup
let wasmReady = initStreaming()

export default {
  async fetch(request: Request): Promise<Response> {
    await wasmReady

    if (request.method !== "POST") {
      return new Response("Method not allowed", { status: 405 })
    }

    try {
      const buffer = await request.arrayBuffer()
      const result = await sip.process(buffer, {
        maxWidth: 2048,
        maxHeight: 2048,
        quality: 85,
      })

      return new Response(result.data, {
        headers: {
          "Content-Type": "image/jpeg",
          "Content-Length": String(result.data.byteLength),
        },
      })
    } catch (error) {
      return new Response(`Processing failed: ${error.message}`, { status: 500 })
    }
  },
}
```

### Multiple Size Generation

```typescript theme={null}
import { sip } from "@standardagents/sip"

async function generateSizes(buffer: ArrayBuffer) {
  const [thumbnail, preview, full] = await Promise.all([
    sip.process(buffer, { maxWidth: 150, maxHeight: 150, quality: 80 }),
    sip.process(buffer, { maxWidth: 800, maxHeight: 800, quality: 82 }),
    sip.process(buffer, { maxWidth: 2048, maxHeight: 2048, quality: 85 }),
  ])

  return {
    thumbnail: thumbnail.data,
    preview: preview.data,
    full: full.data,
  }
}
```

## TypeScript Support

The package includes full TypeScript definitions:

```typescript theme={null}
import type {
  ProbeResult,
  ProcessOptions,
  ProcessResult,
  ImageFormat,
} from "@standardagents/sip"
```

## Limitations

<AccordionGroup>
  <Accordion title="Output is always JPEG">
    No PNG, WebP, or AVIF output. All processed images become JPEG.
  </Accordion>

  <Accordion title="No alpha channel preservation">
    Transparency is discarded. PNG/WebP with alpha become opaque JPEG.
  </Accordion>

  <Accordion title="WASM required for JPEG/PNG">
    Without WASM, JPEG and PNG processing throws an error. WebP and AVIF still work but use more memory.
  </Accordion>

  <Accordion title="maxBytes is best-effort">
    The output may slightly exceed `maxBytes`. The algorithm tries quality reduction first, then dimension reduction.
  </Accordion>

  <Accordion title="No EXIF rotation">
    EXIF orientation is not applied. Handle EXIF separately if needed.
  </Accordion>
</AccordionGroup>

## Error Handling

```typescript theme={null}
try {
  const result = await sip.process(imageBuffer, options)
} catch (error) {
  if (error.message.includes("WASM module not available")) {
    // WASM not built - run `pnpm build:wasm`
    console.error("Run: pnpm build:wasm")
  } else if (error.message.includes("Unknown image format")) {
    // Unsupported format
    console.error("Unsupported image format")
  } else {
    // Other error (corrupt image, etc.)
    console.error("Processing failed:", error.message)
  }
}
```

## Comparison with Alternatives

| Library         | 100MP JPEG Memory | Edge Compatible | Output Formats |
| --------------- | ----------------- | --------------- | -------------- |
| **sip**         | \~50KB            | Yes             | JPEG           |
| sharp (Node)    | \~300MB           | No              | Many           |
| jimp            | \~300MB           | No              | Many           |
| @cf-wasm/photon | \~300MB           | Limited\*       | Many           |

\*@cf-wasm/photon requires paid bindings for large images in some edge environments.

## Next Steps

<CardGroup cols={2}>
  <Card title="React SDK" icon="react" href="/user-interfaces/react">
    Build UIs that connect to agent threads
  </Card>

  <Card title="Tools" icon="wrench" href="/core-concepts/tools">
    Create tools that process images
  </Card>
</CardGroup>