Context Grabber

The Context Grabber LOP collects multi-modal context from across your TouchDesigner network — images from TOPs, text from DATs, custom text snippets, and links to external media (PDF, video, audio) — and packages it for injection into LLM prompts. Other operators like Agent or Chat reference this operator to automatically include rich context in their API calls.

How it Works

This operator doesn’t use wired inputs or outputs. Instead, other LOPs (like Agent) reference it via an OP parameter and automatically retrieve its context before making an API call.

Requirements

• Python packages: opencv-python and numpy (install via the ChatTD Python Manager).

Context Sources

The operator supports six types of context, each with its own toggle and sequence parameters on the Grab Context From Ops page:

Source	Toggle	What it does
Images	Add Images to User	Converts TOPs to base64 and attaches them as vision input
DATs	Add DATs to Context	Reads table or text DATs and includes their content
Text	Add Text to Context	Includes custom text snippets you type directly
PDFs	Add PDF to User	Attaches public PDF URLs for model processing
Video	Add Video to User	Attaches public video URLs for model processing
Audio	Add Audio to User	Attaches public audio URLs for model processing

Each source type uses a sequence parameter — click the + button to add multiple items of the same type (e.g., multiple images or multiple DATs).

Usage Examples

Adding Images as Context

1. Enable Add Images to User on the Grab Context From Ops page.
2. In the Images sequence, set the TOP parameter to a TOP in your network (e.g., a moviefilein or render TOP).
3. Optionally add a Label to describe what the image shows — this label is included as text context alongside the image.
4. Set Image Detail Level to control vision analysis quality (Auto, Low, or High).

Adding DAT Content as Context

1. Enable Add DATs to Context.
2. In the Text or Table DAT sequence, set the DAT parameter to any text or table DAT.
3. Add a Label to describe the content (e.g., “Sensor readings”, “User preferences”).
4. Table DATs are automatically converted to tab-separated text. Text DATs are included as-is.

Adding Media URLs

1. Enable the relevant toggle (Add PDF to User, Add Video to User, or Add Audio to User).
2. Enter a public HTTPS URL in the corresponding sequence parameter.
3. URLs must be publicly accessible — the LLM provider fetches them directly.

Model Support

Not all models support PDF, video, or audio inputs. Check your model’s capabilities before using these features. Gemini models have the broadest media support.

Linking to an Agent

1. Configure the Context Grabber with your desired sources (images, DATs, text, etc.).
2. Set Context Insert Mode to control how text context is injected (System, User, Assistant, or Split).
3. On the Agent operator, find the context OP parameter and drag your Context Grabber onto it.
4. When the Agent makes an LLM call, it automatically retrieves and includes all configured context.

Debugging Context Output

1. Configure the Context Grabber with your desired sources.
2. Pulse Print Context.
3. Check the TouchDesigner textport (Alt+T) to see the exact messages and media that would be sent to the LLM.

Best Practices

• Use descriptive labels on all context sources — they help the LLM understand what each piece of context represents.
• Set Context Insert Mode to “System” for background reference material, or “User” when the context is part of the user’s question.
• Use Split mode with Split Context Location set to “Both” to place context at both the start and end of the conversation for maximum visibility.
• Keep image resolution reasonable — very large TOPs increase token usage and API costs. Consider using a resolution TOP to downscale before feeding into the Context Grabber.

Parameters

Grab Context From Ops

Context Insert Mode (Contextinsertmode) op('context_grabber').par.Contextinsertmode Menu

Default:

System Message

Options:

System Message, User Message, Assistant Message, Split

Split Context Location (Splitcontextlocation) op('context_grabber').par.Splitcontextlocation Menu

Default:

Start

Options:

Start, End, Both

Print Context (Printcontext) op('context_grabber').par.Printcontext Pulse

Print the current context that would be sent to the LLM

Default:

False

Add Images to User (Addimagestouser) op('context_grabber').par.Addimagestouser Toggle

When enabled, adds images to the last user message instead of context

Default:

False

Image Detail Level (Imagedetail) op('context_grabber').par.Imagedetail Menu

Detail level for vision model analysis

Default:

auto

Options:

auto, low, high

Images (Images) op('context_grabber').par.Images Sequence

Default:

0

TOP (Images0top) op('context_grabber').par.Images0top TOP

Default:

"" (Empty String)

Label (Images0label) op('context_grabber').par.Images0label Str

Default:

"" (Empty String)

Add DATs to Context (Adddattocontext) op('context_grabber').par.Adddattocontext Toggle

When enabled, adds text content to context message

Default:

False

Text or Table DAT (Datop) op('context_grabber').par.Datop Sequence

Default:

0

DAT (Datop0dat) op('context_grabber').par.Datop0dat OP

Default:

"" (Empty String)

Label (Datop0label) op('context_grabber').par.Datop0label Str

Default:

"" (Empty String)

Add Text to Context (Addtexttocontext) op('context_grabber').par.Addtexttocontext Toggle

When enabled, adds messages to context

Default:

False

Messages (Text) op('context_grabber').par.Text Sequence

Default:

0

Text (Text0text) op('context_grabber').par.Text0text Str

Default:

"" (Empty String)

Label (Text0label) op('context_grabber').par.Text0label Str

Default:

"" (Empty String)

Add PDF to User (Addpdftouser) op('context_grabber').par.Addpdftouser Toggle

Enable adding PDF URL to user message

Default:

False

Pdf (Pdf) op('context_grabber').par.Pdf Sequence

Default:

0

PDF URL (Pdf0url) op('context_grabber').par.Pdf0url Str

Public HTTPS URL to PDF file (application/pdf)

Default:

"" (Empty String)

Add Video to User (Addvideotouser) op('context_grabber').par.Addvideotouser Toggle

Enable adding Video URL to user message

Default:

False

Video (Video) op('context_grabber').par.Video Sequence

Default:

0

Video URL (Video0url) op('context_grabber').par.Video0url Str

Public HTTPS URL to Video file (video/mp4)

Default:

"" (Empty String)

Add Audio to User (Addaudiotouser) op('context_grabber').par.Addaudiotouser Toggle

Enable adding Audio URL to user message

Default:

False

Audio (Audio) op('context_grabber').par.Audio Sequence

Default:

0

Audio URL (Audio0url) op('context_grabber').par.Audio0url Str

Public HTTPS URL to Audio file (audio/mp3)

Default:

"" (Empty String)

Changelog

v1.0.12026-03-01

• Fix Text sequence treating str parameter as DAT operator - Text{i}text is a str par, use its value directly instead of calling .isTable/.text/.path on it
• Initial commit

v1.0.02025-02-23

Initial release

Related Operators

• Agent
• Chat
• Caption