Skip to content
LOPS Docs

Lyria

v1.0.0

The Lyria LOP connects to Google’s Lyria Realtime API to generate continuous music from text prompts. It streams audio directly into TouchDesigner as CHOP data at 48kHz, allowing real-time music generation that can be mixed, processed, and synchronized with visuals. Multiple weighted prompts can be blended together, and musical parameters like BPM, scale, density, and brightness can be adjusted while music is playing.

Requirements

Section titled “Requirements”
  • Gemini API Key: Obtain a key from Google AI Studio and enter it on the Lyria API page. The key is stored securely via ChatTD’s KeyManager.
  • Python Package: The google-genai package (version 1.16.0 or higher) must be installed. Use the Install/Update google-genai button on the Lyria API page, or Force Reinstall google-genai if you encounter version conflicts.

Input/Output

Section titled “Input/Output”

Inputs

Section titled “Inputs”

None — all configuration is done through parameters and text prompts.

Outputs

Section titled “Outputs”
  • Audio CHOP: Mono 48kHz audio output via the store_output CHOP. This progressive buffer accumulates all generated audio from the current session.

Usage Examples

Section titled “Usage Examples”

Getting Started

Section titled “Getting Started”
  1. On the Lyria API page, enter your Gemini API key in the Gemini API Key field (or pulse Get API Key to open Google AI Studio in your browser).
  2. If the google-genai package is not installed, pulse Install/Update google-genai and wait for the installation to complete. You may need to restart TouchDesigner afterward.
  3. Pulse Connect to Lyria to establish a connection to the Lyria Realtime API.
  4. On the Music Controls page, add a prompt in the Music Prompt sequence — enter descriptive text like “upbeat electronic dance music” and set the Weight (0.0 to 2.0).
  5. Pulse Play Music to start generating audio.

Blending Multiple Prompts

Section titled “Blending Multiple Prompts”
  1. On the Music Controls page, add multiple entries to the Music Prompt sequence using the + button.
  2. Give each prompt a different style description (e.g., “ambient piano” at weight 1.0, “subtle jazz drums” at weight 0.5).
  3. Enable Auto Update Prompts to have changes sent to Lyria automatically as you adjust weights, or pulse Update Prompts on Lyria manually.
  4. Adjust weights in real time to crossfade between musical styles while the stream is playing.

Configuring Musical Parameters

Section titled “Configuring Musical Parameters”
  1. On the Music Controls page, enable Enable BPM and set the desired BPM (30-240).
  2. Enable Enable Scale and choose a key from the Scale menu (e.g., “C Major / A Minor”).
  3. Use Enable Density (0.0-1.0) and Enable Brightness (0.0-1.0) to control the texture of the generated music.
  4. Toggle Enable Bass, Enable Drums, and Enable Other Instruments to mute or enable specific instrument groups.
  5. Set Generation Mode to “Quality (Stable)” for consistent output or “Diversity (Creative)” for more variation.
  6. Pulse Update Music Config to send changes to Lyria, then pulse Reset Stream Context if you changed BPM or Scale (these require a context reset to take effect).

Saving Generated Audio

Section titled “Saving Generated Audio”
  1. On the Playback page, set the Output Folder to your desired save location (relative to the project file).
  2. Optionally set a Base Filename — if left at default, filenames are generated from your prompt text.
  3. Pulse Save Last Audio to save the current audio buffer as a 48kHz mono WAV file.
  4. Enable Autosave on Stop to automatically save whenever you pulse Stop Music.
  5. Toggle Clear on Save to control whether the audio buffer is cleared after saving (disable to keep audio for multiple saves).

Best Practices

Section titled “Best Practices”
  • Use descriptive style prompts rather than artist names. Lyria’s content filter blocks prompts referencing specific artists or copyrighted material. Describe the musical style instead (e.g., “groovy funk bass with syncopated rhythm” instead of a specific artist name).
  • Adjust Temperature and Guidance together. Temperature (0.0-3.0) controls randomness while Guidance (0.0-6.0) controls how closely the output follows your prompts. Start with defaults and adjust incrementally.
  • Enable Auto Connect on Play on the Lyria API page so that pulsing Play Music will automatically connect if you are disconnected.
  • Use Auto Update Prompts for live performance. When enabled, prompt and weight changes are rate-limited (configurable via Auto Update Frequency) to avoid overwhelming the API while still allowing real-time control.
  • Reset context after major changes. After changing BPM or Scale, pulse Reset Stream Context on the Music Controls page to apply those changes properly.

Troubleshooting

Section titled “Troubleshooting”
  • “google-genai not installed”: Pulse Install/Update google-genai on the Lyria API page. If the version is still below 1.16.0 after installing, use Force Reinstall google-genai and restart TouchDesigner.
  • Connection errors: Pulse Diagnose Python Environment to check your SDK version and installation. Verify your API key is valid and that you have access to the Lyria model.
  • Prompt filtered warnings: Lyria blocks prompts that reference specific artists, copyrighted material, or inappropriate content. Rewrite your prompts to describe the musical style generically.
  • No audio after pressing Play: Ensure you have at least one prompt with a non-zero weight in the Music Prompt sequence. Prompts with empty text or zero weight are skipped.
  • BPM/Scale changes not taking effect: These parameters require a context reset. After changing them, pulse Update Music Config followed by Reset Stream Context.
  • SDK version conflicts: Use Diagnose Python Environment to check for multiple google-genai installations. Force Reinstall google-genai clears the pip cache and performs a clean install.

Research & Licensing

Google DeepMind

Google DeepMind is a leading AI research organization focused on developing artificial general intelligence. Their research spans reinforcement learning, natural language processing, computer vision, and creative AI, with a mission to solve intelligence and use it to advance scientific discovery and benefit humanity.

Lyria: Advanced Music Generation Model

Lyria is Google's advanced music generation model that builds upon research from MusicLM. It enables high-quality music generation from text prompts, supporting various musical styles and maintaining coherent structure across long-form compositions. The model is designed for integration into creative workflows and applications.

Technical Details

  • Hierarchical Architecture: Multi-stage generation process with semantic and acoustic modeling
  • Text Conditioning: Natural language descriptions guide music generation
  • Temporal Coherence: Maintains musical structure across extended compositions

Research Impact

  • Creative AI: Democratizing music creation through text-to-music generation
  • Commercial Applications: Integration into Google's creative tools and services
  • Research Foundation: Advancing the field of AI-generated creative content

Citation

@article{agostinelli2023musiclm,
  title={MusicLM: Generating Music From Text},
  author={Agostinelli, Andrea and Denk, Timo I. and Borsos, Zalán and Engel, Jesse and Verzetti, Mauro and Caillon, Antoine and Huang, Qingqing and Jansen, Aren and Roberts, Adam and Tagliasacchi, Marco and Sharifi, Matt and Zeghidour, Neil and Frank, Christian},
  journal={arXiv preprint arXiv:2301.11325},
  year={2023},
  url={https://arxiv.org/abs/2301.11325}
}

Key Research Contributions

  • Hierarchical sequence-to-sequence modeling for high-quality music generation
  • Conditioning on both text descriptions and melodic representations
  • Long-form music generation with coherent structure and style

License

Proprietary - This model is freely available for research and commercial use.

Parameters

Section titled “Parameters”

Music Controls

Section titled “Music Controls”
Status (Status) op('lyria').par.Status Str
Default:
"" (Empty String)
Play Music (Play) op('lyria').par.Play Pulse
Default:
False
Pause Music (Pause) op('lyria').par.Pause Pulse
Default:
False
Stop Music (Resets Stream) (Stop) op('lyria').par.Stop Pulse
Default:
False
Reset Stream Context (Resetcontext) op('lyria').par.Resetcontext Pulse

Use after significant config changes like BPM or Scale to apply them.

Default:
False
Temperature (Temperature) op('lyria').par.Temperature Float
Default:
0.0
Range:
0 to 3
Slider Range:
0 to 3
Guidance (Guidance) op('lyria').par.Guidance Float
Default:
0.0
Range:
0 to 6
Slider Range:
0 to 6
Update Prompts on Lyria (Updateprompts) op('lyria').par.Updateprompts Pulse

Sends the current prompts from the internal table to Lyria.

Default:
False
Auto Update Prompts (Autoupdateprompts) op('lyria').par.Autoupdateprompts Toggle

Automatically update prompts on Lyria when sequence parameters change. Uses rate limiting to avoid API spam.

Default:
False
Music Prompt (Musicprompt) op('lyria').par.Musicprompt Sequence
Default:
0
Text (Musicprompt0text) op('lyria').par.Musicprompt0text Str
Default:
"" (Empty String)
Weight (Musicprompt0weight) op('lyria').par.Musicprompt0weight Float
Default:
0.0
Range:
0 to 2
Slider Range:
0 to 2
Enable Bass (Enablebass) op('lyria').par.Enablebass Toggle
Default:
False
Enable Drums (Enabledrums) op('lyria').par.Enabledrums Toggle
Default:
False
Enable Other Instruments (Enableother) op('lyria').par.Enableother Toggle

Controls other instruments like melody, harmony, etc.

Default:
False
Generation Mode (Generationmode) op('lyria').par.Generationmode Menu
Default:
QUALITY
Options:
QUALITY, DIVERSITY
Update Music Config (Updateconfig) op('lyria').par.Updateconfig Pulse
Default:
False
Enable BPM (Enablebpm) op('lyria').par.Enablebpm Toggle
Default:
False
BPM (Bpm) op('lyria').par.Bpm Int
Default:
0
Range:
30 to 240
Slider Range:
60 to 200
Enable Scale (Enablescale) op('lyria').par.Enablescale Toggle
Default:
False
Scale (Scale) op('lyria').par.Scale Menu
Default:
C_MAJOR_A_MINOR
Options:
C_MAJOR_A_MINOR, D_FLAT_MAJOR_B_FLAT_MINOR, D_MAJOR_B_MINOR, E_FLAT_MAJOR_C_MINOR, E_MAJOR_D_FLAT_MINOR, F_MAJOR_D_MINOR, G_FLAT_MAJOR_E_FLAT_MINOR, G_MAJOR_E_MINOR, A_FLAT_MAJOR_F_MINOR, A_MAJOR_G_FLAT_MINOR, B_FLAT_MAJOR_G_MINOR, B_MAJOR_A_FLAT_MINOR
Enable Density (Enabledensity) op('lyria').par.Enabledensity Toggle
Default:
False
Density (Density) op('lyria').par.Density Float
Default:
0.0
Range:
0 to 1
Slider Range:
0 to 1
Enable Brightness (Enablebrightness) op('lyria').par.Enablebrightness Toggle
Default:
False
Brightness (Brightness) op('lyria').par.Brightness Float
Default:
0.0
Range:
0 to 1
Slider Range:
0 to 1

Lyria API

Section titled “Lyria API”
Status2 (Status2) op('lyria').par.Status2 Str
Default:
"" (Empty String)
Connect to Lyria (Connect) op('lyria').par.Connect Pulse
Default:
False
Disconnect from Lyria (Disconnect) op('lyria').par.Disconnect Pulse
Default:
False
Lyria Model Name (Model) op('lyria').par.Model Str

The specific Lyria model to use for generation.

Default:
"" (Empty String)
List Available Models (Listavailablemodels) op('lyria').par.Listavailablemodels Pulse

Lists all models accessible by the current API key and logs them.

Default:
False
Gemini API Key (Apikey) op('lyria').par.Apikey Str

Enter your Gemini API key from Google AI Studio. It will be stored securely.

Default:
"" (Empty String)
Get API Key (Getapikey) op('lyria').par.Getapikey Pulse

Opens Google AI Studio in your browser to get an API key.

Default:
False
Install/Update google-genai (Installgooglegenai) op('lyria').par.Installgooglegenai Pulse

Installs or updates the google-generativeai Python package via ChatTD PackageManager.

Default:
False
Test Lyria Model Direct (Testlyriamodel) op('lyria').par.Testlyriamodel Pulse

Tests direct connection to the Lyria model (models/lyria-realtime-exp) to verify availability.

Default:
False
Force Reinstall google-genai (Forcereinstallgenai) op('lyria').par.Forcereinstallgenai Pulse

Force reinstalls google-genai>=1.16.0 with cache clearing to resolve version conflicts.

Default:
False
Diagnose Python Environment (Diagnosepythonenv) op('lyria').par.Diagnosepythonenv Pulse

Diagnoses Python environment issues that might prevent SDK updates.

Default:
False
Auto Connect on Play (Autoconnect) op('lyria').par.Autoconnect Toggle

If enabled, hitting 'Play' while disconnected will automatically attempt to connect to Lyria.

Default:
False
Auto Update Frequency (s) (Autoupdatefrequency) op('lyria').par.Autoupdatefrequency Float

Delay in seconds for auto-updating prompts.

Default:
0.0
Range:
0 to 1
Slider Range:
0.2 to 5

Playback

Section titled “Playback”
Audio Device Settings Header
Active (Audioactive) op('lyria').par.Audioactive Toggle
Default:
True
Driver (Driver) op('lyria').par.Driver Menu
Default:
default
Options:
default, asio
Device (Device) op('lyria').par.Device Menu
Default:
default
Options:
default, {0.0.0.00000000}.{d7b929aa-ec27-4f96-bd39-78d6a8c2044a}||Out_1-2_(MOTU_M_Series)||1, {0.0.0.00000000}.{0f011619-4954-432e-b132-2653e7136839}||LEN_P32u-10_(NVIDIA_High_Definition_Audio)||2, {0.0.0.00000000}.{27c8aa00-246f-4537-a091-c52b392d29d4}||LTV_(NVIDIA_High_Definition_Audio)||3, {0.0.0.00000000}.{34b7624e-63b1-49fc-93dc-1d03ca1dd600}||CABLE-B_In_16ch_(VB-Audio_Virtual_Cable_B)||4, {0.0.0.00000000}.{372fb62b-07aa-4580-b062-2c6adba187e7}||Out_3-4_(MOTU_M_Series)||5, {0.0.0.00000000}.{394e3e0c-eac4-4fc9-ad76-1600f0cb570b}||CABLE-A_Input_(VB-Audio_Virtual_Cable_A)||6, {0.0.0.00000000}.{7c163c65-5cda-4fbd-971b-27e1c3b471f0}||F13NA_(NVIDIA_High_Definition_Audio)||7, {0.0.0.00000000}.{896ef90b-e3f1-4838-909a-3c601a272e9d}||LS27A70_(NVIDIA_High_Definition_Audio)||8, {0.0.0.00000000}.{ad8837d6-c905-4a72-adcb-7018ee7baab3}||CABLE_Input_(VB-Audio_Virtual_Cable)||9, {0.0.0.00000000}.{b6526f47-8c31-48d5-8b30-489196c56a6b}||Headphones_(iLoud_Micro-Monitor)||10, {0.0.0.00000000}.{d8576751-f212-4fa7-8cb5-0f825e64c87e}||CABLE-B_Input_(VB-Audio_Virtual_Cable_B)||11, {0.0.0.00000000}.{fddd6891-fc7b-4afa-8c26-9246692a19f0}||CABLE-A_In_16ch_(VB-Audio_Virtual_Cable_A)||12
Volume (Volume) op('lyria').par.Volume Float
Default:
1.0
Range:
0 to 1
Slider Range:
0 to 1
Clear Audio Buffers (Clearaudio) op('lyria').par.Clearaudio Pulse

Clears all generated audio from memory and from the output CHOPs (store_output and full_audio).

Default:
False
Output Settings Header
Save Last Audio (Savelastaudio) op('lyria').par.Savelastaudio Pulse

Manually save the contents of the current audio buffer to a file.

Default:
False
Base Filename (Basefilename) op('lyria').par.Basefilename Str

The base name for saved audio files. A timestamp will be appended to ensure uniqueness.

Default:
"" (Empty String)
Output Folder (Outputfolder) op('lyria').par.Outputfolder Folder

The folder where generated audio files will be saved, relative to the project file.

Default:
"" (Empty String)
Autosave on Stop (Autosaveonstop) op('lyria').par.Autosaveonstop Toggle

Automatically save the audio buffer to a file when the stream is stopped.

Default:
False
Clear on Save (Clearonsave) op('lyria').par.Clearonsave Toggle

Clear the audio buffer after saving. Disable to keep audio for multiple saves.

Default:
False

Changelog

Section titled “Changelog”
v1.0.02025-06-26

First release / finalized version

  • added playback parameter page that controls the playback of the audio.
  • added a new parameter Clearaudio to clear the audio buffers.
  • added auto updateprompts and a poll paramter for updates ever X second
  • added auto saving of the audio on stop stream + a few more configurable options . set on the playback page.
v0.2.02025-05-20

first release

Section titled “Related Operators”