Rime Text-to-Speech

TTS with RimeTTSService (WebSocket streaming with word timing and interruptions) and RimeHttpTTSService for batch synthesis.

Overview

Rime AI provides two TTS service implementations: RimeTTSService (WebSocket-based) with word-level timing and interruption support, and RimeHttpTTSService (HTTP-based) for simpler use cases. RimeTTSService is recommended for real-time interactive applications.

Rime TTS API Reference

Pipecat’s API methods for Rime TTS integration

Example Implementation

Complete example with word timestamps

Rime Documentation

Official Rime WebSocket and HTTP API documentation

Voice Models

Explore available voice models and features

Installation

To use Rime services, install the required dependencies:

uv add "pipecat-ai[rime]"

Prerequisites

Rime Account Setup

Before using Rime TTS services, you need:

1. Rime Account: Sign up at Rime AI
2. API Key: Generate an API key from your account dashboard
3. Voice Selection: Choose from available voice models

Required Environment Variables

• RIME_API_KEY: Your Rime API key for authentication

Configuration

RimeTTSService

api_key · str · required

Rime API key for authentication.

voice_id · str · required · deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(voice=...) instead.

url · str · default: "wss://users-ws.rime.ai/ws3"

Rime WebSocket API endpoint.

model · str · default: "coda" · deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(model=...) instead.

sample_rate · int · default: "None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

text_aggregation_mode · TextAggregationMode · default: "TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries, producing more natural speech. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

aggregate_sentences · bool · default: "None"

Deprecated in v0.0.104. Use text_aggregation_mode instead.

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(...) instead.

settings · RimeTTSService.Settings · default: "None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeHttpTTSService

api_key · str · required

Rime API key for authentication.

voice_id · str · required · deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(voice=...) instead.

aiohttp_session · aiohttp.ClientSession · required

An aiohttp session for HTTP requests.

model · str · default: "coda" · deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(model=...) instead.

sample_rate · int · default: "None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(...) instead.

settings · RimeHttpTTSService.Settings · default: "None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeNonJsonTTSService

A deprecated non-JSON WebSocket service that uses plain-text messages and raw audio responses. Use RimeTTSService for current integrations.

api_key · str · required

Rime API key for authentication.

voice_id · str · required · deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(voice=...) instead.

url · str · default: "wss://users.rime.ai/ws"

Rime WebSocket API endpoint.

model · str · default: "coda" · deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(model=...) instead.

audio_format · str · default: "pcm"

Audio output format.

sample_rate · int · default: "None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

text_aggregation_mode · TextAggregationMode · default: "TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

aggregate_sentences · bool · default: "None" · deprecated

Deprecated in v0.0.104. Use text_aggregation_mode instead.

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(...) instead.

settings · RimeNonJsonTTSService.Settings · default: "None"

Runtime-configurable settings. See RimeNonJsonTTSService Settings below.

RimeTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	None	Model identifier. (Inherited.)
voice	str	None	Voice identifier. (Inherited.)
language	Language | str	None	Language for synthesis. (Inherited.)
segment	str	NOT_GIVEN	Segment type for synthesis.
speedAlpha	float	NOT_GIVEN	Speed alpha parameter.
reduceLatency	bool	NOT_GIVEN	Whether to reduce latency.
pauseBetweenBrackets	bool	NOT_GIVEN	Pause between brackets.
phonemizeBetweenBrackets	bool	NOT_GIVEN	Phonemize between brackets.
noTextNormalization	bool	NOT_GIVEN	Disable text normalization.
saveOovs	bool	NOT_GIVEN	Save out-of-vocabulary words.
inlineSpeedAlpha	str	NOT_GIVEN	Inline speed alpha.
repetition_penalty	float	NOT_GIVEN	Token repetition penalty (coda only, 1.0-2.0).
temperature	float	NOT_GIVEN	Sampling temperature (coda only, 0.0-1.0).
top_p	float	NOT_GIVEN	Cumulative probability threshold (coda only, 0.0-1.0).
timeScaleFactor	float	NOT_GIVEN	Audio playback speed factor (coda only). Values above 1.0 slow down the audio; values below 1.0 speed it up.

RimeNonJsonTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeNonJsonTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	None	Model identifier. (Inherited.)
voice	str	None	Voice identifier. (Inherited.)
language	Language | str	None	Language for synthesis. (Inherited.)
segment	str	NOT_GIVEN	Segment type for synthesis.
repetition_penalty	float	NOT_GIVEN	Token repetition penalty.
temperature	float	NOT_GIVEN	Sampling temperature.
top_p	float	NOT_GIVEN	Cumulative probability threshold.

Usage

Basic Setup (WebSocket)

from pipecat.services.rime.tts import RimeTTSService

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        model="coda",
        voice="luna",
    ),
)

With Customization (WebSocket)

from pipecat.services.rime.tts import RimeTTSService
from pipecat.transcriptions.language import Language

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        voice="cove",
        model="mistv2",
        language=Language.ES,
        speedAlpha=1.2,
        reduceLatency=True,
    ),
)

HTTP Service

import aiohttp
from pipecat.services.rime.tts import RimeHttpTTSService

async with aiohttp.ClientSession() as session:
    tts = RimeHttpTTSService(
        api_key=os.getenv("RIME_API_KEY"),
        settings=RimeHttpTTSService.Settings(
            model="coda",
            voice="luna",
        ),
        aiohttp_session=session,
    )

Non-JSON WebSocket

from pipecat.services.rime.tts import RimeNonJsonTTSService

tts = RimeNonJsonTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeNonJsonTTSService.Settings(
        model="coda",
        voice="luna",
    ),
)

Customizing Speech

RimeTTSService provides a set of helper methods for implementing Rime-specific customizations, meant to be used as part of text transformers. These include methods for spelling out text, adjusting speech rate, and modifying pitch. See the Text Transformers for TTS section in the Text-to-Speech guide for usage examples.

RimeTTSService.SPELL(text: str) -> str

Static method that implements Rime’s spell function to spell out text character by character. Call this method on the class, not on an instance.

# Text transformers for TTS
# This will insert Rime's spell tags around the provided text.
async def spell_out_text(text: str, type: str) -> str:
    return RimeTTSService.SPELL(text)

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("phone_number", spell_out_text),
    ],
)

RimeTTSService.PAUSE_TAG(seconds: float) -> str

Static method that implements Rime’s custom pause functionality to generate a properly formatted pause tag you can insert into the text. Call this method on the class, not on an instance.

# Text transformers for TTS
# This will insert a one second pause after questions.
async def pause_after_questions(text: str, type: str) -> str:
    if text.endswith("?"):
        return f"{text}{RimeTTSService.PAUSE_TAG(1.0)}"
    return text

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("sentence", pause_after_questions), # Only apply to sentence aggregations
    ],
)

PRONOUNCE(text: str, word: str, phoneme: str) -> str

Instance method that implements Rime’s custom pronunciations feature. It replaces word in text with the phoneme tag. Unlike SPELL and PAUSE_TAG, call it on the service instance, not on the class. Its three arguments do not match the (text, aggregation_type) transform shape, so wrap it in a function.

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        # Keep bracket phonemization on for the whole session
        phonemizeBetweenBrackets=True,
    ),
)

# Text transformers for TTS
# This puts a phoneme in place of the word "potato" to define how it
# should be pronounced.
async def maybe_say_potato_all_fancylike(text: str, type: str) -> str:
    if using_fancy_voice:
        return tts.PRONOUNCE(text, "potato", "potato")
    else:
        return tts.PRONOUNCE(text, "potato", "poteto")

tts.add_text_transformer(maybe_say_potato_all_fancylike, "*")  # Apply to all text

PRONOUNCE turns on bracket phonemization for the next message only. If any message may contain a phoneme, set phonemizeBetweenBrackets=True in Settings (as above) so it stays on for the whole session.

PRONOUNCE handles one word at a time. For hundreds of overrides, use the replace_text transform. See Bulk pronunciation overrides.

INLINE_SPEED(text: str, speed: float) -> str

Instance method that implements Rime’s inline speed adjustment feature. It wraps text in [] and adds speed to the inlineSpeedAlpha request field. Like PRONOUNCE, call it on the service instance, not on the class.

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
)

# Text transformers for TTS
# This will make the word slow always be spoken more slowly.
async def slow_down_slow_words(text: str, type: str) -> str:
    return text.replace(
        "slow",
        tts.INLINE_SPEED("slow", speed=0.5)
    )

tts.add_text_transformer(slow_down_slow_words, "*")  # Apply to all text

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Notes

• Word-level timestamps: RimeTTSService provides word-level timing information, enabling synchronized text highlighting.
• WebSocket vs HTTP: The WebSocket service supports word-level timestamps, interruption handling, and maintains context across messages within a turn. The HTTP service is simpler but lacks these features.
• Non-JSON WebSocket: RimeNonJsonTTSService is the deprecated plain-text and raw-audio WebSocket implementation. It does not support word-level timestamps.

Event Handlers

Rime WebSocket TTS services support the standard service connection events:

Event	Description
on_connected	Connected to Rime WebSocket
on_disconnected	Disconnected from Rime WebSocket
on_connection_error	WebSocket connection error occurred

@tts.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Rime")