oxspeak_server/.junie/plans/replace-udp-with-rustrtc-2.md

sessionId

sessionId
session-260912-132834-fuse

Requirements

Overview & Goals

Replace the current placeholder raw-UDP voice relay (src/udp/*) with a real WebRTC media stack built on the rustrtc crate (already declared in Cargo.toml). The server becomes a centralized SFU (Selective Forwarding Unit): each connected client opens one PeerConnection with the server, the server decrypts/receives that client's audio (and later video) track and forwards it to every other member of the same voice Channel.

Scope

In Scope

• Remove src/udp/server.rs, src/udp/router.rs raw-socket logic (metrics module is kept/adapted).
• Add a voice module hosting: PeerConnection lifecycle management via rustrtc, SDP offer/answer exchange, ICE candidate exchange, and SFU-style track forwarding scoped by channel_id.
• Reuse the existing WebSocket gateway (GatewayManager / RealtimeRouter / on_message in src/routes/gateway/mod.rs) as the signaling transport — add a Voice namespace to the existing JSON event envelope (GatewayEvent { namespace, action, content }) instead of introducing a new endpoint.
• Reuse channel::ChannelType::Voice, channel_user, and the JOIN_VOICE / SPEAK permission bits already defined in src/permissions.rs to authorize who may join/publish in a voice channel.
• Update config.toml / NetworkConfig so the single udp_port continues to be the one UDP port used, bound through rustrtc's ice_udp_mux (single-port ICE) instead of a raw UdpSocket.
• Update AppMetrics/core::mod::App::run wiring so the new voice service starts/stops alongside the HTTP server, replacing UdpServer.

Out of Scope

• TURN/STUN server configuration (kept minimal for LAN/local network scenarios for now).
• Video/screen-share specific handling (structure will allow it later, but only audio forwarding is implemented now).
• Client-side (frontend) implementation details beyond the signaling contract needed by the server.

User Stories

• As a user with JOIN_VOICE permission on a voice channel, I want to connect and have my microphone audio heard by other members of that same channel.
• As a user already in a voice channel, I want to hear every other member's audio forwarded by the server without opening a direct connection to each peer.
• As a server operator, I want the previous ad-hoc UDP relay removed and replaced by a maintained WebRTC stack (rustrtc) so voice traffic is encrypted (SRTP) and NAT-traversal (ICE) works out of the box.

Functional Requirements

• A client sends an SDP offer through the gateway (namespace: "Voice"), scoped to a channel_id; the server validates channel membership/permissions, creates a rustrtc::PeerConnection, and returns an SDP answer through the same gateway connection.
• ICE candidates generated by either side are exchanged as additional Voice gateway events.
• Once connected, the server subscribes the client's inbound audio track to every other PeerConnection currently joined to the same voice channel (SFU fan-out), and adds new joiners' tracks to previously-connected peers as they arrive.
• When a user disconnects (gateway socket closes) or leaves the voice channel, their PeerConnection is closed and their track is removed from all other peers' forwarding sets.
• Existing UdpMetrics-style counters (packets/bytes in/out, errors) are preserved in spirit, adapted for voice metrics and reported by the periodic reporter.

Technical Design

Current Implementation

• src/udp/server.rs: binds a raw tokio::net::UdpSocket on NetworkConfig.udp_port, loops on recv_from, and blindly forwards datagrams to registered peers under a hard-coded "default" channel without protocol isolation or encryption.
• src/udp/router.rs: RoutingTable maps ChannelId -> Vec<SocketAddr>; dead placeholder API.
• src/udp/metrics.rs: atomic counters (packets_received, bytes_sent, etc.) + spawn_reporter logging every interval, plugged into AppMetrics.udp (src/metrics/mod.rs).
• src/core/mod.rs: App::build/App::run constructs UdpServer::new(&config.network, udp_metrics) and tokio::spawn(udp_server.run()), joined into shutdown-broadcast pattern.
• src/routes/gateway/mod.rs: GatewayManager tracks client WebSocket connections; GatewayClient::on_message handles incoming messages and acts as the extension point for voice signaling.
• src/services/realtime_registry.rs: maintains channel_id -> HashSet<user_id> membership computed from computed_permission (READ_CHANNEL).
• src/permissions.rs: JOIN_VOICE, SPEAK, STREAM, MOVE_OTHERS, DISCONNECT_OTHERS, MANAGE_VOICE_CHANNEL bits already defined.
• src/models/channel.rs: ChannelType::Voice variant already exists.

Key Decisions

1. Topology: centralized SFU — one rustrtc::PeerConnection per connected client per voice channel; the server forwards each publisher's RTP track to every other subscriber in that channel. No direct peer-to-peer mesh.
2. Signaling transport: existing WebSocket gateway — SDP offer/answer and ICE candidates are carried as Voice namespace events inside the current GatewayEvent envelope, handled in GatewayClient::on_message and dispatched to VoiceService.
3. Single UDP port reuse — rustrtc's RtcConfiguration is set up with ice_udp_mux = true and bound to NetworkConfig.udp_port, preserving existing firewall/network port configurations.
4. No TURN/STUN for now — RtcConfiguration.ice_servers left default/empty for local and direct network setups.
5. Voice presence tracked separately from RealtimeRegistry — an in-memory registry (VoiceRoom per channel_id) tracks active WebRTC connections independently from general WebSocket channel presence.

Proposed Changes

• Replace src/udp module with src/voice:
  • voice/mod.rs: public API, exports VoiceService.
  • voice/service.rs: VoiceService owns rustrtc::RtcConfiguration, a map channel_id -> VoiceRoom, and handler methods for offer, ICE candidates, and channel leaves.
  • voice/room.rs: VoiceRoom holds HashMap<user_id, Arc<PeerConnection>> for a channel and orchestrates track fan-out across peers.
  • voice/metrics.rs: tracks voice counters (packets/bytes/errors), exposed through AppMetrics.voice and the existing reporter cadence.
• Extend GatewayEvent handling: add "Voice" namespace with actions offer, answer, ice-candidate, leave.
• Permission check on offer: verify ChannelPermission::JOIN_VOICE and SPEAK before initializing a PeerConnection.
• Update src/core/mod.rs: replace UdpServer initialization with VoiceService held inside AppState.
• Update src/config.rs: keep NetworkConfig.udp_port feeding rustrtc's ice_udp_mux_port.

Data Models / Contracts

// Gateway JSON contract additions (namespace = "Voice")
{ "namespace": "Voice", "action": "offer", "content": { "channel_id": Uuid, "sdp": String } }
{ "namespace": "Voice", "action": "answer", "content": { "channel_id": Uuid, "sdp": String } }
{ "namespace": "Voice", "action": "ice-candidate", "content": { "channel_id": Uuid, "candidate": String } }
{ "namespace": "Voice", "action": "leave", "content": { "channel_id": Uuid } }

pub struct VoiceService {
    config: rustrtc::RtcConfiguration,
    rooms: RwLock<HashMap<Uuid, VoiceRoom>>,
}

impl VoiceService {
    pub async fn handle_offer(&self, user_id: Uuid, channel_id: Uuid, sdp: String) -> anyhow::Result<String>;
    pub async fn handle_ice_candidate(&self, user_id: Uuid, channel_id: Uuid, candidate: String) -> anyhow::Result<()>;
    pub fn leave(&self, user_id: Uuid, channel_id: Uuid);
}

Architecture Diagram

graph TD
    ClientA[Client A] -- WebSocket: Voice offer/answer/ICE --> Gateway[GatewayManager / on_message]
    ClientB[Client B] -- WebSocket: Voice offer/answer/ICE --> Gateway
    Gateway -- dispatch --> VoiceService
    VoiceService -- creates/owns --> RoomA[VoiceRoom per channel_id]
    RoomA -- PeerConnection A --> PCA[rustrtc PeerConnection A]
    RoomA -- PeerConnection B --> PCB[rustrtc PeerConnection B]
    ClientA == RTP/SRTP media on UDP mux port ==> PCA
    ClientB == RTP/SRTP media on UDP mux port ==> PCB
    PCA -- forward audio track --> PCB
    PCB -- forward audio track --> PCA

Risks

• rustrtc API details: verify track forwarding primitives against rustrtc 0.3.133 during implementation and encapsulate track re-subscription in voice/room.rs.
• Permission enforcement: ensure permission checks take place before creating peer connections to prevent unauthorized audio relay.
• Signaling message volume: ensure gateway handling remains non-blocking for high frequency ICE candidates.

Delivery Steps

Step 1: Scaffold the voice module and RtcConfiguration from NetworkConfig

A new src/voice module exists with a VoiceService capable of building a rustrtc::RtcConfiguration from the app's network config, replacing the old src/udp module's role in core/mod.rs.

• Create src/voice/mod.rs, src/voice/service.rs, src/voice/room.rs, and src/voice/metrics.rs.
• Port UdpMetrics-style counters into voice/metrics.rs (packets/bytes in/out, errors), keeping the Metrics/MetricsSnapshot trait implementations used by crate::metrics::reporter.
• Build RtcConfiguration/RtcConfigurationBuilder in VoiceService::new(&NetworkConfig, metrics) using ice_udp_mux = true and ice_udp_mux_port = network.udp_port, with no external ICE servers configured.
• Remove src/udp/server.rs and src/udp/router.rs; delete unused RoutingTable.
• Update src/core/mod.rs (App::build/App::run) to construct VoiceService instead of UdpServer, store it on AppState, and drop unneeded udp_shutdown_tx/udp_handle tasks.
• Update src/metrics/mod.rs (AppMetrics) to reference the new voice metrics type.

Step 2: Implement per-channel PeerConnection lifecycle in VoiceRoom

VoiceService/VoiceRoom can accept an SDP offer for a given channel, create a rustrtc::PeerConnection, and return an SDP answer, tracking connections per channel_id.

• Implement VoiceRoom (channel_id -> HashMap<user_id, Arc<PeerConnection>>).
• Implement VoiceService::handle_offer(user_id, channel_id, sdp): looks up or creates the room, creates a PeerConnection via rustrtc, invokes set_remote_description/create_answer/set_local_description, and returns the answer SDP.
• Implement VoiceService::handle_ice_candidate and VoiceService::leave, closing and removing the PeerConnection from its room.
• Enforce ChannelPermission::JOIN_VOICE (and SPEAK for publishing) before creating a PeerConnection, reusing the computed_permission lookup pattern from src/services/realtime_registry.rs.

Step 3: Wire SFU track forwarding between peers in the same channel

Audio published by one connected client in a voice channel is forwarded by the server to every other client connected to the same channel.

• On PeerConnection::on_track for a given user's connection, register the inbound track on the owning VoiceRoom.
• For every other PeerConnection already in that VoiceRoom, add and forward the new track (SFU fan-out) using rustrtc's track/MediaCapabilities APIs.
• When a new peer joins an existing room, subscribe it to all tracks already being forwarded by other room members.
• On leave or disconnect, remove the peer's published track from all other peers' forwarding sets and close its PeerConnection.

Step 4: Expose Voice signaling through the existing WebSocket gateway

Clients can perform the full offer/answer/ICE handshake over the existing gateway WebSocket connection using a new Voice namespace, with no new HTTP/WS endpoint introduced.

• Extend GatewayClient::on_message in src/routes/gateway/mod.rs to parse GatewayEvent{namespace: "Voice", action, content} messages (offer, answer, ice-candidate, leave).
• Dispatch parsed messages to VoiceService (accessed via AppState) and send the resulting answer/ICE-candidate events back through the client's existing mpsc::UnboundedSender<Message>.
• On gateway disconnect (GatewayClient::on_disconnect), call VoiceService::leave for any channel the user was actively connected to in voice.
• Update config.toml's DEFAULT_CONFIG_TOML comment for udp_port to reflect its new role as the rustrtc ICE/media mux port.