VTuberWebcam

VTuber WebCam is an app for VTubers. The app was designed for streaming and video recording by 3D VTubers.

Dashboard:

Open Dashboard

The app must be running in order to open the dashboard.

Hotkeys:

• CTRL + Q - exit an app
• R - reset head position and rotation

Supported cameras for tracking:

• Generic UVC cameras (V4L2 and OpenCV compatible)
• Intel RealSense (RGB+depth tracking)
• PS5 Camera (firmware loader, software stereo based depth calculation)

Features:

• Plugin based and Python-scriptable
• 3D model import from VRM files
• Face tracking powered by MediaPipe
• Morph target (blend shape) based emotes
  
• Hands tracking using Leap Motion device
• Spring bones physics
• Streaming to virtual V4L2 device
• XShm-based desktop capture
• Gamepad (DualShock4) tracking using it's own gyroscope and accelerometer
• DonationAlerts integration for triggering custom events on 3D scene or avatar
• Lipsync using Synthesizer V
• Subtitles using VOSK speech recognition API
• VoiceVox integration

Planned Features:

• Streamlabs integration
• Camera based FBT (full body tracking)

Software Used:

• Panda3D game engine
• MediaPipe tracking library by GoogleAI
• LeapSDK v2