Multimodal AI Engineer

Real-Time Multimodal AI Engineer (Digital Human Systems)

We are engaging a Multimodal AI Engineer for building a real-time AI digital human that combines LLMs, voice cloning (text-to-speech) and facial animation / talking avatar, which is led by my client – a global leading strategy led technology build consultancy.

The goal is not a demo, it's a low-latency, production-grade system where responses are generated live, speech sounds natural and personalised, lip movement is synchronised with audio and the entire interaction feels coherent and human,

What You'll Work On

You'll design and implement a real-time multimodal pipeline, connecting multiple AI systems into a single, synchronised experience, including:

• Build streaming pipelines (LLM speech avatar output)
• Orchestrate token/chunk-level data flow across models
• Integrate voice cloning / TTS systems (preferably phoneme-aware)
• Implement lip-sync pipelines (audio viseme animation)
• Handle audio–video synchronisation in real time
• Optimise latency across the full pipeline (<2 seconds target)
• Ensure stability under continuous interaction (not just single-turn demos)

What We're Actually Looking For

This is not a generic AI role, you should be comfortable working at the intersection of:

• real-time systems
• speech processing
• computer vision / animation
• applied AI integration

Must-Have Experience

• Building low-latency or real-time systems (streaming, async pipelines) in OSS
• Integrating multiple AI components (LLMs, TTS, CV models)
• Working with text-to-speech or voice cloning systems
• Experience with lip-sync / talking-head models (e.g. Wav2Lip, SadTalker or similar)
• Handling audio/video synchronisation or time-aligned data
• Strong Python skills (or equivalent for ML/system integration)

Nice-to-Have

• Experience with WebRTC or real-time media streaming
• Knowledge of phonemes / prosody / speech timing
• GPU inference optimisation (latency tuning)
• Exposure to multimodal models (audio + video + text)
• Experience building production-grade AI systems (not just notebooks)

If you are experienced in making multiple AI models behave like a single, real-time human interaction system, we'd like to hear from you. Please contact zxie@hiredgesolutions.com for next step and details.