Our desktop app captures meetings without a bot and streams them to the cloud. For months, the signaling motor was the hardest portion of the merchandise to make reliable. We'd hole 1 people of separator case, vessel it, and a caller 1 would aboveground the adjacent week. Different guidelines causes, aforesaid pattern.
The motor ran successful the render process of our Electron app. We tried the evident fixes: tighter lifecycle management, moving activity disconnected the main thread, isolating it from React's render cycle. Each alteration helped astatine the margin, but nary addressed the existent issue. A render process is the incorrect spot to do realtime audio and video capture. A seizure motor can't tolerate GC pauses, throttling, aliases immoderate of the different things a browser runtime does to enactment responsive.
So we went native: ScreenCaptureKit connected macOS, libobs connected Windows, and a shared Swift furniture tying it together.
Atomic: our Combine-to-Jotai bridge
Bridging a autochthonal runtime to React usually intends penning autochthonal addon bindings by hand. You serialize each worth that crosses the boundary, way events done stringly-typed names, and update 3 files whenever you adhd a property: the Swift class, the C++ binding, and the TypeScript wrapper. It works, but it's retired of sync the infinitesimal anyone forgets a step.
What if each @Published spot successful Swift automatically became a Jotai atom successful React? Fully reactive, type-safe, nary glue code. That's what our soul instrumentality Atomic does.
@NodeExport public last people AudioPlayer { @Published nationalist var isPlaying: Bool = false @Published nationalist var volume: Float = 1.0 nationalist func play() { isPlaying = existent } nationalist func pause() { isPlaying = mendacious } } #AtomicExport(AudioPlayer.self) const subordinate = caller AudioPlayer(); const volumeAtom = atomWithNativeState<number>(player.volume); store.set(volumeAtom, 0.5); // Flows into Swift. player.play(); // Updates travel backmost into React.From React's perspective, these atoms are indistinguishable from immoderate different Jotai atom. The truth that the information lives successful a Swift runtime connected a different thread is invisible.
The @NodeExport macro generates the full span astatine compile time. Types representation automatically (Int → number, String? → drawstring | null). Value changes successful Swift schedule callbacks connected Node's arena loop. Every caller spot we adhd connected the Swift broadside is instantly disposable successful React. And because Atomic is built connected Swift, not Apple frameworks (we usage OpenCombine connected Windows), the aforesaid span runs connected some platforms.
Two seizure engines, 1 interface
On macOS, ScreenCaptureKit gives america hardware-accelerated seizure and the autochthonal contented picker. On Windows, we usage libobs done a Swift wrapper we telephone OBSKit. The 2 engines person fundamentally different architectures.
On macOS, we person earthy sample buffers from 3 independent sources and combine the record ourselves. On Windows, capture, mixing, encoding, and muxing tally arsenic a azygous graph. We configure it and a record show streams recently written bytes to our upload session.
Windows seizure has its ain challenges. We usage Windows Graphics Capture (WGC) arsenic the superior method, and if it doesn't present frames successful time, we autumn backmost to BitBlt. We besides observe all-black frames (common pinch immoderate emulated windows aliases games) and move methods mid-recording.
When clocks disagree
This is wherever the macOS motor earns its complexity. Three seizure sources, 3 hardware clocks, 3 different ideas of what clip it is.
Both audio sources get timestamped and converted to a world framework index. The mixer drains some queues successful lockstep, only producing output erstwhile some person capable data. If 1 root stalls (muted mic, stiff virtual device), the mixer detects it aft 500ms and switches to single-source mode until it resumes.
Then there's a subtler problem: audio drivers that dishonesty astir their sample rate. Some virtual drivers study 48kHz but present buffers astatine 44.1kHz. Over a 30-minute meeting, this drift becomes audible. Our hole is confidence-based correction: we measurement existent buffer cadence, and if it consistently disagrees pinch the reported format crossed 3 consecutive buffers, we reinterpret the watercourse astatine the correct complaint pinch a crossfade to debar clicks.
On Windows, astir of this complexity is abstracted distant by the seizure engine's soul mixer. The tradeoff is control: connected macOS we observe and hole separator cases for illustration lying drivers ourselves; connected Windows we waste and acquisition that granularity for simplicity.
Recordings that past crashes
A regular MP4 writes its metadata astatine the extremity of the file. Crash earlier that, and the signaling is gone. On some platforms, we usage fragmented MP4 instead.
Each conception is self-contained. A clang astatine infinitesimal 30 loses astatine astir the past second. Segments spell to some section retention and the unreality simultaneously. If the web drops, segments persist locally and the upload resumes automatically erstwhile connectivity returns.
Desktop signaling utilized to beryllium 1 of our astir communal sources of support tickets. Now it's a boring portion of the app that conscionable works. The full rewrite shipped successful 2 months, and Atomic is why: erstwhile the span existed, adding a characteristic meant penning Swift and watching the UI update successful existent time.
Not everything belongs successful a render process. Sometimes you request to spell native.
If taking connected problems for illustration this sound absorbing to you, see joining us.
English (US) · 
Indonesian (ID) · 
