NUC AI Lab
Once we get used to cloud models, we want AI to handle more of our work. As workflows grow and calls and token usage increase, I want to explore what can stay on a compact desktop NUC and what is still worth sending to the cloud. This lab starts with real tasks, exploring local applications and ways to connect them with cloud models.
Three experiment tracks
These are directions to test. The right choice depends on the model, NUC configuration, and task requirements. Each example still needs validation.
Start with repeatable tasks such as document classification, field extraction, and speech preprocessing.
To test: can the chosen setup meet the quality threshold with acceptable latency and memory use? For offline use, does any part of the workflow still depend on a network?
Use complex reasoning, synthesis across documents, and cases that fail locally as candidates for the cloud comparison group.
To test: does the cloud actually improve the result? What do the extra input and output tokens, calls, and wait time achieve?
Classify, retrieve, or prepare information on the NUC, then send necessary context to the cloud. Explore how to identify tasks that need escalation.
To test: does routing miss difficult cases? Does compression lose useful information? After retries and human revision, does the full workflow still help?
Which repeatable, bounded tasks can a NUC complete on its own?
Which difficult tasks are still worth sending to a cloud model?
Can local processing reduce what needs to be sent to the cloud?
The same task and quality threshold, then usage and cost.
Starting with these projects
Instead of typing out every thought, let the NUC transcribe Chinese speech locally, then send the text to the selected local or cloud model. A Python / Electron prototype connects the stages; the next experiment compares spoken and typed input in practice.
Next, measure time to a usable prompt, recognition corrections, and model response time. Claude receives text in the cloud, and edge-tts also needs a network. Time savings, token savings, and stability have not been established.
Explore the voice prototype and tradeoffsA friend’s Laya Snake project prompted a workflow selector idea: which requests could stay locally, and which need the cloud?
Jev is a cloud API; Laya offers a local research entry point with a compatible interface. This source-linked idea has not yet been tested by rerunning the project or evaluating routing performance.
Read the routing idea and open questionsSet up cloud, local, and hybrid workflows. Check whether results are usable before measuring tokens, calls, end-to-end latency, NUC resources, and human revision.
The method and recording fields are drafted; model choices, hardware details, and measurements remain open. Local processing is not assumed cheaper: hardware, electricity, and maintenance count too.
Explore the three-workflow comparisonI want to document more than whether a model runs: a conditional account of what belongs on a NUC, what belongs in the cloud, and when a hybrid approach helps. Each experiment will state its setup, quality threshold, and failure cases. Savings and impact claims will follow data.
Experiment ideas
Some ideas are being planned; others have a small prototype. Each article shows its progress, and I would like to hear what interests you.
Progress changes with the work and is separate from topic tags.
Some tasks only need a next step. From Jev and a friend’s local Laya Snake project to model routing and trading research, I want to explore where decision models fit. These are hypotheses and a test plan.
Tell me on Instagram which experiment interests you.
Video notes
I'll add walkthroughs and test videos here, alongside written summaries and setup details. The first video is still in preparation.