Build a local personal assistant with Gemma and Ollama

You want a customizable alternative to a general-purpose assistant: your preferred style, a model on your computer and access from your phone. Build the useful capabilities one at a time. A local model is a starting point, not a reproduction of a product such as Grok and all its services, data and tools.

Choose the first useful capability

Begin with text conversation for summarizing, planning or rewriting. Write five representative requests and define success: correct language, requested format, no unsupported claims and acceptable latency.

Then add either document retrieval or current public-information search. Introducing vision, search, memory and actions together makes it difficult to identify which component caused an error.

Try a Gemma model locally

As of September 27, 2026, the official Ollama catalog includes gemma4:12b. Check its current availability and variant before installing. Download size is not total runtime memory: context and execution add their own requirements.

ollama pull gemma4:12b
ollama run gemma4:12b

Run your five requests on your hardware. If the model does not fit or is too slow, choose a more suitable catalog variant. Record the exact model and parameters so that comparisons are repeatable.

Add mobile access

The Ollama phone tutorial connects a Pacerelle question to the local model. Set OLLAMA_MODEL to your chosen variant, start the agent and verify a reply over mobile data. That initial example handles questions independently.

Conversation memory requires your integration to retain relevant exchanges, isolate conversations and define retention. A personality prompt is not memory. Explain what is stored and how to remove it.

Separate chat, search and documents

For recent information, add a search tool and request links to consulted sources. Asking a model to “verify” does not itself supply current evidence. Search queries leave your computer for the selected provider, so avoid including private information without an explicit decision.

For personal files, use local RAG with citations and access controls. Image support requires both a compatible model and code that actually forwards image inputs. An attachment button does not prove that the model analyzes images.

Account for real costs and data flows

Local inference avoids per-request billing for that computation, but hardware, electricity and maintenance still cost money. Search services and cloud models may add charges. Pacerelle beta terms are separate from those of your other tools.

Inspect every stage: local or hosted inference, external search, logs, history and backups. The encryption guide explains the path Pacerelle protects. Your first assistant is ready when it passes your five practical tasks, acknowledges limitations and provides evidence for sourced claims.