On-device multimodal AI for React Native

Ship native mobile AI experiences with streaming results.

Amaryllis is the base inference module for local text, image, and streaming workflows. A companion module, working name amaryllis-components, extends that foundation into governed adaptive components.

Use on-device inference Explore amaryllis-components

Install

npm install react-native-amaryllis
# or
yarn add react-native-amaryllis
# or
pnpm add react-native-amaryllis

Android iOS Streaming Multimodal Components RFC

What the base module provides

Native inference, streaming hooks, multimodal requests, and context APIs provide the substrate for higher-level adaptive UI modules.

Native LLM engine

Optimized pipelines for Android and iOS with predictable startup.

Multimodal support

Send prompts with images for grounded, visual responses.

Streaming hooks

Use hooks and observables to render partial tokens fast.

Context provider

Centralize configuration for the entire React Native app.

LoRA customization

Fine tune behavior with LoRA adapters on GPU devices.

Developer control

Cancel and manage sessions with explicit APIs.

Best practices for production apps

Keep performance, memory, and safety predictable.

Stream by default

Show partial tokens early for a responsive UX.

Cancel on unmount

Always cancel async inference in cleanup handlers.

Constrain inputs

Limit image count and size to protect memory.

Validate file paths

Prevent invalid or unsafe native file access.

Handle errors explicitly

Surface custom error types and graceful fallbacks.

Document model sources

Track model versions and update strategies.

Companion module RFC

Components for adaptive on-device UI

The proposed companion module adds governed component specs, build-time generation workflows, and safe runtime personalization on top of the base Amaryllis inference layer.

Proposed package

amaryllis-components

Keep react-native-amaryllis focused on on-device multimodal inference. Put adaptive component specs, registries, validators, and personalization contracts in a companion package.

Mode 1

Scaffold

Build-time AI creates component source from a ComponentSpec, then the result goes through validation, review, and publishing.

Mode 2

Customize

AI creates bounded variants within declared slots, layouts, copy rules, and design-token constraints.

Mode 3

Runtime personalization

On-device AI returns schema-validated props, variants, slot text, or JSON patches. It does not emit executable component code.

Source of truth

ComponentSpec

A typed, versioned spec defines UI structure, behavior, allowed AI operations, target runtime, and policy requirements before code is generated.

Generation model

Validated pipeline

Build-time TSX generation moves through validation, static checks, preview, diff review, and artifact publishing instead of direct freeform AI code edits.

On-device AI

Structured customization

Runtime AI can fill slots, choose variants, and select design tokens, but it must return validated data, not executable TSX, JSX, JavaScript, imports, or raw markup.

Governance

Policy enforcement

Import allowlists, accessibility checks, design token rules, network restrictions, provenance metadata, and approval gates keep generated output reviewable.

Spec Validation Build-time code Runtime data Static checks Preview Publish

Quickstart

Wrap your app, then generate from hooks.

Provider setup

import { LLMProvider } from 'react-native-amaryllis';

<LLMProvider
  config={{
    modelPath: 'gemma3-1b-it-int4.task',
    visionEncoderPath: 'mobilenet_v3_small.tflite',
    visionAdapterPath: 'mobilenet_v3_small.tflite',
    maxTopK: 32,
    maxNumImages: 2,
    maxTokens: 512,
  }}
>
  {/* App components */}
</LLMProvider>

Streaming inference

import { useInferenceAsync } from 'react-native-amaryllis';

const generate = useInferenceAsync({
  onResult: (chunk, isFinal) => {
    // Update UI with tokens
  },
  onError: (err) => setError(err),
});

await generate({ prompt, images });

Live demo preview

Sample UI built on top of Amaryllis streaming hooks.