API

Embed

Add an embed block to a string attribute in your schema and Layer computes that attribute’s vector on every write. Put ["Embed", "..."] in rank_by and Layer computes the query vector with the same model. Your application never loads a model or sends a vector of its own.

Example

This write creates the articles namespace with an embedded text attribute and upserts two rows. Layer embeds each row’s text with MiniLM, one of the CPU models, before storing it.

await client.write_namespace("articles", {
    "upsert_rows": [
        {"id": "planet-1", "title": "Planet",
         "text": "Jupiter is the biggest planet in the Solar System."},
        {"id": "photo-1", "title": "Photosynthesis",
         "text": "Plants turn sunlight, water, and carbon dioxide into food."},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "text": {
            "type": "string",
            "embed": {
                "model": "sentence-transformers/all-MiniLM-L6-v2",
                "dims": 384,
                "serving": {"prefer": "local"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "articles", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "planet-1", "title": "Planet",
            "text": "Jupiter is the biggest planet in the Solar System."},
        {"id": "photo-1", "title": "Photosynthesis",
            "text": "Plants turn sunlight, water, and carbon dioxide into food."},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "text": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":   "sentence-transformers/all-MiniLM-L6-v2",
                "dims":    384,
                "serving": map[string]any{"prefer": "local"},
            },
        },
    },
})

await client.writeNamespace("articles", {
  upsert_rows: [
    { id: "planet-1", title: "Planet",
      text: "Jupiter is the biggest planet in the Solar System." },
    { id: "photo-1", title: "Photosynthesis",
      text: "Plants turn sunlight, water, and carbon dioxide into food." },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      embed: {
        model: "sentence-transformers/all-MiniLM-L6-v2",
        dims: 384,
        serving: { prefer: "local" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "planet-1", "title": "Planet",
       "text": "Jupiter is the biggest planet in the Solar System."},
      {"id": "photo-1", "title": "Photosynthesis",
       "text": "Plants turn sunlight, water, and carbon dioxide into food."}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "sentence-transformers/all-MiniLM-L6-v2",
          "dims": 384,
          "serving": { "prefer": "local" }
        }
      }
    }
  }'

Query by meaning with Embed. Layer reads the model from the text schema and embeds the query string with it:

response = await client.query_namespace("articles", {
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"],
})
print(response.rows, response.performance)

response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{
    RankBy:            []any{"text", "ANN", []any{"Embed", "largest planet in the solar system"}},
    TopK:              3,
    IncludeAttributes: []string{"title", "text"},
})

const response = await client.queryNamespace("articles", {
  rank_by: ["text", "ANN", ["Embed", "largest planet in the solar system"]],
  top_k: 3,
  include_attributes: ["title", "text"],
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"]
  }'

{
  "rows": [
    { "id": "planet-1", "$dist": 0.137, "title": "Planet",
      "text": "Jupiter is the biggest planet in the Solar System." }
  ],
  "performance": {
    "embedding_tokens": 7,
    "embedding_ms": 9
  }
}

Where the vector comes from

embed.serving.prefer picks the provider that runs the model:

`embed.serving.prefer`	Who computes the vector
`local`	The locally configured embedder: the `layer-embed` CPU service the gateway reaches at `LAYER_EMBED_URL`, or an in-process Lattice or CLIP artifact. No key, no GPU, no account.
`worker`	A GPU worker in your own cluster, serving an open-weights checkpoint you declared.
`turbopuffer`	Turbopuffer’s hosted embedding service, from its managed model menu.

The provider is independent of the store the vector is written to, so any store that accepts embed can use any configured provider.

Layer never substitutes one provider for another. A profile that selects a provider the deployment has not configured fails validation at write time, and a provider that fails at request time returns an error instead of falling back to a different model.

native is accepted as an alias for turbopuffer, autoscaler for worker, and lattice for local on a Lattice profile.

All three providers are available. Layer validates a turbopuffer profile and forwards it unchanged to the upstream POST /v2/namespaces/{ns} and POST /v2/namespaces/{ns}/query, so Turbopuffer computes the vector and it never passes through Layer. local and worker compute the vector in Layer and write an ordinary ANN column.

Per-store resolution

Omitting serving selects turbopuffer (native). Only Turbopuffer can compute that vector itself. On a store that cannot embed, Layer computes the vector and the store receives an ordinary vector column. The rule depends on the store, never on the request:

Store	`turbopuffer` (the default)	`local`, `worker`
Turbopuffer	Native: forwarded unchanged, and Turbopuffer embeds.	Layer embeds and writes an ANN column.
hev search	Gateway-resolved through the configured Turbopuffer embedding provider.	Layer embeds.
Postgres	Gateway-resolved: the `local` embedder if it lists the model, otherwise the configured Turbopuffer embedding provider.	Layer embeds.

Layer resolves a gateway-resolved profile once, on the write that declares the attribute, and the namespace keeps that provider for good. If neither provider serves the model, the write returns 503 and nothing is stored.

CPU models

Layer serves three CPU embedding models, small enough to run on a laptop under Docker. They use the same schema as the larger GPU and hosted models, with lower retrieval quality. Community Edition Compose starts the text service next to the gateway, and the Helm chart runs it as a gateway sidecar; neither needs a key, GPU or download after the image pull, and the service has no route out of the machine.

Model	Dims	Modality	Use it for
`sentence-transformers/all-MiniLM-L6-v2`	384	text	Start here. 22M parameters, symmetric, no prefixes.
`BAAI/bge-small-en-v1.5`	384	text	Better retrieval for 50% more parameters. Try it when MiniLM’s results are weak.
`openai/clip-vit-base-patch32`	512	image	Text-to-image search. Both towers run on CPU.

The two text models are served by layer-embed, a separate CPU process whose image contains both sets of weights. The gateway reaches it over a private network at LAYER_EMBED_URL (for example http://embed:8081); nothing is downloaded at startup, and the gateway image carries no model weights. Set the variable and prefer: local resolves any model the service advertises. Leave it unset and a prefer: local write that names a text model returns 422. CLIP runs inside the gateway process from the directory named by LAYER_LOCAL_CLIP_MODEL_PATH; see Images.

There is no default model: a prefer: local profile always names one. Omit dims and Layer uses the dimension the service advertises for that model. Every model here pins a fixed artifact; revision, instructions, and chunk are not accepted on a local profile.

Query embedding has a 10 second budget and each write 60 seconds, shared by every batch of that write. Set LAYER_EMBED_QUERY_TIMEOUT_MS and LAYER_EMBED_WRITE_TIMEOUT_MS to change them. A service that is still loading returns 503 for local profiles only; native and GPU profiles are unaffected.

Text

The two text models are embedded differently, and Layer handles both; the example at the top of this page uses MiniLM. bge expects an instruction prefix on queries and MiniLM expects none, so the gateway adds the prefix exactly once, only when it embeds a query, and never on the text it stores. bge uses CLS pooling and MiniLM uses mean pooling. Both outputs are L2-normalized, so cosine_distance scores are comparable across rows. Switching a namespace between them requires a full re-index.

Images

With modality: image on a CLIP model, Layer embeds each attribute value with the image tower at write time and embeds Embed query text with the text tower, against the same vector column. Both towers run on CPU beside the gateway.

await client.write_namespace("photos", {
    "upsert_rows": [
        {"id": "commons-131", "title": "Sunset at Porto Covo",
         "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "image_url": {
            "type": "string",
            "embed": {
                "model": "openai/clip-vit-base-patch32",
                "dims": 512,
                "modality": "image",
                "serving": {"prefer": "local"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "photos", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "commons-131", "title": "Sunset at Porto Covo",
            "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "image_url": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":    "openai/clip-vit-base-patch32",
                "dims":     512,
                "modality": "image",
                "serving":  map[string]any{"prefer": "local"},
            },
        },
    },
})

await client.writeNamespace("photos", {
  upsert_rows: [
    { id: "commons-131", title: "Sunset at Porto Covo",
      image_url: "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" },
  ],
  distance_metric: "cosine_distance",
  schema: {
    image_url: {
      type: "string",
      embed: {
        model: "openai/clip-vit-base-patch32",
        dims: 512,
        modality: "image",
        serving: { prefer: "local" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "commons-131", "title": "Sunset at Porto Covo",
       "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "image_url": {
        "type": "string",
        "embed": {
          "model": "openai/clip-vit-base-patch32",
          "dims": 512,
          "modality": "image",
          "serving": { "prefer": "local" }
        }
      }
    }
  }'

Query the image column with text:

response = await client.query_namespace("photos", {
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"],
})

response, err := client.QueryNamespace(ctx, "photos", &hevlayer.QueryRequest{
    RankBy:            []any{"image_url", "ANN", []any{"Embed", "sunset over water"}},
    TopK:              16,
    IncludeAttributes: []string{"title", "image_url"},
})

const response = await client.queryNamespace("photos", {
  rank_by: ["image_url", "ANN", ["Embed", "sunset over water"]],
  top_k: 16,
  include_attributes: ["title", "image_url"],
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"]
  }'

{
  "rows": [
    { "id": "commons-131", "$dist": 0.412, "title": "Sunset at Porto Covo",
      "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" }
  ],
  "performance": {
    "embedding_tokens": 5,
    "embedding_ms": 134
  }
}

Write responses report embedding_images instead of embedding_tokens. CPU CLIP handles query-time text embeds and write-time image embeds for small and medium corpora. A bulk image backfill needs a GPU. The provider is set per namespace, so a backfill can run on a GPU while queries embed on CPU.

Declare the checkpoint for that backfill under open-weights models on your own GPU.

The lens demo runs this setup: Wikimedia Commons Quality images embedded and searched on CPU, with the gateway’s performance echo shown beside each result. Source at github.com/hev/lens.

Image input

An image profile accepts an HTTP(S) URL or a base64 image string and fetches at most 20 MiB per URL. A rate-limited image host returns HTTP 429 with error upstream_error; a server error from the image host returns HTTP 503 with error service_unavailable. Layer preserves the host’s Retry-After response header for both. Malformed URLs or base64, non-retryable 4xx responses, oversize content, and undecodable images return 422 validation_error.

Changing models requires a re-index

The two text models both emit 384 dimensions, so switching between them leaves the namespace schema valid. It still requires re-embedding every row.

Different models place vectors in unrelated spaces. If documents were embedded with MiniLM and queries are embedded with bge, the distances compare incompatible vectors and the ranking is meaningless. The same applies when moving from a CPU model to a hosted or GPU model, and when the weights, tokenizer, pooling, or prefixes behind a model id change: equal dimensions never make two vector spaces compatible.

Layer records the artifact fingerprint a local profile was declared with. A write that names a different model for an attribute that already has rows, or a write or Embed query after the service starts serving a different artifact for the pinned model, returns 422 and names the pinned model. Re-index from your source data into a fresh namespace so every vector comes from the new model, verify it, then switch your application to it.

Bringing your own weights

You can mount a model directory into the layer-embed service to add or override text checkpoints. Set LAYER_EMBED_MODELS_DIR on that service to a read-only bundle in the same manifest format as its baked models: a directory with a manifest.json listing each model’s files with their sizes and SHA-256 hashes. Mounted entries override baked entries by model id. The service verifies every listed file, loads and warms the whole menu, and only then reports ready; the gateway discovers the merged menu at startup. Only BERT text models with 384-dimensional CLS or mean pooling are accepted. The mount is optional and the bundled models need none. Overriding a model id with different weights changes its artifact fingerprint, so existing namespaces on that id need a full re-index after the restart.

Under Compose, mount the directory into the embed service read-only and set LAYER_EMBED_MODELS_DIR to its path in a Compose override file. The stock Compose file mounts nothing.

On Kubernetes, the Helm chart provisions a mounted bundle with the opt-in gateway.localModels values. An init container copies the bundle from a private S3 prefix, verifies manifest.json against a pinned SHA-256 before the pod starts, and mounts the directory read-only into the embedding sidecar; the sidecar then verifies each file against the manifest. A checksum mismatch fails the pod instead of serving unverified weights. The gateway container itself never downloads a checkpoint.

For a single static retriever that trades retrieval quality for microsecond embeds and a few megabytes of deployment size, see Lattice.

Open-weights models on your own GPU

To use a stronger retrieval model than the CPU menu offers, name any Hugging Face checkpoint in the schema and run it on a GPU worker in your own cluster. Compared with the CPU example, only model, dims, and serving.prefer change:

await client.write_namespace("articles", {
    "upsert_rows": [
        {"id": "planet-1", "text": "Jupiter is the biggest planet in the Solar System."},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "text": {
            "type": "string",
            "embed": {
                "model": "BAAI/bge-m3",
                "dims": 1024,
                "serving": {"prefer": "worker"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "articles", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "planet-1", "text": "Jupiter is the biggest planet in the Solar System."},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "text": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":   "BAAI/bge-m3",
                "dims":    1024,
                "serving": map[string]any{"prefer": "worker"},
            },
        },
    },
})

await client.writeNamespace("articles", {
  upsert_rows: [
    { id: "planet-1", text: "Jupiter is the biggest planet in the Solar System." },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      embed: {
        model: "BAAI/bge-m3",
        dims: 1024,
        serving: { prefer: "worker" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "planet-1", "text": "Jupiter is the biggest planet in the Solar System."}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "BAAI/bge-m3",
          "dims": 1024,
          "serving": { "prefer": "worker" }
        }
      }
    }
  }'

Queries are unchanged. Moving a namespace from a CPU model to a GPU checkpoint means editing those fields and re-indexing.

Declaring what the cluster serves

The cluster decides which checkpoints it serves, so they are declared once in InfraRules/default, next to the compute pools that run them:

apiVersion: hevlayer.com/v1alpha1
kind: InfraRules
metadata:
  name: default
spec:
  embedders:
    - name: bge-m3
      model: BAAI/bge-m3
      revision: 5617a9f61b028005a4858fdac845db406aefb181
      dims: 1024
      scaling:
        pool: gpu
        mode: autoscale
        warmWindowSeconds: 300
        replicas:
          min: 0
          max: 2

Field	Purpose
`model`	The Hugging Face repo id a schema may name. A checkpoint absent from this list is refused at write time.
`revision`	Pins the checkpoint. If omitted, the worker resolves the repo’s default branch once and records the revision it got.
`dims`	The dimension the worker emits. A schema declaring a different `embed.dims` fails validation.
`scaling`	The same workload scaling block Pipelines and Functions use. `pool` names a GPU compute pool and `max` may not exceed that pool’s `maxReplicasPerWorkload`.

Embedding workers scale to zero like any other GPU workload. Set warmWindowSeconds here for the same reason as on pipelines. A cold start provisions a node, pulls a multi-gigabyte image, and loads the model before it embeds anything, and a warm window lets consecutive batches reuse the same node.

Adding a checkpoint is an edit to this object and needs no gateway image rebuild or redeploy.

Errors

Condition	Response
Schema names a checkpoint no `InfraRules` embedder declares	`422 validation_error`, naming the requested model
Declared checkpoint whose weights will not load	`503 service_unavailable`, with the worker’s condition on the `InfraRules` status
No embedder declared at all, on a `worker` profile	`422 validation_error`
Worker pool at its replica ceiling	`429`, with `Retry-After`

An undeclared checkpoint is a configuration error you can fix, so Layer returns a 4xx that names the model. 503 means a declared model is unavailable.

Query with `Embed`

["Embed", text] computes a query vector from text using the model settings of the attribute being ranked. When rank_by names the source attribute, Layer reads the model from its schema:

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10,
})
print(response.rows)

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"text", "ANN", []any{"Embed", "chest pain radiating to left arm"}},
    TopK:   10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10
  }'

When rank_by names the derived vector attribute embed_<attr>, pass the model explicitly. Omitting it returns 422 with a model name must be provided.

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
        "model": "acme/clinical-retrieval-v3",
    }]],
    "top_k": 10,
})

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"embed_text", "ANN", []any{"Embed", "chest pain radiating to left arm",
        map[string]any{"model": "acme/clinical-retrieval-v3"}}},
    TopK: 10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
    model: "acme/clinical-retrieval-v3",
  }]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
      "model": "acme/clinical-retrieval-v3"
    }]],
    "top_k": 10
  }'

Query vectors are cached for 60 seconds by default. Set LAYER_EMBED_CACHE_TTL_MS to change the TTL. Instruction prefixes are part of the cache key. A missing provider returns 503 service_unavailable.

`Embed` with `Auto`

An inline Embed lets query routing run a semantic or fused query in one request:

response = await client.query_namespace("articles", {
    "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
        "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}],
    }],
    "top_k": 10,
})

response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{
    RankBy: []any{"title", "Auto", "how plants turn sunlight into food", map[string]any{
        "vector": []any{"Embed", "how plants turn sunlight into food",
            map[string]any{"field": "text"}},
    }},
    TopK: 10,
})

const response = await client.queryNamespace("articles", {
  rank_by: ["title", "Auto", "how plants turn sunlight into food", {
    vector: ["Embed", "how plants turn sunlight into food", { field: "text" }],
  }],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
      "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}]
    }],
    "top_k": 10
  }'

The first tuple field (title) is the attribute for the lexical legs. field inside Embed names the attribute whose schema supplies the embedding profile and vector column (text above). Omit field when both are the same attribute.

Layer chooses the route before it resolves Embed, so a short input routed to hybrid_text never calls the embedding provider. A semantic or fused response reports the routing decision (routing.policy: "v1", routing.executed: true) and merges embedding measurements into performance. Without a vector or an inline Embed, the router returns routing.executed: false and leaves embedding to the caller.

Model settings

The full embed block:

Field	Meaning
`model`	A provider-namespaced Hugging Face repo id.
`dims`	The vector dimension. Must match what the model emits.
`attribute`	Where the vector is stored. Defaults to `embed_<attr>`, and may not be the source attribute.
`serving.prefer`	`local`, `worker`, or `turbopuffer`. See where the vector comes from.
`modality`	`text` (the default) or `image`. `image` requires a CLIP-family model.
`revision`	Pins a checkpoint revision, including a finetuned one.
`instructions.document` / `instructions.query`	Prefixes for asymmetric retrieval models. Both affect the query-cache key.
`chunk`	Splits the attribute into one row per chunk on write. See chunking long documents.

The first write that establishes an embedded schema must also set distance_metric.

revision, instructions, and chunk are Layer extensions. Layer never forwards them upstream, and they require a GPU-served profile, except that a CLIP image profile may use local.

Chunking long documents

A single vector for a long document matches queries poorly, so long sources should be split into chunks and each chunk embedded as its own row.

Add embed.chunk to the schema and Layer splits the attribute on write. The split runs on the gateway’s CPU during the write request, and only the chunks go to the GPU worker for embedding. This profile splits text into windows of up to 512 tokens that overlap by 64:

await client.write_namespace("manuals", {
    "upsert_rows": [
        {"id": "pump-manual", "title": "Pump maintenance", "text": long_text},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "text": {
            "type": "string",
            "embed": {
                "model": "BAAI/bge-m3",
                "dims": 1024,
                "serving": {"prefer": "worker"},
                "chunk": {
                    "strategy": "recursive",
                    "unit": "tokens",
                    "tokenizer": "BAAI/bge-m3",
                    "size": 512,
                    "overlap": 64,
                },
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "manuals", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "pump-manual", "title": "Pump maintenance", "text": longText},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "text": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":   "BAAI/bge-m3",
                "dims":    1024,
                "serving": map[string]any{"prefer": "worker"},
                "chunk": map[string]any{
                    "strategy":  "recursive",
                    "unit":      "tokens",
                    "tokenizer": "BAAI/bge-m3",
                    "size":      512,
                    "overlap":   64,
                },
            },
        },
    },
})

await client.writeNamespace("manuals", {
  upsert_rows: [
    { id: "pump-manual", title: "Pump maintenance", text: longText },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      embed: {
        model: "BAAI/bge-m3",
        dims: 1024,
        serving: { prefer: "worker" },
        chunk: {
          strategy: "recursive",
          unit: "tokens",
          tokenizer: "BAAI/bge-m3",
          size: 512,
          overlap: 64,
        },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/manuals" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "pump-manual", "title": "Pump maintenance", "text": "..."}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "BAAI/bge-m3",
          "dims": 1024,
          "serving": { "prefer": "worker" },
          "chunk": {
            "strategy": "recursive",
            "unit": "tokens",
            "tokenizer": "BAAI/bge-m3",
            "size": 512,
            "overlap": 64
          }
        }
      }
    }
  }'

Layer keeps the original row and adds one row per chunk. A chunk row copies the original’s attributes, replaces the embedded attribute with the chunk text, and gets the id {id}#{n} plus two generated attributes: _hevlayer_parent_id (the original id) and _hevlayer_chunk_index (n). A query returns chunk rows, and _hevlayer_parent_id groups them back to their document. A row whose text fits in one chunk is written unchanged.

`embed.chunk` field	Meaning
`strategy`	`fixed`, `recursive`, `sentence`, `markdown`, `section`, or `none`.
`unit`	`characters` (the default) or `tokens`.
`size`	Maximum chunk length in `unit`s. Required unless `strategy` is `none`.
`overlap`	Units repeated between adjacent chunks. Must be smaller than `size`.
`tokenizer`	The Hugging Face tokenizer that counts tokens. Required when `unit: tokens`.

The section strategy splits a structured row by field before windowing. Set sectionSource: jsonFields and list the source attributes in fields, and each field becomes its own section. The chunk id becomes {id}#{field}#{n} and the field name is stored in sectionAttribute (default section). Put the window settings for each section in a nested split block.

Chunking has these limits:

The profile must use serving.prefer: worker.
A namespace can have one chunked embedding attribute.
Chunking cannot be combined with modality: image.
Writes must use upsert_rows with a string id on every row. Columnar upsert_columns writes are rejected.

Because the split runs inside the write request, a large document makes that write slower and uses gateway CPU that is also serving queries. For bulk ingest, or for sources that need extraction first (PDFs, scans), use a two-stage pipeline. Its CPU worker extracts and chunks outside the gateway and scales separately, and its GPU stage embeds the staged chunks. The pipeline takes the same chunk settings; see chunking.

Performance accounting

Write and query responses report embedding work under performance:

{
  "rows": [ /* ... */ ],
  "performance": {
    "embedding_tokens": 8,
    "embedding_ms": 42
  }
}

Queries omit embedding_tokens on a cache hit. Layer merges provider measurements into the same object and exports the work as hevlayer_embed_tokens_total and hevlayer_embed_compute_seconds_total, labeled by namespace, store kind, model, and serving mode.