Documentation

Build on unilayer

unilayer is an open-source, self-hosted backend layer. It puts your data and files behind one secure API, with access of its own for every app, so your applications talk to a single, consistent interface instead of to databases and buckets directly.

01 / Collections

Database tables you create and change at runtime through the API.

02 / Assets

Versioned files behind a storage service, on local disk or S3, with on-the-fly image transforms.

03 / Access

An API user per app or service, with a role, policies and fine-grained permissions.

How it fits together

Your apps call unilayer over REST or GraphQL. unilayer stores structured data in PostgreSQL and caches hot reads and token lookups in Valkey. File content goes through the storage service, which runs next to the API and keeps the files on the local filesystem or any S3-compatible store such as MinIO. The API hands out signed transfer orders, and clients move the bytes with the storage service, or straight with S3.

  Your apps  (Blazor · Web · Mobile · Services)
        │                         │
        │ REST · GraphQL          │ file content (signed URLs)
        ▼                         ▼
  unilayer API  ── jobs ──▶  storage service
        │                         │
   ┌────┴─────┐                   ▼
   ▼          ▼            Local disk / S3 (MinIO)
PostgreSQL  Valkey
5 minutes

Quick start

This gets a single unilayer instance running locally with Docker Compose – PostgreSQL, Valkey, the API and the storage service – then creates your first collection.

1

Get the compose file

Take compose.yaml from the root of the repository. It uses the published images code.talaryon.dev/unilayer/unilayer and code.talaryon.dev/unilayer/unilayer-storage.

2

Create the signing key and the secrets

The API signs the transfer orders of the storage service with an EC P-256 key. Next to compose.yaml, create the key and a .env file. Pick a long random string for the static API token: it becomes the root user's token.

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out storage-signing-key.pem

cat > .env <<EOF
POSTGRES_PASSWORD=change-me
UNILAYER_STATIC_TOKEN=my-root-token
EOF
3

Start unilayer

docker compose up -d

The API listens on http://localhost:5380, the storage service on http://localhost:5390. The database schema is migrated automatically on startup. Check that everything is up:

curl http://localhost:5380/health
4

Create a collection and add data

TOKEN=my-root-token

# Create a "tasks" collection – unilayer creates the table for you
curl -X POST http://localhost:5380/api/v1/collections \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"tasks","fields":[
        {"name":"title","type":"TEXT","nullable":false},
        {"name":"done","type":"BOOLEAN"}]}'

# Add an item
curl -X POST http://localhost:5380/api/v1/collections/tasks \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"title":"Try unilayer","done":false}'

# Read it back
curl -H "Authorization: Bearer $TOKEN" \
  "http://localhost:5380/api/v1/collections/tasks?filter=done:eq:false"
Explore further: open http://localhost:5380/api/v1 for the interactive REST reference, or /api/v1/graphql/ui for the GraphQL explorer.

Requirements

ComponentVersionPurpose
.NET SDK / runtime10.0Build and run the server (not needed when using the Docker image)
PostgreSQL14+System tables and all collection data
Valkey7.0+Cache and token lookups (Redis-compatible); a node is not ready without it
Storage servicesame releaseMoves and transforms file content (unilayer/storage, image unilayer-storage)
S3 / MinIOoptionalAsset storage; required in cluster mode, at least for snapshots

Installation

Backing services

Install PostgreSQL and Valkey with your platform's package manager, or run them as containers.

PostgreSQL

# macOS
brew install postgresql@16 && brew services start postgresql@16

# Ubuntu / Debian
sudo apt install postgresql postgresql-contrib
sudo systemctl enable --now postgresql

Then create a database and user:

CREATE DATABASE unilayer;
CREATE USER unilayer_user WITH PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE unilayer TO unilayer_user;

Valkey

# macOS
brew install valkey && brew services start valkey

# Ubuntu / Debian
sudo apt install valkey-server
sudo systemctl enable --now valkey-server

# Docker
docker run -d --name valkey -p 6379:6379 -v valkey-data:/data valkey/valkey

MinIO (optional)

docker run -d -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"

Run from source

The .NET client, the shared models and the storage service are submodules (libs/client, libs/storage), so clone with them:

git clone --recurse-submodules https://code.talaryon.dev/unilayer/unilayer
# or, in an existing clone
git submodule update --init

Create an appsettings.yaml (see Configuration) in the working directory, then:

dotnet run --project src/unilayer

The API listens on http://localhost:5380 by default. Start the storage service next to it and point it at the API's public keys:

dotnet run --project libs/storage/src/unilayer.Storage.Host -- \
  --storage:keys:url=http://localhost:5380/api/v1/storage/keys

Run with Docker

The published images are code.talaryon.dev/unilayer/unilayer and code.talaryon.dev/unilayer/unilayer-storage; both listen on port 8080. compose.yaml shows how they fit together. To build the API image yourself:

dotnet publish src/unilayer -c Release -o publish
docker build -t unilayer .

The image ships with pg_dump/pg_restore for cluster snapshots. They must be at least as new as your PostgreSQL server; the default is version 18. Override it with --build-arg PG_CLIENT_VERSION=<major>.

Configuration

unilayer reads its settings from three places:

  • YAML file: appsettings.yaml in the working directory is loaded automatically. Point to another file with --config=<path>.
  • Environment variables: every key has an UNILAYER_* equivalent, which suits containers.
  • Command-line arguments: e.g. --valkey:host=cache.local.

Need a YAML file from environment variables? scripts/generate-config.sh <output.yaml> builds one for you.

Example appsettings.yaml

endpoint: "0.0.0.0:5380"

authentication:
  staticApiToken: "your-secure-api-token"

database:
  providerType: "PostgreSQL"
  connectionString: "Host=localhost;Database=unilayer;Username=unilayer_user;Password=...;Port=5432"

valkey:
  host: "localhost"
  port: 6379
  password: ""
  database: 0

storage:
  provider: "service"      # or "local", "minio" / "s3", "none"
  service:
    url: "http://localhost:5390"
  signing:
    privateKeyFile: "./storage-signing-key.pem"

# s3:
#   endpoint: "http://localhost:9000"
#   accessKey: "minioadmin"
#   secretKey: "minioadmin"
#   bucketName: "unilayer-assets"

Core

KeyEnvironment variableDefaultDescription
endpointUNILAYER_ENDPOINT0.0.0.0:5380Address and port the API listens on
authentication:staticApiTokenUNILAYER_AUTHENTICATION_STATICAPITOKEN–Admin token, and the root user's token (see permissions). Plaintext or hashed
authentication:staticApiTokenSecondaryUNILAYER_AUTHENTICATION_STATICAPITOKENSECONDARY–A second static token accepted next to the first, for rotating it
tls:certificatePathUNILAYER_TLS_CERTIFICATEPATH–PKCS#12 certificate. When set, the API is served over HTTPS
tls:certificatePasswordUNILAYER_TLS_CERTIFICATEPASSWORD–Password of the certificate
tls:reloadIntervalSecondsUNILAYER_TLS_RELOADINTERVALSECONDS30How often the certificate files are checked for a renewal. 0: never, a renewal needs a restart
rateLimit:globalPerMinuteUNILAYER_RATELIMIT_GLOBALPERMINUTE1000Requests per client and minute (see Security)
rateLimit:writesPerMinuteUNILAYER_RATELIMIT_WRITESPERMINUTE50Writes per client and minute
idempotency:enabledUNILAYER_IDEMPOTENCY_ENABLEDtrueAccept the Idempotency-Key header (see Security)
idempotency:retentionHoursUNILAYER_IDEMPOTENCY_RETENTIONHOURS24How long keys are remembered
idempotency:maxResponseBytesUNILAYER_IDEMPOTENCY_MAXRESPONSEBYTES64 KBLarger responses are replayed with their status only
compression:enabledUNILAYER_COMPRESSION_ENABLEDtrueBrotli/gzip for JSON and other text responses. Turn it off when a reverse proxy compresses
compression:levelUNILAYER_COMPRESSION_LEVELfastestfastest, optimal or smallest
compression:mimeTypesUNILAYER_COMPRESSION_MIMETYPEStext-like typesComma-separated MIME types that replace the default list
metrics:endpointUNILAYER_METRICS_ENDPOINT–A separate listener for /metrics, e.g. 0.0.0.0:9464 (see Operations)

Database

KeyEnvironment variableDefaultDescription
database:providerTypeUNILAYER_DATABASE_PROVIDERTYPEPostgreSQLDatabase provider
database:connectionStringUNILAYER_DATABASE_CONNECTIONSTRING–Npgsql connection string

Valkey

KeyEnvironment variableDefault
valkey:hostUNILAYER_VALKEY_HOSTlocalhost
valkey:portUNILAYER_VALKEY_PORT6379
valkey:passwordUNILAYER_VALKEY_PASSWORD–
valkey:databaseUNILAYER_VALKEY_DATABASE0
valkey:sslUNILAYER_VALKEY_SSLfalse
valkey:connectTimeoutUNILAYER_VALKEY_CONNECTTIMEOUT5000 ms
valkey:syncTimeoutUNILAYER_VALKEY_SYNCTIMEOUT5000 ms

Storage

KeyEnvironment variableDefaultDescription
storage:providerUNILAYER_STORAGE_PROVIDERlocalservice (only the storage service reaches the files), local, minio / s3 (any S3-compatible store) or none (no assets)
storage:quotaBytesUNILAYER_STORAGE_QUOTABYTES0The most all assets may take together; 0 for no limit. Beyond it uploads get 507
local:pathUNILAYER_LOCAL_PATH./storage/assetsBase directory for local storage
s3:endpointUNILAYER_S3_ENDPOINT–S3 / MinIO endpoint URL
s3:accessKeyUNILAYER_S3_ACCESSKEY–Access key
s3:secretKeyUNILAYER_S3_SECRETKEY–Secret key
s3:bucketNameUNILAYER_S3_BUCKETNAMEunilayer-assetsBucket name
s3:regionUNILAYER_S3_REGIONus-east-1Region
s3:basePathUNILAYER_S3_BASEPATHassetsKey prefix inside the bucket
s3:useSslUNILAYER_S3_USESSLfalseUse HTTPS towards the store
s3:createBucketIfNotExistsUNILAYER_S3_CREATEBUCKETIFNOTEXISTStrueCreate the bucket on startup
s3:publicUrlBaseUNILAYER_S3_PUBLICURLBASE–Public base URL for asset links

The storage service reads the same local:path and s3:* keys, so both reach the same files. The settings of the storage service, transfers and signing keys are in Uploads & downloads; file types, image transforms, optimization and retention in Assets; GraphQL and cluster settings in GraphQL and Cluster mode.

Concepts

Collections

A collection is a real PostgreSQL table that you define through the API. Creating, changing or deleting a collection changes the table at runtime; there are no migrations to write. Every collection has explicitly typed fields.

Field types

TypeAliasesHolds
TEXTSTRING, VARCHAR, CHAR, NCHAR, NVARCHARText. Returned exactly as stored, even if it looks like JSON
INTEGERINT, SMALLINT, BIGINT, LONGWhole numbers (stored as BIGINT)
REALFLOAT, DOUBLE, DECIMAL, NUMERIC, NUMBERExact decimal numbers (stored as NUMERIC)
BOOLEANBITTrue / false
DATETIMETIMESTAMPA point in time, stored with time zone and returned in UTC
DATE–A calendar date
TIME–A time of day
UUID–A UUID; the type of every reference
BYTESBYTEA, BLOBBinary data
JSONJSONBA JSON object or array, returned as JSON

Fields can also be nullable or required, have a default value, be unique, and be indexed (isIndexed) for fast lookups and filters. Field names are checked against PostgreSQL reserved words.

System fields

unilayer manages four fields on every collection. The unilayer_ prefix is reserved, so you can't define fields that start with it.

FieldTypeMeaning
unilayer_idUUIDPrimary key, a time-ordered UUIDv7
unilayer_indexBIGSERIALInsertion order, the default sort
unilayer_created_atTIMESTAMPCreation time, in UTC
unilayer_updated_atTIMESTAMPLast update in UTC, or null

Responses keep these separate from your data: your fields are in data and system values are in metadata, so names can never collide.

Relationships

Mark a UUID field with isReference: true and a referenceCollection to create a foreign key. A reference always points at the target's unilayer_id; with cascadeDelete: true, deleting the target deletes the referencing items too. When you list items, referenced records are expanded and nested automatically, up to 3 levels deep. References into collections the caller can't read stay plain ids, and reading a single item by its id doesn't expand them.

{
  "name": "users",
  "fields": [
    { "name": "username", "type": "TEXT" },
    { "name": "profileImage", "type": "UUID",
      "isReference": true, "referenceCollection": "images" }
  ]
}

Bulk writes and counts

Up to 1,000 items can be added, updated or deleted in one request, in one transaction: POST /collections/{name}/bulk, PUT /collections/{name}/bulk and POST /collections/{name}/bulk/delete. GET /collections/{name}/count?filter=… counts the matching items. See the API reference.

Schema history

Every schema change is written to a system history. You can export this history and replay it on another installation to move a collection structure between sites. See Operations.

Querying

The same query language works on every collection: filter, sort, paginate and pick fields using query parameters.

ParameterExampleMeaning
filterage:gt:18,status:eq:activeConditions: , means AND, | means OR
sortname,unilayer_created_at:descOne or more fields, including the system fields, :asc / :desc
limit20Page size, 1–1000 (default 100)
offset40Items to skip
fieldsname,profileImage.urlField selection. A dotted path expands a reference; system values are always in metadata

Filter operators

Filters use the form field:operator:value.

OperatorMeaningExample
eq / neEquals / not equalsstatus:ne:deleted
gt / ge / gteGreater than (or equal)price:gte:100
lt / le / lteLess than (or equal)age:lt:65
likePattern matchname:like:%John%
inValue in liststatus:in:active\,pending
isnull / notnullNull checksdeletedAt:isnull:

A backslash escapes ,, | and : in values, so the values of in are separated by \, – a plain comma starts the next condition. like compares the field as text. Points in time are compared as instants; a value without an offset is UTC.

Values are parsed according to the field's type and always sent to the database as parameters; a value that doesn't parse gives 400. Field names and operators are checked against the schema and an allow-list, so filters are safe from SQL injection.

Assets

Assets are files with metadata. Each asset has a UID, name, file type, MIME type, size, a SHA-256 checksum and a version number that increases every time the content is replaced. You can attach custom metadata as JSON. Deleting an asset is a soft delete, so it can be restored (assets:write) until it is purged.

File types

An asset's type comes from its file extension:

TypeExtensions
Documentpdf, docx, pptx
Imagepng, jpg, jpeg, gif, svg, webp, tif, tiff, heic
Videomp4, mov
Audiom4a, mp3, ogg, opus, wav
Texttxt or no extension
Spreadsheetxlsx, csv
Archivezip, stored as it is
OtherExtensions allowed through assets:allowedExtensions that fit no other type

Other extensions are refused; executables and scripts always are. The first bytes of an upload must fit its extension (a .png must start like a PNG), otherwise it is refused with 400. The default upload limit through the API is 100 MB per file.

KeyDefaultDescription
assets:allowedExtensions–Extensions allowed in addition, e.g. dwg,step
assets:verifyContenttrueWhether uploads must match their extension

Image transforms

Images can be resized and re-encoded on download by adding query parameters to the asset URL, so there is no need to store separate thumbnails.

/api/v1/assets/{uid}?width=200&format=webp
/api/v1/assets/{uid}?width=300&height=300&fit=cover&quality=70
/api/v1/assets/{uid}?scale=0.3
ParameterValues
width, heightTarget size in px. Aspect ratio is kept when only one is set
scaleMultiplier, e.g. 0.3. Can't be combined with width/height
fitcontain (default), cover, stretch
formatwebp, png, jpg, gif
quality1–100 (default 80), for jpg and webp
  • EXIF, XMP, IPTC and ICC data (such as GPS positions) is stripped from transformed images.
  • Variants up to 1 MiB are cached per asset version, so uploading new content invalidates them automatically. Responses support ETag / 304.
  • Outputs are limited to 4096 px per side and sources to 50 megapixels. SVGs are returned unchanged.
  • Transforms run in a limited number of slots per node; a request that waits too long gets 503 with Retry-After. The limits are settings under assets:transform:*.

Optimized images

With assets:optimize:enabled, JPEG, PNG and GIF assets get a lossless optimized copy in the background: the same pixels and metadata, typically 5–30 % smaller. Downloads without modifiers serve the copy; ?original=true always returns the stored file, which is what Size and Checksum describe. Every copy is verified against its original before it is kept. Run the server once with --remove-optimized-assets to remove the copies again.

Compression

Text-like downloads (text, CSV, JSON, XML, SVG, …) are compressed with Brotli or gzip when the client accepts it. Images, video, archives and office documents are sent as stored.

Purge and retention

DELETE /api/v1/assets/{uid}?purge=true removes a deleted asset for good: its content, copies, variants and record. It needs assets:purge, and the asset must have been deleted long enough ago that no kept backup still holds its record.

KeyDefaultDescription
assets:retention:minAgeDays30How long an asset stays deleted before it may be purged
assets:retention:purgeBefore–Only assets deleted before this time may be purged, e.g. your oldest kept backup
assets:retention:purgeAfterDays0Purge deleted assets automatically after this many days; 0 turns the job off
assets:retention:intervalHours24How often the retention job runs

Uploads & downloads

With the storage service running next to the API, file content no longer passes through the API and isn't base64-encoded. The API hands out a transfer order – a URL with a signed token – and the client moves the bytes with the storage service. The .NET client does all of this for you (UploadAsync, ResumeUploadAsync, GetDownloadUrlAsync).

Upload

  1. Create the asset with POST /api/v1/assets.
  2. POST /api/v1/assets/{uid}/upload-session with the file's size and sha256. The type, size and quota are checked here. The answer has a session and a transfer order (URL, method, headers, expiry).
  3. Send the file to the order's URL. From 16 MiB the order comes in parts: upload them in any order and in parallel, then post the list of parts with their ETags.
  4. POST /api/v1/assets/{uid}/upload-session/complete with the session. The API checks the file and returns the metadata with the new version.

An order works for 15 minutes. …/upload-session/renew hands out a new one for the same upload, keeping the parts already stored, so long and interrupted uploads can be resumed. Uploads that are never completed are cleaned up after a day.

Download

GET /api/v1/assets/{uid}/download-url returns a GET order for the content, the optimized copy (unless original=true) or, with image modifiers, a variant. The URL works without a token until it expires and is stable within a time window, so browsers and CDNs can cache it. The storage service answers Range requests.

Quotas

Besides the global storage:quotaBytes, an upload session may carry a quotaScope (e.g. a workspace id) and your application's quotaLimitBytes for it. An upload that would take the scope over its limit is refused with 507. Open uploads count with their announced size, so parallel uploads can't overbook. GET /api/v1/assets/quota?scope=… tells what a scope takes.

Straight to S3

With storage:transfer:mode=direct and an S3 store your clients can reach, the API hands out presigned S3 URLs instead, and content goes between the client and S3 directly. The orders look the same, so clients don't change. The bucket needs CORS for browser clients and a lifecycle rule that aborts incomplete multipart uploads.

Settings of the API

KeyDefaultDescription
storage:service:url–The storage service as clients reach it; turns the endpoints on (otherwise they answer 501)
storage:service:internalUrlstorage:service:urlWhere the API itself reaches the service, e.g. http://storage:8080
storage:signing:privateKey / privateKeyFile–The ES256 key (EC P-256, PEM) orders are signed with; the same on every node
storage:signing:keyIdunilayer-1Change it together with the key
storage:signing:standbyPrivateKey, …File, standbyKeyId–A second key during a rotation: published and accepted, never signed with
storage:signing:issuerunilayerThe name the storage service knows this instance by
storage:transfer:modegatewaydirect: straight to S3 with presigned URLs
storage:transfer:directEndpoints3:endpointThe S3 store as clients reach it
storage:transfer:apiUrlthe request'sThe API as clients reach it, behind a proxy or in a cluster
storage:transfer:uploadExpiryMinutes15How long an upload order works
storage:transfer:downloadWindowMinutes60Window in which a download URL stays the same
storage:multipart:thresholdBytes / partSizeBytes16 MiBFrom which size uploads go in parts, and how large a part is
storage:legacy:sunset–Date sent as the Sunset header of the deprecated content endpoints

Environment variables follow the usual pattern, e.g. UNILAYER_STORAGE_SERVICE_URL. To rotate the signing key without downtime: set the new key as standby, swap the two (key ids included), and remove the standby key after a day.

Settings of the storage service

KeyDefaultDescription
storage:providerlocallocal (one node only) or s3 / minio, with the same local:path or s3:* as the API
storage:keys:url–The API's public keys, GET /api/v1/storage/keys
storage:cors:originsnoneWeb origins allowed to call it; * for any
storage:valkey:configurationnoneA Valkey for the job queue and variant cache, e.g. the instance's
storage:maxUploadBytes10 GiBThe largest upload, whatever a token allows
storage:modesinglecluster for several storage nodes behind a load balancer (S3 and a Valkey of its own required)
The old content endpoints – PUT /api/v1/assets/{uid} with base64 content, and the content of GET /api/v1/assets/{uid} – still work, but are deprecated once the storage service is on. They answer with Deprecation and Link headers pointing at their successors.

Users & permissions

Every request is authenticated with a bearer token:

Authorization: Bearer <token>

The model

Policy

A named list of permission strings.

Role

Bundles one or more policies.

User

Has exactly one role and its own API token.

A user is a client of the API – an app, a service, a script –, not an end user of your app: there is no sign-up, login or password. Give every app a user of its own, e.g. one that writes for your admin UI and one that only reads for your website. Signing in the people who use your app stays with your app or your identity provider.

Permission strings

Permissions follow the form resource[:scope[:name]]:action. A write permission includes read.

PermissionGrants
collections:read / :writeAll collections
collections:{scope}:readCollections with a scope prefix
collections:{scope}:{name}:writeOne specific collection
assets:read / :writeAssets
assets:purgeRemove deleted assets for good
users:read / :writeUser management
security:read / :writeRoles and policies
cluster:adminCluster membership and status

Nobody can grant more than they hold: creating or changing a policy, assigning a policy to a role, or giving a user a role requires the caller to hold every permission involved.

First start

On first startup unilayer creates three immutable entities:

  • fullaccess policy, which contains every permission. Permissions added by later versions are added to it at the next start.
  • admin role, which is assigned the fullaccess policy.
  • root user, which has the admin role and uses authentication:staticApiToken as its token.
While authentication:staticApiToken is set, it authenticates as an admin, and the root user is keyed to it at every start. Only its hash is stored in the database. Use it to create real users, then keep it somewhere safe. To replace it without downtime, set the new token as authentication:staticApiTokenSecondary, switch your clients, then make it the primary token.
Integrations

GraphQL

unilayer serves a GraphQL API next to REST. The schema is generated from your collections and rebuilt automatically when a collection or field changes. Permissions and write paths are the same as for REST.

  • Endpoint: POST /api/v1/graphql. Authentication is required, including for introspection.
  • Explorer: /api/v1/graphql/ui, served when both enableUi and enableIntrospection are on. Add your Authorization header in its headers panel.
  • SDL download: GET /api/v1/graphql/schema, for codegen.
query {
  postsList(where: [{ field: views, op: GTE, value: "5" }],
            orderBy: [{ field: title }], limit: 10) {
    items { id title views createdAt }
    hasMore
  }
}

mutation {
  createPosts(input: { title: "Hello" }) { id }
}
KeyDefaultDescription
graphql:enabledtrueTurn GraphQL on or off
graphql:path/api/v1/graphqlEndpoint path
graphql:enableUitrueServe the explorer
graphql:enableIntrospectiontrueAllow schema introspection
graphql:maxQueryDepth10Maximum query depth
graphql:maxQueryComplexity5000Maximum query complexity
graphql:defaultPageSize / maxPageSize20 / 100Paging
graphql:collectionNamePatterns*Collections to expose (wildcards)
graphql:excludedCollectionssystem_*Collections to hide (wildcards)

Environment variables follow the usual pattern, e.g. UNILAYER_GRAPHQL_ENABLED. Mutations count against the same write rate limit as REST.

Collection and field names are visible through introspection to every authenticated user. If that matters, disable introspection. Subscriptions, upserts and total counts are not supported yet.

.NET client

Unilayer.Client is a typed client for .NET 10, MIT-licensed and developed in its own repository. It maps your classes to collections with attributes and turns LINQ into server-side queries. The wire-format models ship as Unilayer.Models, released with the same version.

1. Install and register

dotnet add package Unilayer.Client
builder.Services.AddUnilayerClient(options =>
{
    options.Url   = "https://unilayer.example.com";
    options.Token = builder.Configuration["Unilayer:Token"];
});

// Several backends? Register named clients and use IUnilayerClientFactory.
builder.Services.AddUnilayerClient("Archive", options => { /* ... */ });

2. Describe your model

[UniCollection("products")]
public class Product
{
    [UniField("name", IsRequired = true)]
    public string Name { get; set; } = "";

    [UniField("sku", IsUnique = true)]
    public string Sku { get; set; } = "";

    [UniField("price", IsIndexed = true)]
    public decimal Price { get; set; }
}

3. Sync and query

await client.SyncCollectionSchemaAsync<Product>();   // creates / updates the collection

await client.CreateAsync(new Product { Name = "Widget", Sku = "W-1", Price = 9.90m });

var cheap = await client.Query<Product>()
    .Where(p => p.Price < 20)
    .OrderBy(p => p.Name)
    .Take(50)
    .ToListAsync();

Bulk writes and files

BulkCreateAsync, BulkUpdateAsync and BulkDeleteAsync write up to 1,000 items in one transaction. UploadAsync uploads through the storage service, in parallel parts for large files, and ResumeUploadAsync continues an interrupted upload. GetDownloadUrlAsync returns a download URL, optionally for an image variant or the original. Responses are compressed.

Field encryption

Mark a property with IsEncrypted = true and set FieldEncryptionMasterKeyBase64 (32 random bytes, base64-encoded). Values are encrypted with AES-256-GCM in the client, so the server only ever stores ciphertext.

Encrypted fields can't be filtered or sorted in a meaningful way. Keep the master key stable and out of source control: if you lose it, or rename the collection or property, existing values can no longer be decrypted.

Built-in resilience

The client retries with exponential backoff (3 retries by default), has a circuit breaker and uses a 30 s timeout. All of these can be configured on UnilayerClientOptions. It also supports schema versioning and migrations for evolving models.

Production

Security

  • HTTPS: set tls:certificatePath and the API listener serves HTTPS only. In cluster mode this is mandatory. Renewed certificate files are picked up without a restart.
  • Tokens: only hashes of API tokens are stored. Newly issued tokens are never kept for idempotent replays.
  • Rate limiting per client: 1000 requests/min globally and 50/min for writes (every write endpoint and GraphQL mutations), both configurable. A request over the limit gets 429 with Retry-After at once. Failed token attempts are throttled separately, at 30 per client and minute.
  • Safe retries: send an Idempotency-Key header with a write, and a retry after a timeout or a 503 gets the first response again (Idempotent-Replay: true) instead of being applied twice.
  • Least privilege: give each app its own user and a role that is scoped down to the collections it needs.
  • Sensitive data: use client-side field encryption for values the server should never see in plain text.

Cluster mode

For high availability, run several unilayer nodes. Each node has its own PostgreSQL, Valkey and WAL directory, and all nodes share one S3 store. The nodes elect a leader with Raft, and writes are replicated through the Raft log to every node. A write sent to a follower is forwarded to the leader, and reads are served locally.

Requirements

A node refuses to start unless all of these are true:

  • storage:provider is service, minio or s3, and s3:* is set: snapshots (and, without the storage service, the assets) live in the shared store.
  • The public API has a TLS certificate (tls:certificatePath).
  • Node-to-node traffic uses mutual TLS with a private CA.

Certificates

CERT_PASSWORD=secret ./scripts/generate-cluster-certs.sh ./certs node1 node2 node3

Give each node its own .pfx and ca.crt, and keep ca.key off the nodes. Certificate files are reloaded in place, so they can be renewed without a restart. When a certificate has less than 30 days left, /health reports Degraded so you can alert on it in time.

Settings

KeyDefaultDescription
cluster:enabledfalseTurn cluster mode on
cluster:nodeId–Unique node name
cluster:bindEndpoint–This node's address. Must be reachable (not 0.0.0.0). A node that isn't in peers joins later
cluster:peers–The members a node starts with the first time, as a YAML list or comma-separated. Use either all IPs or all hostnames
cluster:walDirectory–Raft write-ahead log directory
cluster:internalApiPort–Internal mTLS-only port, the same on every node
cluster:tlsCertificatePath–Node .pfx (serverAuth + clientAuth)
cluster:tlsCertificatePassword–Password of the .pfx
cluster:tlsCaCertificatePath–CA public certificate
cluster:valkeyLocalEndpoint–This node's Valkey, which switches to primary or replica with leadership
cluster:lowerElectionTimeoutMs / upper…500 / 1000Election timeouts
cluster:connectTimeoutMshalf of lowerMust be shorter than the lower election timeout
cluster:snapshotIntervalMinutes1440pg_dump snapshots to S3. 0 disables them
cluster:snapshotPrefixcluster-snapshots/Key prefix of the snapshots
cluster:initialSnapshottrueA leader without a snapshot takes one right away
cluster:requestTimeoutMs60000How long a forwarded write may take
cluster:readyMaxLag100A node further behind than this many entries is not ready
cluster:valkeyPort6379Port of the nodes' Valkeys

Environment variables follow the usual pattern, e.g. UNILAYER_CLUSTER_PEERS=10.0.0.1:7000,10.0.0.2:7000,10.0.0.3:7000.

Members

cluster:peers is only the list a node starts with the first time. From then on the member list is part of the replicated log and changes only through the membership API, which needs cluster:admin:

CallDoes
GET /api/v1/cluster/statusRole, leader, term, log indexes, snapshot and members
GET /api/v1/cluster/membersThe members
POST /api/v1/cluster/membersAdds a node ({"endpoint":"host:port"}); it is brought up to date from a snapshot
DELETE /api/v1/cluster/members/{host:port}Removes a node
POST /api/v1/cluster/resignThe leader gives up leadership, e.g. before restarting it

To add a node, start it with cluster:peers set to the current members, without itself, then add it through the API. A node that lost its disks just starts again under the same address and is restored from the latest snapshot.

Storage service

Every node uses the storage service the same way, with the same signing key, and keeps s3:* for its snapshots. Only the leader sends jobs to the service. For high availability, run the storage service as a cluster of its own: storage:mode=cluster, S3 and a Valkey of its own on every storage node.

Consistency

  • A write is acknowledged once a majority has it and the leader has applied it.
  • Writes are checked before they are replicated, and a write that fails when the leader applies it fails the request, as it would on a single node.
  • Writes are fenced by leadership. A non-leader rejects writes with 503. After a 503 caused by lost leadership, the write's outcome is unknown: send writes with an Idempotency-Key and retry with the same key. The key travels through the log, so the write is never applied twice.
  • Reads on a follower can lag by about one heartbeat (~250 ms by default). To read your own write, read from the leader.
If your Valkey instances use a password, also set masterauth to that password on every instance so replicas can follow the current primary.

Operations

Health checks

PathUse
/healthOverall status with all checks: database, Valkey, storage or storage service, and cluster
/health/liveLiveness probe (database)
/health/readyReadiness probe: database, Valkey, storage service and, in cluster mode, whether the node has joined, knows a leader and has caught up

Metrics

With metrics:endpoint (e.g. 0.0.0.0:9464) the server opens a second listener that serves /metrics in the Prometheus format: requests and their duration, database and asset sizes, image optimization, and in cluster mode role, leader, log indexes, replication lag, snapshots and certificate expiry. It is plain HTTP without authentication, so keep it inside your monitoring network.

API reference

Every instance serves an interactive REST reference at /api/v1, and the OpenAPI document at /api/v1/openapi/{document}.json.

Moving schemas between sites

Export the collection schema history from one installation and replay it against another installation's API:

# On the source (same configuration as the running instance)
dotnet unilayer.dll --export-history-file=schema.jsonl

# Replay against the target
dotnet unilayer.dll --import-history-file=schema.jsonl \
  --import-target-url=https://target.example.com \
  --import-target-token=$TARGET_TOKEN

Use --export-from-sequence=<n> and --import-from-sequence=<n> to do this incrementally or to resume a replay.

Maintenance commands

--truncate-all drops the tables of all collections with their metadata and cache – users, permissions and history stay. --reinitialize-database drops every table. Both destroy data, refuse to run in cluster mode, run once and then exit. Use them only on development instances or with a backup.

--remove-optimized-assets removes the optimized copies of images, e.g. after turning optimization off.